Information Governance Benefits Big Data Initiatives

An article of mine was just published in the Opinion section of Computerworld regarding the importance of information governance to Big Data and analytics. Just wanted to share on the blog since I’ve not had a chance to post recently. Working on a future post to discuss how object storage is a key component of the information infrastructure. Stay tuned . . .

W/out proper information governance, enterprises likely to lose out on valuable insights http://t.co/fGDsPOrHvV @Computerworld #infogov

— CGOC (@CGOC_Council) April 21, 2015

Storage Growth: A Symptom of Inadequate Information Governance

<Syndicated from my CGOC blog>

Explosion, tsunami and avalanche are adjectives that have all been used over the past decade to describe the amount of new data being created. The Petabyte (1,000,000 Gigabytes) has become the common order of magnitude used to describe the amount of storage capacity many enterprises are approaching or already have under management. The increasing costs associated with the technology and resources necessary to store and protect data at this magnitude is burdening organizations across a myriad of industries. IT departments are in a frantic search to find ways to reduce cost and capacity by moving data to the cloud and/or analyzing legacy data to determine how much of it can be deleted.

Cloud and Information Governance are Not Mutually Exclusive

<Syndicated from my CGOC blog>

Overuse is a word that people often think of when the term cloud is mentioned. Over the past several years it has been thrown around by IT and business practitioners, as well as vendors, as though the cloud holds all the answers. Today, we see more pragmatism about the benefits of the cloud and where it can be best put to use. The term not often mentioned or associated with cloud is information governance. This is especially true when discussing public cloud services; however, the private cloud is proving to deliver some very unique information governance capabilities. An interesting example can be found in how the NSA is using its private cloud.

An article published by NetworkWorld, Exclusive: Inside the NSA’s Private Cloud, describes how the federal agency is leveraging the private cloud to improve costs, efficiency and even information governance.

Information and Data: Virtual Assets with Tangible Future Benefits

(Syndicated from my CGOC blog)

The theme for a recent CGOC executive meeting held in New York had the theme of treating information as an asset that needs to be managed based on its value. When it comes to information governance, many of the attendees at CGOC events are practitioners in industries such as financial services & banking, oil & gas, insurance, manufacturing and the like. Perceiving and managing information/data as an asset isn’t necessarily natural for such organizations, especially when it comes to historical or archived elements. That’s beginning to change with the emergence of Big Data and analytics, and it’s worth taking a brief look at an industry that has always approached its information and data as assets, media & entertainment.

Dark Data: Time to Take Action Now

<Syndicated from my CGOC blog>

Over the next two years, nearly 60% of professionals are making analytic strategies and tools top priorities according to Forrester Research’s “The State of Digital Customer Experience.” And that’s not surprising considering organizations across all industries now want to make sure they are extracting maximum value from their data. But in order to get the maximum benefit from analytics, organizations must finally address their “dark data”. So what exactly is dark data?

Information Tai Chi

One of the objectives of tai chi as a martial art for self defense is to gently absorb the force of an aggressor and guide the energy away or turn it against him.  It could also be stated as using that energy to benefit the tai chi master.  The massive number of constant data streams our senses are attacked with every day (individually and organizationally), which I described in a previous post (Driven to Distraction: Insatiable Information Input), is demonstrating the need for a similar skill.  It is the ability to effortlessly absorb the information, determine what should be guided away and what has value (energy) and can be used for our benefit.

It’s Just Metadata (Updated)

That’s the only thing being collected says the NSA, just metadata. It’s not the “actual” data so there’s no need to worry. We’re not listening to every phone call or reading every e-mail just getting tiny, almost imperceptible bits of your data. Of course, I’m paraphrasing here for brevity. Regardless of your political views (left/right/indifferent) there is certainly reason enough to be concerned. Metadata is critical for managing and tracking information. It is the “key to the kingdom” as I indicated in a post here quite some time ago albeit in a much different context. However, when I heard this is what the government had said I started thinking about the various ways metadata could be used with potentially harmful results, intended or otherwise. I would venture to wager that the vast majority of people outside of high-tech don’t know what metadata is or the utility value it presents.

Example of TV metadata enabling rich/complex relationships

Some may know terms such as “poll data”, “raw data”, “ticker data”, and “Data” the dude from Star Trek Next Generation with the pasty complexion, but not metadata. Data about data? It is only within the past few years that we’ve started hearing more about metadata and its usefulness to Big Data and analytics that are all the rage. Take the massive amount of data being generated, ingest it to some behemoth data center and then churn on it with massive compute farms to glean hidden or unseen relationships and identify the “diamond in the rough” or the “bad guy out for no good” (NSA focuses on the latter).

Driven to Distraction: Insatiable Information Input

Perhaps the FIFO (first-in-first-out) method applies to one’s ability to retain information, or lose it with the constant bombardment of ever more coming in all the time. Maybe the sitcom “Married With Children” was prescient in the episode where Al fills Kelly’s head with sports facts and old trivial information that was in there is jettisoned, or first out.

In all seriousness though, there was a recent report on one of the cable news networks about the increase in ADHD diagnosed in children over the past ten years.  It referred to an article in the New York Times indicating that a CDC study showed the rate of kids between the ages of 4 and 17 being diagnosed with ADHD has jumped 41% over the past ten years.  This started me wondering what the impact of the always-on, always-connected, content-everywhere lifestyle might be as a contributing factor.  There is a near-infinite number of information streams flowing continuously 24 hours a day.

Begin With the End in Mind

Having a clear vision of where you want to go provides the context to define the path that will get you there.  The path is more a navigational tool for a journey rather than a concrete highway to get from start to finish.  Obstacles and barriers always present themselves.  Having the clarity to see beyond them allows the traveler to effectively navigate around and through while keeping line of site to the finish line.  Begin with the end in mind is the second of “The 7 Habits of Highly Effective People”  presented by author Stephen Covey in his book and one , which had a significant resonance for me.  In addition to an individual work habit, it is an important guiding principle when designing a solution and strategy for effectively managing the information life cycle.

This was highlighted recently when a new process was announced at my previous employer that would enforce rules for end users to classify the business value of any e-mail or file (document, spreadsheet, presentation) they created and saved to a company resource.  “Finally,” I thought, “we just might see more emphasis placed on managing information and starting at the source.”  That’s right.  You.  The person between the chair and the keyboard.  It’s a big company implementing this process with around 100,000 employees worldwide (give or take…mostly take) and it is no small undertaking.

Waves Required for Surfing the Web

In an earlier post I made the statement that the Web is just one massive content storage repository and how this aligns with the concept of the cloud.  The intent here is to expand on that concept further with respect to the file system.  The dominant paradigm over the past 30 plus years for organizing, structuring and managing computer generated information artifacts stored digitally as files.  Content or unstructured data in the form of blogs, tweets, social networking and pinteresting things has provided users new and more dynamic methods of storing, managing and sharing stuff.  Surfing the Web is the new common paradigm and you can’t surf without waves (or WAVS).

The file system is the most familiar interface to storage for end users, applications and developers.  The introduction of NAS for presenting a network accessible file system for shared storage extended the capabilities of file system connectivity.  However, we’re seeing these technologies reaching their breaking point as the number of files and amount of capacity are exceeding the expectations of the original architecture.