Big Data: How to Ensure Your Organization Is Ready

Published On November 09, 2017

A fast growing trend in today’s economy is big data business intelligence. Everyone wants it, even if they may not know exactly what it is. Sadly, many organizations risk failure by rushing into a business intelligence project, purchasing software with the hopes of it solving all their problems before doing the due diligence to create a good data set. This is the age-old IT axiom of “garbage in, garbage out” rearing its head once again.

Big data refers to large data sets that, when analyzed, reveal patterns, trends and associations that can be valuable to your organization’s bottom line. The trick is to create a “data lake” of trusted information, which your organization can then use business intelligence platforms to “go fishing” in. Creating that data lake requires information governance (IG), and it’s an ongoing process.

IG begins with understanding the Four Ws:

• What information do you have?
• Where is it located?
• When can you get rid of it?
• Who owns it, maintains it and needs access to it?

Your organization has to do the hard work of ridding itself of redundant, obsolete and transitory information (ROT), both hard copy and electronic, structured and unstructured. All organizations have ROT to one degree or another, taking up storage space and making it difficult to find the real information you want. It takes the active participation of everyone from records management, legal and IT across your organization to make this happen and to maintain it over the long term.

The next step is to build a data lake with your cleansed and normalized data. There are several important infrastructure considerations to keep in mind, such as the amount of local or cloud storage capacity. If there are plans to move to the cloud, how much bandwidth is required to use it? Are all of the systems of record at the current version? And is there a master data management system in place? What is the plan for the manpower needed to maintain these systems? These are just a few of the critical points to building a data lake.

With the data lake built, it’s time to go fishing! The good news is that today’s business intelligence software offers many user-friendly tools that simplify analytics, so users don’t need a data science degree to use them. This puts the power of business intelligence in the hands of people who understand the business. The bad news is that most of these systems can become overwhelmed if they’re connected to too many data systems at one time. This is another critical consideration when planning your program.

Big data has many rewards for any organization that takes the time to build it in a systematic and measured way. Like everything in life, if it’s worth doing, it’s worth doing right.