Starting Small with Big Data

I recently read an excellent IBM paper on best practices for achieving line-of-business (LOB) performance increases through enterprise data analytics. This is an impressive and convincing piece on how top-down planning at the enterprise level can enable positive business outcomes across all business units by leveraging analytics applied to all enterprise data. It is highly recommended for large organizations that are ready to be serious about Big Data and analytics across the board.

But what about those organizations that do not have an enterprise-wide master vision about where to go with Big Data and analytics? Must they wait until all of the pieces are in place before pressing the GO button? From a top-down perspective, some may argue they probably should wait. However, there certainly are business units with valuable data stores (primarily transactional data and customer data) that have a vision about what this data could be used for—long before the parent organization and other business units are ready to press GO.

The truth is, it can make sense for business units like these to proceed—with proper preparation and by communicating intent, ROI and status to other groups within the organization, particularly corporate finance and corporate IT. In fact, in situations where the need is great, and where other departments are unwilling to invest or participate, a limited Big Data and analytics project could be very beneficial, as long as the ultimate goal can eventually be extended to an organization-wide objective.

So how would such a project be beneficial? It makes perfect sense for a dry-run or controlled-prototype project to be confined to a single business unit or specific geographical market so that valuable lessons can be learned with little or no risk to the organization as a whole.

Implementation

If and when limited department-level analytics projects are considered, I would assume there would be analytics champions—those individuals driving the project—in place. I would also assume that those driving the project would have analytics intelligence goals in mind—otherwise, such an endeavor is a nonstarter. Further, I would assume that funding would also he managed at the departmental level but, if not, then buy-in from corporate IT would be required, which would require much more planning and active inter-department collaboration.

Whether such a project is managed by corporate IT organization or a specific line of business IT organization, a solid business case is an absolute necessity. This would need to include details on the business benefits, funding requirements and involvement of IT personnel. It would also need to include a detailed start-to-finish project plan, a listing of all stakeholders and at least one clearly identified executive sponsor. Time lines would also be necessary, as would a solid estimate of ROI and a forecast for a rapid payback period. After all, a Big Data and analytics project can only be seen as successful if financial benefits are realized, and those benefits must extend past the cost of the project, including costs caused by “surprises” and unplanned delays.

And, of course, before you can even come close to getting started, there are a myriad of tough decisions to make, plus some important considerations that cannot be ignored:

• Data: Which data are to be included in the project?
• Time sense: Do you plan to run analytics on historical data, or will you need real-time data?
• Data stores: Where will the data be collected for analysis (corporate data center, private cloud, external cloud, etc.), and what platforms will be used (Hadoop or Cassandra on Linux, Pure Data on the mainframe, etc.)? And will your decisions here hamstring future organization-wide activity?
• Data movement: Toolsets and solid data-movement strategies must be in place, and they should be compatible with not only your planned data store/platform, but also with anything that could be considered when the entire organization adopts a Big Data and analytics strategy.
• Analytics tools: Which will you use? They should be compatible with not only your current plans, but also be flexible enough to account for the future.
• Personnel: Certainly, the will to proceed with an aggressive project includes the leadership to see the project through, but all of the resources need to be in place, including expertise from most IT disciplines. Hopefully, personnel will include some with Big Data and analytics experience, as well as data scientists and/or in-house personnel who have intimate familiarity with your data.
• Project and budget tracking: As US President Dwight D. Eisenhower said, planning is everything, so your plan must include careful tracking of all aspects of the project, especially keeping a close eye on your planned versus actual budget.

Finally, at some point you will need to demonstrate your successes to other departments and to the organization as a whole. This would include detailing what you learned, how your actionable intelligence was able to translate into the identification of new business opportunities and/or drive new revenue growth, how customer satisfaction and loyalty were enhanced, and how business risk was mitigated.

Note that success will be measured not only by meeting a measurable ROI and rapid payback, but also on how well your business outcomes will result in a propagation of similar activity throughout the organization.

The bottom line is that you can do self-contained, business unit-level Big Data and analytics projects, and long as you plan, prepare and track properly, and set yourself up for success. While not always recommended, small starter projects can help to set expectations for future enterprise-level analytics projects, leveraging recent experience (good or bad), and helping to ensure success as the enterprise moves forward.