Big data is booming, and part of your job is determining how to support big data without disrupting the budget or the infrastructure. According to CIO Magazine, big data projects will continue to grow at a rate of 30 percent annually through 2019, and half of those projects will fail because of operational inefficiencies, including poor staffing. Head count is always a concern, but what specific talent do you need for big data staffing?
If you appreciate the specific talents you will need, then you can be able to assemble a big data strike force that can maximize the value of your company’s big data investment. The challenge is to balance the demand for specific big data staffing roles with a mindset that promotes customer service, especially for analytics.
The Five Pillars of Big Data Staffing
According to the McKinsey & Company, there are five key roles for any big data project:
- Data Hygienists – When working with big data, you have to be sure you start with clean data and that the data stays clean throughout the analytic process. For example, are time and date values the same? Are you using standard time or military time? Do calendar days reflect 365 days in a year or 260 working days in a year? Data has to be clean and consistent to be comparable for analysis.
- Data Explorers – These are the experts who sift through the mountains of data to find the nuggets that are of value. Much of the data of real value wasn’t originally intended for analysis, so expert Explorers need to find a way to uncover that data. For example, retail receipts were originally intended to gauge P&L, but they have now evolved to project consumer product demand and supply chain needs.
- Business Solution Architects – These big data professionals are responsible for assembling the data and organizing it for real-time analysis. By structuring the data properly, it can be queried in appropriate timeframes by all users to yield needed information. For example, some data will need to be updated every minute while other data is updated hourly or daily.
- Data Scientists – Once big data is organized, the data scientists create analytical models. For example, they might use big data to create price optimization models by market, or to predict consumer behavior on a seasonal basis or by customer segment.
- Campaign Experts – These big data gurus take the models and turn them into results. Their expertise matches the big data findings to the mechanics of business operations. For example, for marketing they can determine what messages customers receive at specific points in the sales process. Campaign experts also can create models to prioritize messages and channels, and set the timing for follow-up messages.
The data progresses from step to step, with each member of the big data staff taking responsibility for his or her part. Mapping each step in the process ensures accountability at each stage so the final product is accurate and useful.
Big Data Staffing Facing a Shortage
So while the key roles in your big data strike team are well defined, finding the right personnel to fill those roles is another challenge. According to McKinsey Global Institute, the United States will be facing need an additonal 140,000 to 190,000 big data professionals by 2018, not to mention the 1.5 million managers and analysts who understand what to do with the data.
There are efforts afoot to overcome the big data staffing shortage. IBM has launched new big data curricula that they are spreading through universities to address the big data staffing shortage. IT professionals also will have to retrain to take on big data processing and analysis. But in the short term, IT managers will have to start seeking out data scientists, or grooming their own data scientists to support big data initiatives.
The skills required for data scientists include strong data skills, analytical skills, knowledge of statistics, and the ability to program algorithms. Rather than trying to find all those stills in one professional, consider doling it out among big data staffers, or even promoting and training your own IT people. A big data analyst doesn’t have to have programming skills or build algorithms, although a strong background in SQL will help them understand the analytics involved. And the analysts need to be able to interpret the data. Those are skills you can cultivate within your IT team. Or consider outsourcing to hire the necessary big data expertise.
Once you understand the basic requirements of big data analysis, you can assemble a strike force capable of handling any big data project. So how will you tackle your big data staffing needs? Will you hire, train, outsource, or find another way to assemble the right team of experts?