With sales of big data technology expected to reach $32.4 billion and a CAGR of 27 percent through 2017 there are going to be lots of new jobs in big data. Unfortunately, there continues to be a shortfall in big data talent, especially big data scientists. There are expected to be 4.4 million jobs in big data in 2015, and McKinsey is already forecasting a shortage of up to 190,000 data scientists by 2018. So while there are plenty of jobs in big data, it’s going to be difficult finding the talent to fill those jobs.
In the hunt for marketing analysts, for example, the latest CMO Survey shows that only 3-4 percent of marketing executives feel they have the analytics talent they need. The same CMO Survey says there is greater marketing ROI for companies with the right analytics talent – +4.18 percent as opposed to +2.51 percent. And companies with better analytics talent showed a profit increase of +4.69 percent versus +2.71 percent for those with below-average analytics skills.
So where are the jobs in big data going to be, and more importantly, where are we going to find the talent to fill those jobs? Here are some predictions for 2015:
New Jobs in Big Data
Gartner predicts that only one-third of the jobs in big data will actually be filled in 2015. The reason for the shortfall is simply a lack of qualified talent. Some big data skills are transferable from other professions or by applying related expertise, but much of the big picture analytics and interpretation can be highly specialized. Here are some of the new jobs that are emerging thanks to big data:
- Data scientists – This is not a new role but it has taken on new responsibilities with the big data boom. Applying a background in statistics and mathematics, or even artificial intelligence and natural language process, big data scientists are responsible for taking the lead in determining what data types will yield the most insight, and interpreting the results of big data analytics. This role is very much in demand and is going to be the most difficult to fill.
- Data architects – As part of the analytics process someone has to build the data models, including mapping out which data sources will yield the most insight, and how the data sources map to the analytical tools to fit the pieces together.
- Data engineers and data operators – These are the IT experts who keep the big data machinery moving, developing and maintaining the architecture to access and analyze data on an ongoing basis.
- Data stewards – One of the functions of big data is to break down data silos within the organization. Someone has to be responsible for taking inventory of all those data sources and makings sure everything is accounted for. Data stewards also are responsible for managing a master data repository and a strategy to keep track of data sources.
- Data visualizers – Once the analytics are delivered they have to be interpreted, and presented to the executives responsible for putting big data insights into action. Someone has to build the dashboards, graphs, and alerts to turn the information into actionable insight that is quick to grasp. It takes a special kind of skill to explain data in context and in plain language.
- Big data change agents – Uncovering actionable insight is one thing; putting it into practice is something else again. Change agents will have responsibility for evangelizing innovation based on big data findings, driving change in internal operations, and identifying new ways to do business.
- Data virtualization specialists – What has become clear is that enterprise networks aren’t equipped to store the petabytes of data required by some big data projects. Cloud computing and data virtualization are proving their value in delivering the kind of elastic data storage resources required for big data analytics, and that means someone has to know how to build and support these cloud resources.
Some of these roles will be included as part of existing job descriptions while some will emerge as new jobs in big data projects.
Education is working to catch up by offering specialty courses and degrees in statistics and big data programming and support. The IBM Academic Initiative, for example, has partnered with 1,000 universities to support studies in big data and analytics. In the meantime, you can train your staff in big data tools and techniques. For database administrators and data warehouse experts it’s not a big leap to learn how to deal with massively parallel processing databases. And you can adapt UNIX, Java, and SQL expertise to Hadoop to develop Hadoop clusters. You can always bridge the big data talent gap by training your existing staff rather than waiting for the right experts to come along.