Big data is in demand and that means more demand for big data consulting firms. Scores of companies are either starting a new business to serve the big data market, or expanding their existing consulting practices to embrace big data. If you want to launch your own big data consulting practice, then you need to be prepared to compete in a growing market.
IDC forecasts that the big data market will grow 27 percent annually reaching $32.4 billion by 2017; six times the growth rate of the IT/communications market. There is an incredible amount of room for value added resellers, system integrators, and service providers to make the most of this growth with consulting services, if they have the right approach and skillset.
The Four Elements of Big Data Engagement
When you consider what it takes to sell a big data engagement, you can break down the types of big data consulting needed into four specific segments:
- Business consulting – If big data is the solution you first have to define the problem. Your first step in big data consulting is working with the senior executive team to understand what business problems they are trying to solve – increasing sales, improving customer intelligence, opening new markets, designing better products, streamlining operations, etc. You have to serve as a business consultant, defining the business problem in such a fashion that it can be solved by big data analytics.
- Software consulting – Once the problem has been defined, you have to develop the software to analyze it. This requires defining the appropriate big data resources, using Hadoop or NoSQL to create algorithms and analytics frameworks, testing and refining the software, and assessing data sets and correcting anomalies until you are confident of the results. This requires being able to define the business problem in terms that can be solved using Hadoop or some other big data framework. Ideally the software experts will be familiar with different programming and scripting languages, such as Python, R, Java, Ruby, Clojure, Matlab, Pig or SQL, and have an understanding of Hadoop, Hive, HBase, MongoDB and/or MapReduce.
- Hardware consulting – Once you have defined the scope of the software requirements, you can determine what kind of network infrastructure you need to run the analysis. This will require gathering together the right data sources, allocating enough data storage, designing sufficiently powerful computing systems to run the analyses, and so forth. This also requires a knowledge of network attached storage (NAS), software-defined storage (SDS), cloud storage, virtualization, software-defined networking, and more.
- Analytics – Once the infrastructure has been built and the data analyzed and re-analyzed, someone has to interpret it. A critical part of big data consulting is turning the analysis into strategy. If you don’t have a data scientist at hand, then you need to have an expert who can map the analytic results back to the original problem and graphically present trends that will inform an action plan.
Only the larger big data consulting firms will be prepared to offer all of these services. It will be more likely you will specialize in one or perhaps two areas and ally yourself with other experts to fill in the gaps.
The Characteristics of a Successful Big Data Consultant
The real role of the big data consultant is that of project manager. Even though you won’t have the expertise to execute every aspect of a big data project, you should be able to assemble the necessary experts and oversee the project to achieve the desired results.
In addition to being able to oversee translation of business problems into big data analytics, clients will expect you to have all the other trappings of a big data consulting practice. Technical expertise is important but you will need other credentials and skills :
- An advanced degree in a related field;
- An understanding of the business climate, including your client’s vertical market, and its driving factors;
- Excellent interpersonal skills;
- A natural curiosity that leads you to want to solve complex problems;
- The ability to handle multiple responsibilities and tasks simultaneously;
- The ability to meet regulatory deadlines and work under pressure; and
- Project management skills, including being able to develop and review project plans, manage workflow, and both participate as an independent leader and as part of a team.
The role of a big data consultant is similar to that of an orchestra conductor – you have to understand how the various parts of the orchestra contribute to the overall piece and be able to direct them accordingly to achieve the desired result. Understanding how big data works is important, but understanding how to bring the right big data experts together to design and execute a successful big data project is absolutely essential.
What do you see as your strongest skill for big data consulting?