Big data is becoming more and more important to businesses that want to find insight in the data they collect, but companies are struggling to take advantage of the technology. In 2014, the IDG Enterprise survey found that 40 percent of big data projects struggled due to a skills shortage. For solution providers, this creates the opportunity to sell big data consulting services to help companies implement their projects. Wikibon estimates that the big data market will be more than $61 billion by 2020, with professional services more than 40% of that market.
In order to participate, solution providers need big data consulting skills that can support four different aspects of the big data market.
1. Business Analysis Services
Finding value in big data means analyzing it with a purpose, but for many companies, determining the business problems and end goals of big data projects remains a challenge. Vendors that understand the business domain and can help companies define the use cases for their big data projects will find lots of opportunity to work with companies starting at the earliest stages of development. While there are use cases that apply to almost every domain, bringing knowledge of the vertical's special challenges will allow solution providers to identify industry-specific analytical use cases that offer a competitive advantage to their client.
Once the company's big data priorities are established, the solution provider can work with the client in order to develop an implementation plan. This may include determining the structure of the big data team, performing gap analysis in order to discover where the needed technology is missing, identifying the kinds of analytics needed to solve the problem and ensuring the necessary data is available.
2. Hardware Infrastructure Services
Big data requires big hardware: lots of storage space, lots of CPU capacity and plenty of network bandwidth for moving the data around. As InfoWorld points out, big data projects require tailored hardware solutions. A company's existing server farm may not meet the parallel processing, real-time analytic demands of big data projects. Providing the necessary processing capacity requires high-performance, high-CPU, high-memory, largely virtualized servers that can easily scale in order to meet demand. Storage capacity also needs the ability to handle rapidly growing demand that reaches petabytes and beyond.
Big data consultants need to understand the processor and storage technologies and architectures in order to provide the needed capacity. Their recommendations should reflect whether the customer needs to support real-time analytics or batch processes, which have different demands. Because companies may not be able to implement the necessary architecture in their own environments, a solution provider's big data consulting skills should include an understanding of how the needed architecture can be achieved in the cloud.
3. Software Infrastructure and Development Services
Along with an enhanced hardware infrastructure, companies need to deploy new software infrastructure in order to support big data projects. This goes beyond merely establishing a data warehouse, but identifying the software platform to be used for the big data applications. The best platform for the business should be selected through careful analysis. For many companies, the platform of choice will be Hadoop, so big data consulting skills need to include the ability to deploy and train customers on that tool. Solution providers should also have skills working with NoSQL databases and analytical software products such as MATLAB. Consultants should understand the analytical methods used in big data projects, including text mining, predictive analytics and social media analytics.
Customers are likely to turn to consultants for support developing proofs of concept or even complete applications, so big data consulting skills should include a solid foundation of programming in traditional languages like Java and Python as well as working with the big data-specific products. Consultants need a thorough understanding of how to implement an effective ETL (extract, transform and load) process in order to make data movement efficient. Consultants should know how to work with the big data features available through Google Cloud Platform and Amazon Web Services.
4. Support Services
Once the big data environment and applications are deployed, they need ongoing maintenance, support and monitoring for performance and security issues. Big data consulting skills can include the services needed in order to support the production use of big data analysis through integration with other enterprise applications. Consultants can help companies address issues such as backup and disaster recovery, which are complicated by the volume of data involved.
With the right big data consulting skills, solution providers can help their customers succeed in big data. There's big value in this—in one IDG survey, less than half of respondents considered their big data projects successful. Helping customers work through the challenges of big data will lead to big rewards for vendors with the necessary skills. Partner with Ingram Micro and draw on our resources to achieve success selling big data services to your customers.