How do you know when you big data team succeeds? Unlike other types of technology sales, selling big data is not about speeds and feeds. It’s about selling actionable insight that customers can use to improve operations or make money. Your big data team needs to be able to use big data to answer big questions that will justify the big data investment.
According to a 2013 Gartner report, 64 percent of organizations are either already investing or planning to invest in big data versus 58 percent in 2012. Seventy percent of companies are already using pilot programs, and 80 percent are planning to start a big data initiative in the next two years. The Gartner report says that organizations are now moving from big data experimentation to big data deployment, so now is the time to assemble a big data team that can help customers mine real insight from big data.
There are three staffing components needed to deliver big data projects, each with its own metrics for success:
1. It All Starts with the Sale
The first string of your big data team is sales. Unlike IT-driven sales where your technical experts talk to the IT management or CIO about infrastructure, the big data sale starts in the executive suite with a business problem. Before you start talking about the infrastructure you need to define a business case that big data can address.
Many organizations dive into big data without a clear objective in mind, and those projects will fail. To benefit from big data you have to start with a specific question that needs to be addressed: how to improve production, open a new market, streamline the supply chain, or some other definable problem. For example, the Gartner report says that 55 percent of companies are using big data to improve customer experience and 49 percent are looking to improve business processes.
To be successful, the big data sales team has to be able to talk to senior management in the right industry language. It’s more of a consultative sale, and the successful outcome will be a commitment to launch a pilot project that will yield real returns.
2. Designing the Big Data Center
Once the sale has been made you are ready to call in the second string of your big data team, the big data architects. The design team needs to be able to help the customer build out a big data blueprint that includes defining the scope of the project, finding the right data sources, and building out the system to store and analyze the data.
The design team needs to identify the data and processing power required. Big data encompasses structured, unstructured data, and semi-structured data. For example, to improve the customer experience could require sales data (structured data), customer support data (structured and unstructured), and social media data (unstructured). The nature of the data streams also will dictate computing resources, such as cloud-based and on premise data storage.
The data center has to be massively scalable with massively parallel processing capabilities and fast access to data. There are other parameters that need to be considered, such as how to prioritize data in storage, but the big data architects will be responsible for creating an infrastructure that can support data gathering, sorting, processing, and analytics.
3. Bring in the Analytics Experts
As your big data team is designing the infrastructure in place, the analytics experts can start working on developing the necessary software and analytic models.
Most experts say that you need a data scientist on your big data team who understands regression analysis, cluster analysis, and organization techniques. The challenge is that big data scientists are hard to come by so rather than finding one expert spread the responsibility among different members of your big data team.
The data science role includes analytics and analysis, data technology expertise, and visualization expertise. Analytics expertise can be drawn from your business intelligence experts and who understand basic programming and have a familiarity with SQL or NoSQL. The data technology experts should know how to extract stored data and manipulate data sources. The visualization experts will be able to create graphs, charts, animation, and even dashboards to help make sense of the big data findings.
In the end, your big data team should be able to deliver insight to answer the customer’s big data questions. And once you have assembled a team of big data experts their experience will grow with each engagement.
So do you think you can assemble a big data team to deliver a big data project from start to finish? What do you see as your biggest big data staffing challenge?