Our clients often ask us “What is big data?” And the answer they are really looking for is not the Wikipedia definition but actually what the connection is between big data and market research.
To answer this question purely in the context of MR, we start by telling them what big data is not.
- Big data is NOT the solution to all your problems
- Big data is NOT a black box where magic things happen
The reality is that a lot happens behind the curtains before you get to the analysis stage. Big data is about organizing large sets of data so it can be processed faster.
One of the most important aspects to consider about big data is that it only makes sense if the data sets are large enough, and it needs to be done with a specific, defined business-need in mind.
At Ugam, one of our solutions caters to the retail industry. Our retail team crawls the Internet daily, collecting gigabytes of pricing information for products and using the information to offer price intelligence solutions (amongst other things).
Because we know what we are going for, we let our robots loose to get this data for us and bring it in within a certain structure. The volume of data we collect is so large that we need tools to help us aggregate and query that data. This is one practical example of big data in action. Every day more and more data is being collected and our data gets richer because we are collecting it with a purpose.
The Market Research Thing
We all agree, market research needs to evolve, but it is not only about making surveys shorter (which is needed) or mobile surveys (which is happening) or passive monitoring (which will get better). We have to get better at understanding and merging other sources of data. For example, one area that requires our attention is using our clients’ transactional data. Clients have loads of transactional data sitting inside their systems that is very valuable for understanding their customers. The answers to many of their questions could potentially lie within that data.
In the spirit of evolving our industry, it is also critical that we figure out a meaningful way to utilize democratic data—that which is freely available on the internet. I think we have passed the stage where we ignore it (maybe because it scares us?) and are now in the stage where we underestimate it. I keep hearing, “This is not representative,” or “It’s nice, but you really can’t do anything with it.” Eventually, we will take it seriously and give it the attention it deserves. For the researchers of tomorrow, this will be the most important source of data to understand consumers in newer and deeper ways.
New Times Require New Measures
Until not long ago, the only option for storing digital photos was CDs. Google recently gave us all the free access we need to store an unlimited amount of photos on their cloud servers. No more stress about CDs failing and losing your most precious memories. Today we can be confident that our photos are safer in the cloud than anywhere else. Similarly, for big companies, in the past, it was difficult to collect and store massive amounts of data – let alone make use of it.
Today it is different because technology has given us the tools to capture more data, storage is only getting cheaper, and we keep gaining more and more processing power. All these technological advancements are making processing and querying this ocean of data possible.
If, at this point, you have already realized that this data needs to be structured while you store it, then your future clients will see the benefits from this knowledge. You will know more about your customers than any of your competitors ever can.
The time is now to start advising clients about this. Structuring data is not a task that MR firms typically would give advice on, but if we are able to guide our clients on how to organize all this internal data, it will make future market research integrations a lot easier. In the end, our goal is to help them understand their customers in new ways. In this new world of multiple sources of data, big data is simply the underlying technologies and tools used to unearth insights and make all these instruments sing together.
Content from this article originally appeared in RW Connect on September 4th, 2015.
Felix Rios is a Market Research Technology Manager at Ugam. He is passionate about technology and beyond the office walls, also enjoys photography.