The Big Data juggernaut -- miraculous and awe-inspiring -- rolls along. On the one hand, data volumes have become so large that oftentimes only advanced analytics and other data analysis technologies can make sense of them. On the other hand, humans can glean valuable insights from data that escape even sophisticated analytics software. It's not enough to turn algorithms loose to figure things out, because computers are just too literal, that is to say, they can't decipher ambiguity of language and mystery of reasoning… In cases of data visualization and strategy, it's ALL ABOUT allowing the humans to get the insight, form new hypotheses, and make decisions. (5 Compelling Reasons Why Big Data Should Have A Human Touch). So tell Hal not to get rid of all the humans yet.
Let's take another look at why having humans working with the predictive analytics of pricing is so crucial:
1. Raw data often contains outliers (an outlier is an element of a data set that distinctly stands out from the rest of the data) that a computer may not recognize as important.
2. Humans are able to recognize valuable symmetries, relationships, and adjacencies in data that analytics cannot recognize. Some people want to understand how the data is being processed and look for patterns and insights…
3. Good visualizations make it possible for users to apply filters to data and drill down on items of interest, and then communicate their findings more efficiently at the appropriate level of abstraction.
4. Excessive dependence on computer models can create blind spots. Humans should check the assumptions of a model and verify that the right questions are being asked in the first place.
5. Human checking gives accuracy to make sure that you are matching apples to apples, absolutely critical when it comes to pricing intelligence. The product data on sites examined for pricing intelligence purposes is highly unstructured and irregular; it can be noisy and it can be partial. These pose significant challenges to data extraction and to matching -- and that's why human quality checking by leading providers of price and assortment intelligence is essential. (Shai Geva, Upstream Commerce co-founder and Chief Technical Officer in How The Most Advanced Pricing Intelligence System Tracks Your Competitors’ Prices).
6. You still can't program a computer to clean, drive, or cook the way a human does, or perform human quality checking. Humans must still perform non-routine physical jobs and highly-skilled, complex mental jobs.
7. A computer cannot do things to make people feel good, take care of others, be artistic and creative for its own sake, or express emotions and vulnerability. (Anya Kamanetz, The Four Things People Can Still Do Better Than Computers, in Fast Company).
Bottom Line For Your Bottom Line:
Some say the sheer volume of data being generated makes most modes of traditional manual analysis unrealistic at best. That doesn’t mean that we should simply automate the entire process and walk away.
"When it comes to pricing data, accuracy is paramount. Noise or source of the data directly affects access and accuracy, and, most important, it affects bottom line. So, it's not enough to count on automated systems alone to verify themselves. Human-based, quality assurance checks, performed on a regular basis are imperative to ensure that your data is 100% accurate."