A Response to Annie Pettit’s Piece of October 30th: “Do You Need A Data Scientist Or A Marketing Researcher?”

I try not to let myself get trolled by inexpert opinion dressed up as fact on the Internet, but this post, referenced above, begs for repudiation.

The differences between a data scientist and a market (not “marketing”, please) researcher are so blatantly obvious, I could not sit back and let it go.

Let’s begin with the conclusion you reach at the end of the piece: “Market researchers have a pretty good handle on big data. Perhaps it’s time to realize that Data Scientists are simply Market Researchers by another name.” Here’s the problem with that statement; I don’t think you could be more wrong if you tried. It's like confusing a proctologist with a neurologist - which could only make sense if you've got your head in the wrong place.

I have worked in the domain for over 30 years, built a MR company, and and am now into year two of a big data integration platform. I have met with more data scientists and researchers in my lifetime than most people ever will, and can tell you without doubt that they could not be more different than chocolate and sea salt.

To break it down to a very basic level; market researchers discover insights from primary data collection, i.e. non-existing data. Data scientists gather insights from transactional data; i.e., existing data. Data science overlaps with computer science; it’s directly related to IT. Market research overlaps with behavioural psychology; it’s directly related to marketing.

Do actual experts in these respective fields know the difference? To find out, I took 5 key research terms any market researcher would know, and 5 descriptive terms that any data scientist would be familiar with, and contacted ten market researchers and ten data scientists. I asked the data scientists to define the five basic working vocabulary words/phrases for market researchers, then asked market researchers to do the same for the data science words/phrases. Here are the words/phrases I gave each of these two groups:

Data scientists' basic vocabulary

  • Data wrangling
  • Hadoop
  • Multidimensional array
  • Technical metadata
  • Data governance 

Market researchers' basic vocabulary:

  • Conjoint analysis
  • Respondent fatigue
  • Brand tracker
  • Judgment sample
  • Ratio scale

The results? Zero for zero. Not one market researcher knew what I was talking about with regard to the basic data science terminology, and visa versa. I even went as far as asking each one whether they considered their skills were applicable for the other’s role. “No way!” was the answer I got, across the board.

Data science is knowledge discovery through data inference and exploration. It uses mathematic and algorithmic techniques to analytically complex business problems, leveraging masses of raw information to uncover hidden insights. Most in this field are engineers or computer science majors and specialize in a deep understanding of multiple BI systems and architectural frameworks.

Market research is about gathering information about consumers' needs and preferences. Yes, we develop algorithms and run different analyses on raw data, but all data is collected specifically for the purpose of the particular project at hand.

How could the same person possibly fulfill both of these job functions, when they’re not remotely the same? Short answer; they can’t and they don’t. They can work together and do – but to conflate them is to confuse them. In fact, the siloed nature of these roles speaks to how differentiated they are in all enterprises. I was out for lunch last week with the head of research of a large financial service institution; I asked him who the CIO in his company was - and he didn’t know! 

I rest my case…