Anthropology and Market Research

A long time ago, I was offered a job in a qualitative research company. I ended up not taking it, and returned to academics. That part of it is another story, but the company hired me through campus recruitment. They wanted someone with my set of skills. I remember being happy that they understood the value of well-done research with a solid foundation.

Flash forward about ten years or so. To last week, in fact. I was wandering through a mall with a friend; let’s call her Amy. A woman holding a clipboard came up to us and asked us, “Do you want to do a survey? You’ll get seven dollars! It’s about product packaging, and it will only take twelve minutes.”

I confess I have a hard time refusing people who administer surveys. I’ve done their job, when I was in college. I spend days calling people and trying to get them to answer surveys. It’s a terrible job. I had to meet a quota, they didn’t want to talk to me, many people yelled at me, and (as is generally the case with surveys) I didn’t have enough responses. It never occurred to me to fudge my data, but I discovered later that some other people who were doing the same job filled out some of their surveys themselves. It was a few more years before I really came to realize that that rendered the data meaningless.

Anyway, so here are Amy and I in the mall, and we say, “okay, we’ll take your survey, we could use seven dollars.” So she turns to me and this is how the conversation goes.

Woman: How old are you?

Me: XX years old (you don’t need to know, gentle reader)

Woman: Do you earn more than $XX000 in the year as a household?

Me: Yes.

Woman: Have you ever used X brand of paper plates?

Me: (thinking very hard now) I think so.

Woman: Do you use them more than five times a week?

Me: (horrified at the thought of such waste) No!

Woman: Have you ever used body wash?

Me: (okay, here’s what we’re really about) Yes.

Woman: Do you use it more than five times a week?

Me: No.

Woman: (to Amy) How old are you?

So at this point, I’m a little stumped. I’ve realized a few things:

1. She’s interested in body wash. I don’t use body wash, but it’s not like I don’t shower more than five times a week. I simply use bar soap. I already see an area in which their survey fails, which is that it could be a vehicle for increasing the product market, i.e. bringing the body wash to the notice of someone who hitherto does not use body wash on a regular basis. Would the new packaging do this? Aren’t companies interested in increasing their market?

2. I’m not getting seven dollars.

So anyway, Amy, being smarter than I am, answers “yes” to the body wash question. The lady takes us upstairs to where a young man asks us if we have ever taken a survey, explains it, and gives Amy a bunch of papers to sign, including a confidentiality agreement. He did not ask me to sign one, even though I was in the room and trailing around behind them. But because I’m nice, I won’t tell you what it is. I’m sure someone, somewhere, cares.

While she’s reading the paperwork, I talk to him.

Me: So, what’s your target demographic?

Man: Well, you know how we asked you all those screening questions? We’re looking for people between 34-54 who use body wash or bar soap at least five times a week.

Error! Error!

Anyway, I decide I don’t want to get the screener in trouble, she probably gets paid chickenfeed anyway. So I don’t tell the young man that the screening wasn’t quite administered properly. Maybe they were on a budget, or maybe she figured I didn’t shower everyday.

Man: We also screen out people who work for Wal-Mart, or the companies which produce these products.

Me: How do you get that information?

Man: We ask the survey takers.

Me: Nobody asked us that question.

Man: Huh?

Me: Nobody asked us that question. You don’t know where we work.

Man: (after a pause) Well, there you go!

Amy: *rolls eyes* I’m done.

Now things move on to the next phase. She is given a clicker, and pictures of display shelves in supermarket aisles are flashed on a screen in front of us. A camera positioned in front of Amy supposedly tracks her eye movements. I watch the man administering the survey. He is intent on his computer screen, though he continues to explain things to Amy. Have I mentioned he talks to us like we are five years old? Very annoying.

Then we get up and move to another room, where there is a computer with multiple-choice questions. Amy is shown some photos of the body wash in different packaging, and sits down to answer questions. I notice we have been taking the survey now for eighteen minutes, and we are nowhere near done. I wander around before walking into this room, and the original screening lady very sweetly asks me if I would like to wait outside.

“You might get bored. It’s just a bunch of questions.”

Me: (brightly) Actually, I’m terribly interested. I’m an anthropologist, you see.

She: (blankly) Oh. Okay. *shrugs whatever-ly*

I wander back into the room. At some point Amy starts clicking a little randomly. She is bored. Another flaw! The survey is too long. This is something so basic to survey administration, I’m surprised they didn’t think of it.

Finally we are done. Or so we think. The man escorts us into another room, with a mock display unit set up. He asks Amy to locate the product she currently uses. Incidentally, she couldn’t remember it, and when we were done and left, suddenly said, “I just remembered! I actually use their body wash!”

However, she still thinks she uses Dove, so she goes ahead and locates the Dove on the shelf. Then he asks her to locate Product X, their body wash. She quickly scans the shelf and does so.

Man: How did you locate it? Were your eyes drawn to the middle? Did the packaging stand out?

Amy: No, I just scanned the shelf left to right, top to bottom, and stopped when I saw it.

Man: Okay, so the packaging stood out. Was it easy to locate?

Amy: Well, once I saw it it was. Had I been standing in front or it, instead of at an angle, I might have seen it sooner.

Man: Okay, so the packaging stood out.

We gave up. As a matter of fact, I was right in front of the display, and saw Product X right away while Amy was still scanning, so her hypothesis stands a good change of being correct.

Why am I relating this long story? Well, because, as an anthropologist, I have some experience doing qualitative research, and a lot of basic things were being done wrong here. That meant the survey was inefficient, data gathering was flawed, and the resultant data were unreliable. Company X will base its packaging decisions on data like this, which means millions of dollars are eventually spent on data which are not entirely accurate.

Here are the flaws and inefficiencies, as I saw them:

  1. The screening was not done correctly and potentially introduced errors and confounding factors into the data. I’m merely talking about the questions here, because I don’t have enough information on how they randomized their sample or if there were other conditions prior to randomizing it.
  2. The screening eliminated potential customers who might have been willing to try Product X were it brought to their notice
  3. Errors, when pointed out, were not corrected.
  4. The survey was too long. This leads to the danger of boredom and of the survey takers filling out things arbitrarily just to be done.
  5. The most glaring flaw of all–not listening to the respondent. As with the last set of questions, the administrator overlaid his pre-defined responses onto Amy’s, thus falsifying data. What he said she said was not what she said at all.

How do you fix these errors? Well, you can and you can’t. All these errors were a result of careless or untrained survey administration. It’s not enough to tell administrators what questions to ask. You also have to impress upon them the importance of efficient and ethical data collection. It seemed to me that the survey administrators were a firm hired by Company X. In this case, the company may not have much option but to trust them to do their job carefully.

What does this have to do with anthropology? Easy enough. Companies need people with the skills to administer and oversee such market research projects. Anthropologists have all the required skills to do this job and do it well. We are trained observers and qualitative researchers. who understand the critical importance of sound data. As an observer, I was able to spot areas in which survey administration could be more efficient and useful to Company X. An anthropologist on their payroll could do a lot more.

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

3 Responses to Anthropology and Market Research

  1. An impressive share! I’ve just forwarded this onto a friend who was conducting a little research on this. And he in fact bought me dinner because I found it for him… lol. So let me reword this…. Thanks for the meal!! But yeah, thanx for spending the time to talk about this topic here on your internet site.

  2. Chris says:

    Interesting article, however I think you are doing the professional market research industry a huge disservice. The problem here is not that professional market researchers are oblivious to the issues you point out (and need anthropologists to step in and show them the way) – they are experts in designing and conducting research projects, that’s all they do, day in, day out. They marry this with their marketing and branding expertise.

    The problem here is that the hall test was not conducted by the market research professionals who designed the project – it was conducted by unskilled interviewers who have had minimal training. (If it had been conducted by a qualitative market researcher, by comparison, you would have had no cause to complain – they’ve conducted far more fieldwork and used more methodologies than your typical anthropologist) You also experienced the perennial problem of the recruiter who can’t be bothered to ask all the questions on the screener and just wants ‘bums on seats’. Moreover, the researchers overseeing the project were probably also aware it was too long, however clients tend to have smaller and smaller budgets and want more and more questions answered….and they have the final sign-off however strongly the overseeing researcher advises against it.

    So that’s one issue. Now, the sample issue. The researcher doesn’t just make up the sample out of thin air, it is based on the client’s business and marketing objectives. Researchers are highly adept at recommending methodologies and samples based on the objectives. They are also very good at challenging the brief, for example, asking why non-users of the body wash are not being included in the sample and suggesting that they should be (if indeed they think they should be). So there are two possibilities here:

    – It may be that the client’s focus at the time was not on bringing in new users, but, for example, on encouraging current users to ‘upgrade’ in terms of pack format (= more profit).

    – What I think is more likely, however, is that the hall test was not the only stage of research, nor the only methodology used. I suspect there had already been several phases of qualitative pack design development and evaluation research with both users and non-users of the product. The hall test situation you encountered is mainly used for ‘quick checks’ after the main qual work has been conducted. In this case, I suspect there had already been qual work with non-users, and the hall test was just a quick check to ensure the pack redress hadn’t ‘lost’ the current users.

    So for you to say, with no knowledge of the client objectives, that non-users should have been included in the sample, and that they were missing a trick, is a tad naive.

    I think there is a lot of room for anthropologists in market research (I have been a qual market researcher for 18 years and am also a more recent anthropologist, so I stand on both sides) in terms of bringing a cultural perspective. However suggesting anthropologists are the people best equipped to advise on methodology, sample and moderation issues is erroneous in my opinion. It does a disservice to professional market researchers who spend their working lives doing nothing but that. I would suggest that what your experience actually shows is that more training and spot checks are required when using non-professional interviewers. It is a quality control issue.

  3. anthrocharya says:

    Hi Chris,
    Thanks for stopping by and for your very informative input. My apologies for taking so long to respond. I bow to your argument on all points 🙂 I was being somewhat flippant, and really don’t have much knowledge about qualitative market research. I saw what seemed like flaws in data collection, and it bothered me enough that I wanted to post about it…but I was tripped up by flippancy and ignorance. That said, I did mention that the errors were “a result of careless or untrained survey administration”, so I’m absolutely with you on the quality control aspect. Thank you for explaining a little bit about how this process works–always glad to learn new things!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s