How to Check Market Research Agency Credentials

(part of Chapter 7: Market Research Agency Selection)

 

There are literally hundreds of market research agencies from which you can choose. Of course, finding ones that have the right qualifications can be tricky. There is no professional certification required in the field, so you need to do a little work.

Read on for some tips on checking potential research partners for appropriate credentials.

 

The credentials for research professionals vary for qualitative and quantitative methods.

Qualitative Research Credentials

If your project will involve focus groups or in-depth interviews (IDIs), ask about each moderator’s qualifications (the moderator is the person who leads the discussions). Anyone who is trained for moderating focus groups is usually also well-prepared for IDIs. I have occasionally worked with people who are strictly IDI interviewers, but that is rare.

Be sure to ask what kind of training moderators have completed. In some cases, people will say, “Oh, yes. They’ve been certified. They’ve completed focus group moderator training at either Burke or Riva.” Both organizations have high-quality, widely respected focus group moderator training programs.

I do prefer moderators who have had formal training. Some people will admit that their training is on the job, and that doesn’t mean that they’re going to be bad moderators. But in my experience, the best moderators have had formal training.

Unfortunately, a lot of people who are good conversationalists assume that they’d also be good focus group moderators or interviewers, but it takes real skill to be able to do things such as:

• Manage a dominant respondent.


• Stay on topic (avoid the temptation to abandon the client-approved guide).


• Diffuse peer-group competitiveness.


• Understand when you’re not getting a truly honest opinion from a passive respondent.


• Get at the “why” behind responses without making participants feel like they’re being attacked.


• Keep the momentum going (whether it’s a 30-minute IDI or a 2-hour focus group, if the participants are bored, they will not give you great information).

Truly, there are a lot of tricks to the trade.

Also ask if the moderators are on staff or subcontracted. Does it really matter? Not always. Many agencies have regular subcontractors for this function, but if they do subcontract, it’s wise to ask questions such as these:

• How many times have you worked with this moderator?


• For what types of projects? (To which you hope they reply with something relevant to your topic.)


• Does he or she have expertise in my industry?


• How many years of experience does he or she have with interviewing or moderation?


• Is his or her experience more with moderating discussions with consumers, or with business professionals?


• What is this person’s greatest strength as a moderator or interviewer? (To which you hope they reply: “excellent ability to get past superficial responses,” “knows how to probe effectively for the reasons behind specific behaviors and attitudes,” or even something like, “keeps participants engaged during the process. Keeps energy levels high”; those would all be comforting things to hear).

 

Quantitative Credentials

If you’re doing quantitative work, you need to know something about the qualifications of the people who are going to be doing the statistical analysis. You also need to know how much of the quantitative process is handled in-house vs. outsourced.

Some research agencies do outsource statistical analysis tasks (although any shop that has 10 or more employees will at least handle basic statistics in-house: descriptives, cross-tabs, etc.). Outsourcing can be fine, especially in the following scenarios:

• You want an agency that has deep expertise in your industry. Let’s say you are in the pharmaceutical industry, where a lot of industry expertise is required to design a meaningful project and analyze the findings with the right context. In this case, you might be fine with a firm that outsources the higher-end statistical analysis because it simply isn’t possible to find an agency that has both sets of deep expertise in-house.


• You have purposely decided to work with a smaller agency (a perfectly legitimate choice). Some smaller agencies stay small by using a network of trusted partners for handling overload times and less frequently required skills.

It’s also important to understand a firm’s comfort with statistical analyses, even if it does outsource them. What types of quantitative techniques has it used? At the low-end extreme, it might simply be generating frequencies and cross-tabs. For some projects, that is absolutely fine. But if you are doing a project in which you are trying to forecast behaviors, segment a market, or otherwise model customer attitudes—projects that have some sort of predictive aspect—then you need a firm with more robust analytic skills.

Here are some techniques to listen for:

• Numerous types of multiple regression including linear, non-linear, logistic, and multinomial logistic regression


• Tradeoff analyses, including Conjoint, Discrete Choice, and related hybrid designs


• Factor Analysis and Principal Components Analysis (PCA)


• Latent class modeling


• Decision tree analytics including CHAID and Exhaustive CHAID


• Hierarchical Bayes modeling

If an agency informs you it routinely applies these methods, it probably keeps up with the current state of quantitative methods. Alas, this is important: I have run into MR firms in the past that insist they can handle quantitative work, but when I ask them questions like “What techniques do you apply in segmentation studies?” their answers reveal that in reality, they are not well-versed in the currently available set of analytic tools to perform such projects with excellence.

 

Professional Associations

A truly reputable market research company that follows best-in-class practices will most likely be a member of at least one of these professional associations: AAPOR, CASRO, ESOMAR, or the MRA. Here’s a hint: if a firm has never even heard of these associations, buyer beware!

• The American Association for Public Opinion Research (AAPOR) is a U.S.-based organization of public opinion and survey research professionals. Its members include professionals from academia, government, media, non-profits, and private industry. APPOR provides various materials and educational resources on its website and also publishes Public Opinion Quarterly.


• CASRO does a lot of work for the MR industry in terms of establishing best practices for conducting MR and for managing the industry’s relationship with people who actually take surveys.
  
  For example, there’s a Code of Standards and Ethics for Survey Research that CASRO member organizations agree to abide by, which is quite important: It helps make sure that the MR industry maintains a high level of credibility. Credibility is crucial, so that when people are asked to take surveys, they’re comfortable complying with our requests to participate.


• ESOMAR is a 5,000+ member international organization that promotes the role of market research and related best practices. It also publishes Research World, and hosts events for MR professionals.


• Professional MR organizations are also often members of the MRA. Also, as mentioned in Chapter 6, the MRA publishes an annual directory, the Blue Book, which is available online. The Blue Book lists MR providers by location and areas of specialty. The MRA is more U.S.-centric (as compared to ESOMAR), and hosts various conferences and events.

If you find yourself evaluating firms that don’t belong to at least one of these organizations, I would ask them why they don’t. This is a good way just to make sure that you’re talking with somebody who isn’t just claiming to do MR, but somebody who really is a professional practitioner.

 

Publications

Have the MR firm’s executives or project managers had articles published in magazines such as Quirk’s, the Journal of Marketing Research or Marketing News (an AMA publication)? Authoring articles lends some credibility to a company.

That said, smaller MR agencies often don’t have time to author articles, let alone go through the tedious process of placing them. Don’t let a lackof publications prohibit you from choosing a company. But if the firm has published, it can give you some insight into its areas of expertise and provide evidence to back up any claims it might have about a particular area of expertise.

 

Speaking Engagements

MR agencies often will send their executives to present at conferences hosted by the MRA, ESOMAR, and other professional groups. This can be a great way to meet agency representatives with the pressure of being in a current sales process. However, smaller agencies don’t always have the resources to participate in such events.

 

This is an excerpt from the book, "How to Hire & Manage Market Research Agencies," which is available on Amazon. Published by Research Rockstar LLC. Copyright © by Kathryn Korostoff. All rights reserved.