Tips For Reading Market Research Proposals

(part of Chapter 7: Market Research Agency Selection)

 

Reading proposals can be tricky. Obviously, a market research agency is trying to sell itself to you, so they’re going to represent their abilities in the best way possible. So what can you do? Learn how to read proposals carefully and look out for details that are too good to be true.

Be realistic when considering your needs versus an agency’s abilities, and trust your gut. Read on to learn some practical proposal-reading tips.

 

So what should you look for as you read through these proposals? Here are some tips that will help you focus on the most important content.

 

1. Avoid comparing solely on price. When comparing proposals, the temptation is to immediately flip to the last page and compare fee estimates. Is Company A proposing $50,000 and Company B proposing $100,000? Don’t dismiss Company B right away—it might simply be using more realistic pricing assumptions.

As a general rule, if the proposed prices vary by more than 20 percent, you can be certain that the agencies used different assumptions. Pricing will vary if they used different assumptions about the difficulty of finding qualified respondents (also referred to as incidence rate), required deliverables, or the types of data analysis required. Perhaps one agency built in some analyses that would be overkill for you. Perhaps another assumed you were providing the sample and as a result proposed a markedly low fee.

There’s certainly nothing wrong with calling agencies back and redirecting any incorrect assumptions, but why waste the time? If your RFP is precise, you should get fee estimates in the same ballpark. If your RFP is precise, and fees are still really divergent, ask the high and low bidders to comment.

 

2. Look at the biographies carefully. It’s always a good idea to compare proposals based on the descriptions of the team bios submitted. A reputable MR firm will tell you something about who’s going to be assigned to your project, should it be selected. Read the language carefully: Is it sending you the bios of representative team members or those of the professionals it actually intends to assign?

Some large MR firms have reputations for bringing in their big guns (senior consultants and VPs) to sell a project, but then relegating actual day-to-day project management to a junior-level person. If your project is simple, you might be better off with more of a junior- to mid-level person because their daily rates are going to be lower, so the research will cost you less. But if you are doing something that’s very significant, then you want senior-level project management.

As an aside, the bait-and-switch, when it happens, is often unintentional. If you send out an RFP in January, but don’t select a vendor until April, there is a good chance that the initially assigned consultants will be booked with other clients.

Here are a few other things to look for as you review bios:

• Are the bios from full-time employees or subcontractors? Using subcontractors might not be a bad thing, but it does say something about a firm’s in-house expertise and ability to handle project loads.


• Do the bios indicate what other clients the team members have worked with? Have they worked with clients in your space? Do they work with your competitors?


• If they do work with your competitors, don’t dismiss them right away. After all, this may mean they have a lot of relevant contextual knowledge from which you can benefit. Of course, you should ask about conflicts of interest and how they would protect your confidential information. One option: simply ask them to not engage with your competitor while they are engaged with you.


• Do the bios show relevant expertise? How many of the proposed team members have worked on similar projects in the past? For example, if you are doing a brand awareness project, what relevant experience is highlighted in the bios?


• How long have team members been with the agency? Have the key members been with the agency for at least three years? That shows that they have a track record with that agency’s processes and best practices. After all, if you buy into a sales pitch that includes this firm’s approach to, say, project management, you will want someone managing the project that has been applying those project skills for more than a year.

 

3. Reality check the proposed timeline. If one agency is proposing a timeline that is markedly shorter than anybody else’s, it’s a red flag. Of course, markedly longer is a red flag as well. But in reality, I’ve seen more clients get burned by agencies that proposed an unrealistically short timeline.

Typically, if one agency’s timeline is significantly shorter than everybody else’s, it’s making overly aggressive promises to win your business. Of course you want the MR data tomorrow, so the promise of a short timeline is enticing. But the truth of the matter is that even in the world of online research (which has quicker data collection than other options), a high-quality primary MR project still takes at least four to six weeks.

Think about it. At minimum, you’re going to need two weeks for questionnaire design or moderator guide development alone. And if you have 2 or more internal clients who need to be in on approval cycles or a legal department that has to okay the content, it can easily be more. Rushing through that process is going to cause heartache for everyone at the end of the project. So if a bidding agency proposes a timeline that seems just too good to be true, it probably is. You really need to ask them what their assumptions are for each phase of the project.

Do exceptions exist? Yes. Online or phone surveys can be done in a day or two, provided most of the following conditions are met: if it’s a consumer study; if you have a standard questionnaire that you are fielding; if you literally have five or six questions; if you have a great sample source that can execute in 24 hours; if you are buying into an ongoing survey program (referred to as omnibus surveys); and if you don’t need any real reporting—just the data.

 

4. Review the project management approach. Compare proposals based on how the firms talk about project management.

Do they really seem to understand the importance of managing the project on a day-to-day and week-to-week basis? How do they describe the kickoff process? Do they advocate for in-person kickoffs, or do they seem to prefer a more casual approach? Do they state that they’re going to provide weekly written status memos? How do they say they’re going to track key milestones? Are they planning to have weekly status conferences with you, the client? Although the level of project management required varies by project complexity, it’s always good to ensure that a potential supplier’s approach will align with your expectations.

 

5. Look for mentions of QA. Do they mention QA in their proposals? What kind of process have they built in for doing quality checks throughout the process, not just at the end once the data are collected? What information or best practices do they cite in the proposal that addresses quality at key junctures?

If you ask, “What steps do you take to make sure the questionnaire is programmed correctly?” how do they respond? This is a completely valid question. Imagine if you were just done with data collection, only to find that due to a programming error, a key question was displayed with the wrong answer options? Ouch!

 

6. Highlight any content that seems odd or wrong. MR proposals should pass a basic gut check: Does the proposal seem honest and believable? Is the price too good to be true? Are any statements unclear? You should ask about any content you are unsure of.

 

7. Beyond boilerplate. I always prefer a proposal that shows some original thought as opposed to just regurgitating what was in the RFP and adding in some boilerplate. Some MR firms overuse boilerplate. Using boilerplate doesn’t mean a proposal is bad, and it doesn’t mean the firm is not a good company. But won’t you feel better reading a proposal that was clearly written for you, addressing your needs?

 

8. Value add (if you are looking for a full service agency). Look for any evidence that suggests that the MR firm is going to add real value to the project, as opposed to just moving through a process in a robotic fashion. For example, has it identified in the proposal what it thinks the greatest risks are in doing this project successfully and identified what it thinks it can do to mitigate those risks? Has it made it clear that it understands the critical success factors that will make this project successful for you and the rest of your team upon completion? Has it shown that it understands the topic by identifying some preliminary hypotheses that could be relevant to the study?

These things show that (a) it cares enough about the work to really put some thought into the proposal, (b) that it is going to be a partner that is going to add value to the process, and (c) that it really understands your business and will be able to speak credibly with your internal colleagues.

 

This is an excerpt from the book, "How to Hire & Manage Market Research Agencies," which is available on Amazon. Published by Research Rockstar LLC. Copyright © by Kathryn Korostoff. All rights reserved.