Marketing Research’s Biggest Challenge Is Not the Data

One of the most significant challenges in marketing research is not collecting data. It is knowing what question you are actually trying to answer before you collect anything. Most research projects fail not because the methodology was wrong or the sample was too small, but because the team started gathering before they had decided what decision the research was supposed to inform.

That gap, between data collection and decision clarity, is where most research budgets quietly disappear. And it is far more common than the industry likes to admit.

If you are building out a research function or trying to get more commercial value from the work your team already does, the Market Research and Competitive Intelligence hub covers the full range of methods, tools, and frameworks worth knowing.

Why Most Research Projects Start in the Wrong Place

Early in my career, I watched a client commission a large-scale brand tracker. Quarterly waves, nationally representative sample, the works. It ran for two years. When I asked the marketing director what decisions had changed as a result of the data, she paused. Then she said something I have never forgotten: “It’s more of a reassurance thing.”

That is not research. That is expensive wallpaper.

The problem is structural. Research is often commissioned by people who are not the ones making the decisions, delivered to people who were not involved in scoping it, and interpreted by analysts who do not fully understand the commercial context. By the time the findings reach someone who could act on them, the window has closed or the conclusions have been softened into something that offends nobody and changes nothing.

This is not a criticism of researchers. Most of the research professionals I have worked with are sharp, careful, and genuinely curious. The failure is almost always upstream, in how the brief was written and what question was actually being asked.

The Confirmation Bias Problem Nobody Wants to Talk About

Confirmation bias is the most persistent challenge in marketing research, and it operates at every level. It shapes which questions get asked, how surveys are worded, which segments get highlighted in the analysis, and which findings get presented to the board.

I have sat in research debrief sessions where the agency presenting the findings had clearly been briefed, implicitly or explicitly, to validate a campaign the client had already decided to run. The data was real. The methodology was sound. But the framing was designed to confirm rather than challenge. When one slide showed a finding that contradicted the preferred narrative, it was buried in the appendix with a note about “sample size limitations.”

This is why qualitative methods, when run properly, can be more valuable than quantitative work on certain questions. A well-moderated focus group does not let you hide from uncomfortable answers the way a survey can. People talk around things, contradict themselves, say one thing and mean another. That messiness is information. It is harder to suppress in a live session than in a data table.

If you are thinking about when qualitative approaches make sense and how to run them without the common pitfalls, the piece on focus groups and qualitative research methods is worth reading alongside this one.

The discipline required to run research that genuinely tests your assumptions, rather than validates them, is harder than it sounds. It means being willing to find out you were wrong. Most organisations say they want that. Fewer actually tolerate it when it happens.

The Speed vs. Precision Trap

There is a tension in marketing research that does not get discussed enough: the gap between the speed at which marketing decisions need to be made and the time required to conduct research properly.

When I was running performance marketing at scale, managing substantial ad spend across multiple verticals, we were making campaign decisions weekly, sometimes daily. A properly scoped quantitative study takes weeks at minimum. A brand tracker takes months to build a useful baseline. By the time the research arrives, the market has moved.

This is where many teams make a critical error. They either abandon research altogether because it feels too slow, or they commission research at the wrong level of rigour for the decision they are making. You do not need a statistically significant nationally representative sample to decide whether to test a new ad format. You do need something better than gut instinct to restructure your entire go-to-market approach.

The organisations that handle this well are the ones that have developed a clear sense of what level of evidence different decisions require. Directional confidence is enough for most tactical choices. Strategic decisions warrant more rigour. Treating every question as if it needs a PhD-level methodology is as damaging as treating every question as if a quick poll will do.

Understanding your customer at a structural level, including how to define and score who your best customers actually are, changes how you frame research questions entirely. The ICP scoring rubric for B2B SaaS is a useful reference point for how to make that customer definition concrete rather than vague.

When Research Becomes a Political Tool

This one is uncomfortable to write, but it is real. Research is frequently used inside organisations not to inform decisions, but to legitimise decisions that have already been made. Someone senior has decided on a direction. Research is commissioned to provide cover. If the findings support the decision, they are cited prominently. If they do not, the methodology is questioned.

I have seen this pattern across agency-side and client-side roles. It is particularly common in larger organisations where decisions involve significant internal politics and where being wrong publicly carries career risk. Research becomes a shield rather than a compass.

The damage this does is twofold. First, it wastes the budget and time spent on research that was never going to change anything. Second, it erodes confidence in research as a discipline. When teams see findings ignored repeatedly, they stop believing research is worth doing. That creates a vacuum that gets filled by whoever shouts loudest or has the most senior title.

The fix is not a methodological one. It is cultural. It requires leadership that is genuinely willing to be surprised by what the data says, and that models that willingness visibly. Without that, no amount of research sophistication will help.

The Data Availability Illusion

We have never had access to more data. That is not the same as saying we have never had better insight.

The explosion of available data sources, from CRM systems to social listening tools to search intelligence platforms, has created a new challenge: teams are drowning in signals they do not know how to prioritise. When everything can be measured, the skill is not measurement. It is knowing which measurements matter for the specific decision you are trying to make.

Search data is one of the most underused research assets available to most marketing teams. What people type into a search engine is one of the most honest signals of intent and need that exists, because nobody performs for a search bar. Understanding how to read that signal properly, rather than just using it for keyword targeting, is a genuine research skill. The piece on search engine marketing intelligence goes into this in more depth.

There is also a category of data that most teams have access to but systematically underuse: the grey areas between official market data and primary research. Industry reports, regulatory filings, patent databases, job postings, pricing pages, and distributor behaviour all tell you things about market dynamics that never appear in a commissioned study. Grey market research is one of the most cost-effective ways to build competitive intelligence without commissioning expensive primary work.

The challenge is not access. It is analytical discipline. It is the ability to look at a dataset and ask: what decision does this actually help me make? If the answer is none, the data is not useful regardless of how clean it is.

The Gap Between Research and Commercial Reality

One of the most consistent frustrations I have encountered across two decades of agency and client-side work is the gap between what research recommends and what the business can actually do. Research findings arrive with implications that assume unlimited budget, complete organisational alignment, and the ability to pause everything else while a new strategy is implemented.

Real businesses do not work that way. They have legacy systems, fixed contracts, teams with existing skill sets, and boards that need convincing. Research that does not account for those constraints is not wrong, but it is incomplete.

I built my first website myself because the MD said no to the budget. That experience taught me something I have applied ever since: the most useful analysis is not the one that tells you what the ideal world looks like. It is the one that tells you what you can do with what you actually have. Constraints are not obstacles to good strategy. They are the conditions within which good strategy has to operate.

This is particularly relevant when research is being used to inform technology or operational decisions. A SWOT analysis or strategic alignment exercise that does not account for implementation reality produces recommendations that look good in a deck and go nowhere in practice. The technology consulting and business strategy alignment framework is a useful lens for thinking about how research translates into decisions that can actually be executed.

The researchers and strategists who are most valued commercially are not the ones who produce the most sophisticated analysis. They are the ones who understand the difference between what is theoretically optimal and what is practically achievable, and who build that understanding into their recommendations from the start.

Pain Points Are Harder to Research Than Most Teams Assume

One of the specific research challenges that comes up repeatedly in B2B and services marketing is the difficulty of accurately identifying customer pain points. The obvious approach, asking customers what their problems are, produces answers that are often incomplete, politely framed, or shaped by what the customer thinks you want to hear.

People are not always able to articulate what frustrates them about a product or service. They will tell you the surface-level complaint. The underlying issue, the one that would actually change their behaviour if you solved it, is often several layers deeper. Getting to that requires research design that goes beyond direct questioning.

Indirect methods, observational research, analysis of support tickets, churn interviews, and competitive displacement conversations, tend to surface more honest and more actionable pain point data than a satisfaction survey ever will. The marketing services pain point research framework is a practical starting point for teams trying to get beyond surface-level customer feedback.

At lastminute.com, we ran a paid search campaign for a music festival and generated six figures of revenue in roughly a day from a relatively simple campaign. What made it work was not the execution. It was the clarity we had about what the customer actually wanted: a fast path to a specific experience, with minimal friction. That clarity did not come from a formal research process. It came from paying close attention to search behaviour and conversion patterns. Sometimes the most valuable research is the kind that happens in the margins of other work, if you are paying attention.

Writing that connects with a specific pain point also depends on understanding the language your customers use, not the language your internal team uses. Persuasive copy is built on that gap between how brands describe themselves and how customers actually experience their problems.

What Good Research Practice Actually Looks Like

After two decades of watching research work and fail across dozens of categories, a few consistent patterns separate the teams that get genuine value from research from those that do not.

The first is decision-first scoping. Before any methodology is chosen or any data is collected, the team must be able to complete this sentence: “This research will help us decide whether to…” If that sentence cannot be completed clearly, the research is not ready to start.

The second is pre-commitment to action. Before the research is fielded, the team agrees on what they will do if the findings come back in each possible direction. This sounds bureaucratic but it is significant. It prevents the political manipulation of findings after the fact, because the actions were agreed before anyone knew what the data would say.

The third is proportionate methodology. Not every question needs a large-scale study. Not every insight requires statistical significance. Matching the level of research rigour to the scale and reversibility of the decision is a commercial skill that most teams undervalue.

The fourth is honest interpretation. This requires people in the room who are willing to say “this finding challenges what we assumed” without immediately reaching for reasons to discount it. That is a cultural requirement as much as a technical one.

Organisations that want a broader framework for thinking about research quality and competitive intelligence will find the full range of approaches covered in the Market Research and Competitive Intelligence hub.

The challenge of marketing research is not technical. The tools exist. The methods are well understood. The data is more available than it has ever been. The challenge is human: the willingness to ask hard questions, design research that might produce uncomfortable answers, and then act on what you find rather than on what you hoped to find. That has always been the hard part. It still is.

About the Author

Keith Lacy is a marketing strategist and former agency CEO with 20+ years of experience across agency leadership, performance marketing, and commercial strategy. He writes The Marketing Juice to cut through the noise and share what works.

Frequently Asked Questions