Creating a stellar user experience (UX) is a literal goldmine for your website or app.
Research shows that for every $1 invested in UX, there is a $100 return—an ROI of 9,900%. Additionally, research shows the top companies leading in user experience outperformed the S&P index by 35%.
Blowing customer expectations out of the water with a great user experience pays off—and nailing UX design starts with capturing customer insights via solid UX research.
But how do you do solid research when it’s time-consuming, costly, and often requires help from a savvy researcher?
“Good research doesn’t require a complex study design or lots of data, but it does require clarity around the question you want answered and an intentional and thoughtful design—it’s a learned skill. If an organization doesn’t have a researcher on staff, they should bring one in to provide feedback and support,” says Kristen Hill, Founder & CEO at Kirsten Lee Hill Consulting.
If you know the value of customer research but don’t feel like you have the resources for a full-fledged study or to hire a trained researcher, it begs the following question—is no research better than lousy research?
This article will look at:
- What proponents of “no research is better than bad research” have to say
- What advocates of “any research is better than no research” argue
- How to get started with UX research with limited resources
No research is better than bad research—or is it?
Many researchers believe that no research is better than flawed research. Spoiler alert: the team at Great Question mostly disagrees. But it’s always worth looking at both sides of the coin to generate healthy debate.
Let’s take a closer look at the idea that “no research is better than bad research” and why.
1. Academic researchers often believe no research is better than bad research
There’s a reason you can't build an academic study in a day. Academic researchers have large hypotheses, stringent research requirements, and a strict peer-review process.
Moving confidently forward with a study means following requirements to a tee, including getting informed consent, choosing an accurate sample, using replicable and unbiased research methodologies, and more.
If any step in the research goes wrong, it could invalidate an entire study. For academic researchers, bad research isn’t an option, especially if publication is the ultimate goal.
Another argument academic researchers make is that if any part of the research methodology is flawed, the results are inaccurate and unhelpful.
Dr. Michael Cummings, Assistant Professor of Strategy, Entrepreneurship, and Venture Innovation at the University of Arkansas, explains why.
“I help my students think about problem-solving, generating business ideas, and customer discovery. One of the examples I use when teaching them how to research is the importance of choosing the right audience. If they target the wrong audience, it doesn’t matter what questions they ask because the results won’t matter. Similarly, if my students identify the right people to talk to, but they ask the wrong questions, it’s equally as unhelpful. When a researcher gets one part of the study wrong, the answers will be irrelevant or wrong, and that’s a waste of time,” says Professor Cummings.
2. There are risk associated with bad research
It goes without saying that there are risks associated with bad research. Some potential risks include:
- Not targeting the right audience
- Prioritizing the wrong group of consumers
- Research bias
- Data inaccuracy
- Poorly analyzed data
- And more!
According to Professor Cummings, one of the most significant risks of flawed research is having faulty or biased results informing decision-making.
“If you provide stakeholders with information that is backed by incorrect or biased data, it can negatively affect decision-making and lead your organization down the wrong path,” according to Professor Cummings.
Kristen Hill echoes this sentiment when she says, “I believe that no research is better than bad research, but also that there is no excuse for bad research. With no research—the message you are sending is ‘we don’t know, yet’ or ‘this is something we need to look into further;’ with bad research, you risk making poorly informed decisions and/or further spreading bad (wrong or overstated) information and misleading people.”
Harri Thomas, former co-founder of research recruitment platform Respondent shares a similar message.
Where research can be dangerous - and may risk falling into the ‘bad research’ category - is unlikely to be in the research itself, but rather in how the research is used to effect change afterwards. If a researcher has an agenda they're looking to realize, they can cherry pick customer data that proves their point and use it to silence alternative perspectives.
3. Ethics may be compromised in bad research studies
Another common argument is that no research is better than bad research when a study involves compromised ethics to generate specific results.
Filiberto Amati, Partner at Amati & Associates, provides an interesting example.
“Researchers compromise ethics when they take shortcuts, and they prioritize allegiance to the client and not to the study and methodology. A client may conduct a study with certain expectations in mind, and this can place pressure on the researcher,” says Amati.
He continues, “The researcher’s job is to build a sound method and execute it, and then provide accurate results, whether the client might like it or not. When researchers prioritize client expectations over true results, ethics are compromised.”
Any study where ethics are compromised is probably worth skipping altogether.
Here’s what we think—any research is better than no research
In some contexts (e.g., academic research or intentionally biased studies), it makes sense that no research would be better than bad research. But, no matter where your company falls in terms of UX research know-how, starting somewhere is better than not starting at all.
User experience research supports this idea. Consider the following stat—companies that were less invested in UX (they considered themselves simply “design-centric”) still saw their sales increase by 60%.
This means even companies that do minimal UX research still see fantastic results. Here are some additional reasons why participating in some UX research—even if it’s bad—is better than conducting no research at all.
1. Any user insights help you build a better website or app
More and more people are starting to understand the value of customer research. In fact, 70% of enterprise CEOs see UX & CX as a competitive differentiator.
As such, more organizations are engaging in customer research. Demand for UX research has increased by 30% in the last year, and 73% of companies not currently conducting user testing will start in the next year.
But, here’s the thing. While UX and CX research are critical to helping organizations stand out, not every organization has a team of trained data scientists at its disposal.
And that's okay.
With the advent of UX and customer research technology, it’s possible to arm a novice researcher with the right tools that follow proven methodologies, provide training, and make it possible to create polished UX research studies.
Shonavee Simpson-Anderson from Firewire Digital offers another good point about why bad research is better than no research.
Simpson-Anderson says, “When it comes to website design and digital marketing, any research, even bad research, is better than no research at all. The risk of bad research is that the research doesn’t provide you with the result you want, but the upside of that is you have discovered something that doesn’t work and now know to try something else. This, of course, leads to more research. In this way, even bad research leads you to do basic testing.”
Chief Experience Officer at HelpScout, Mariah Hay, expounds on this idea. I think people say ‘bad research is better than no research,’ because they worry that poorly executed research will lead teams and companies to make poor decisions based on the faulty conclusions the research could potentially lead to. But when you look at the alternative, which is not doing any research and completely guessing at those decisions, that premise breaks down,” says Hay.
She continues, “I once heard someone say that anything worth doing, is worth doing poorly. Which means that it’s better to at least make an attempt and gain a little ground, than none at all. For example, if I am trying to improve my health, eating more nutritious meals one day a week is an improvement over eating them no days a week (even though I should be eating them 7 days a week). Attempting to improve my health is worth doing, even if I do it poorly. The same goes for research.”
2. You have to start somewhere
While there are risks associated with poor research, it’s also worth mentioning there are also risks of doing no research at all.
“The risk of flawed research is that you know Concept A and B exist, but you do not understand their boundaries, and the insights are less actionable. However, poor research beats no research all the time,” according to Filiberto Amati, Partner at Amati & Associates.
The reason? While a poorly designed UX study may result in less actionable insights, doing no research can result in complete inaction, and that also won’t help your organization.
Caroline Lee, Marketing Director and Co-founder at CocoSign, agrees that bad research is better than no research, and the risks of no research outweigh the risks of bad research.
Lee says, “The risk of performing no research is delivering a solution that does not meet user needs and requirements. Bad research may lead to some failure to meet users’ expectations. However, any attempt at research helps you understand your customers’ requirements. No research means you are creating a solution and expecting it to land the right audience by chance.”
If you want a chance at learning what your customers and users think about your products and user interface, you have to ask them. Even if you’re not sure where to start.
Emma Travis, CX Strategist at Speero, has great advice for organizations that want to start researching but don’t have the resources to hire a full-fledged research team.
“If you’re not in the position to recruit experienced researchers but want to start running research, pay special attention to how you set up the research, consider how to emulate a genuine user experience as much as possible and read up on how to ask effective non-leading questions,” says Travis.
By getting your team to do some research, they will start to engage with research tools, familiarize themselves with proven methodologies, and improve over time.
3. UX research doesn’t have to be complicated
Remember how we talked about how complicated academic research is? Here’s the good news—user research isn’t as complex.
Stats show that 85% of UX problems can be solved by testing only five users.
The risks involved in user research are minimal, especially when you only need feedback from a handful of users.
While it’s true there will always be a place for dedicated, highly experienced UX researchers, especially for strategic initiatives, there is also an ongoing need for small, more tactical research studies.
And you can easily delegate these smaller usability studies to UX designers and other team members.
Emma Travis, an experienced CX strategist, says when you are doing some UX research, “you are at least taking a step towards customer-centricity. The act of doing user research means you acknowledge that your users’ feedback is important and insightful and is better than your opinion or guesswork.”
Travis offers expert insight into another way to make customer research easier. She says, “If you can only run a small piece of research, then consider what other data or insights you can gather to support your insights to reduce the risk of basing decisions on small samples, for example.”
Ready to get started with UX research? Check out Great Question
I'll close out with these words from Harri Thomas:
Bad research is really about bad actors.
If you assume good intent in your colleagues, then I’d re-allocate time worried about bad research towards encouraging research, and having team mates adopt research as part of their process. This is a much harder challenge and a better use of your time.
Focus on democratizing research first, then you can quality control.
Great Question has been built from the ground up to help UX research professionals spend less time on operations, and more time on the research itself. We also make it easy for these teams to democratize access to research by providing access controls, free observer accounts and - coming soon - sub-panels of customers to research with.