Why it's time for research to move beyond UX

Over the past year, we’ve seen massive layoffs in the tech industry, and alongside those layoffs have come a variety of thought pieces about the future.

Some have even called it a “reckoning”, and while they all include a vision of the future, I’ve yet to see one that addresses a fundamental issue: our current org structures are not setting us up for success. That’s why I’d like to propose a schism: it’s time to separate Research from User Experience.

UX Design has served as an excellent landing spot for Research as the discipline has transitioned out of academia over the past 30 years, but if we’ve learned anything from all of the thought pieces, it’s that Research has outgrown this cradle. How do we get from where we are to where we want to be? Why is Research part of Design in the first place? Let’s dig in.

How did we get here?

We’re not going to cover the entire history of Research in business (Jakob Nielsen has an excellent overview of the history of UX which goes all the way back to the 1950s!). Instead, let’s start in 2010, where the UX profession really began to take off. Nielsen’s essay estimates the number of UX professionals grew by nearly a factor of 10 from 2010 to 2020:

According to an August 2023 report analyzing 10 years of Indeed Data, we can also see that the share of UX jobs is shifting from predominantly Design roles to a more equal ratio of Researchers:

This data supports the most common response to a question posted on Learners earlier this year: What are the arguments for Research reporting into design vs. not?

• Noam Segal: the biggest reason is because design is hired first.
• Saeideh Bakhshi: designers are hired first and have higher ratios than researchers.

We’ve also seen a lot of people arguing that Research has been harder hit by layoffs. That claim isn’t supported by data, but why would any or all of this be true? If UX has grown overall, why did Design take a disproportionate number of the roles, leaving Research to catch up later? As with most things, there’s likely a variety of factors at play, but I think it boils down to something pretty simple.

Companies invest in Product. Product invests in Design. Design invests in Research.

What I mean by this is simple: when you’re scaling up a product organization, the first thing you need are people who can build the thing you’re trying to provide (Engineers). You also need people who can sell the thing you’re building (Sales).

Small organizations with tight feedback cycles and narrow scopes make it easy for the Sales team or the cofounders to act as the de facto Research function, gathering insights from early adopters and making adjustments.

As the organization scales, though, product scope expands and becomes more complicated. Engineers often recognize their limits and will advocate for a Design hire to help make the product more user-friendly. Those early Design hires are generalists, and fulfill a hybrid role spanning across the entire UX discipline: research, interaction design, visual design, motion design, content design, etc. As the product continues to grow, the need for specialization emerges. Usually a co-founder will play the role of Product leadership up to this point, which they’ll then invest in dedicated Product roles. Product and Design will likely carry on doing their own Research for a while, but eventually recognize their limits the same way Engineering did earlier. Only then do they finally invest in Research.

This process usually takes years to unfold at companies. (1010data was founded in 2000, I was the first Director of UX in 2019; Auth0 was founded in 2013, I was the first Head of Research in 2020.) While some manage to survive, the number of uninformed decisions made along the way is staggering. This is undoubtedly a contributing factor to the ~90% startup failure rate. How many of those could’ve been avoided by investing in Research earlier? 🤔

Why this is bad

You might be thinking that if everyone does it this way, it must be the best way. There are certainly benefits to Research being tightly integrated with Design and Product (notably, being closer to the product and the people that make it happen). But overall, investing in Research late and pigeonholing it into the Design team is harmful both to the organization and to the discipline as a whole.

Research, at its core, is a de-risking activity. Our job is to discover and share insights about customers, users, and the market in order to uncover new opportunities and increase clarity for upcoming decisions. If that’s the mission statement of your Research function, it makes no sense to limit it to one part of the organization. Teams all over have questions about current, potential, or recently abandoned users, and we as Researchers have the ability to answer those questions. Cross-functional collaboration is something that’s gained more attention recently, and while it’s better than completely ignoring potential partners in other departments, it’s not the solution.

It doesn’t matter how cross-functional you try to be: at the end of the day, your reporting chain controls your budget and will have an immense impact on your priorities.

I also believe that being confined to Design sells Research short. We’re being held back by having our focus only directed at what’s right in front of us: the product and the designs that shape it. The research you and your team does has the ability to impact your entire company and even the entire industry vertical you work in if it’s properly planned, executed, and leveraged. The limited scope we get stuck with not only harms the company; it shortchanges the investment we’re getting. Without having an impact on the entire business, it’s hard to justify funneling resources toward Research.

But if we aren’t happy with the current regime, what should a new one look like?

What needs to change

I’m going to start calling this The Era of Just Research. What I mean by that is we need to drop the prefix to the work that we do, because I think it does more to pigeonhole and divide us than it does to serve us.

Instead of calling ourselves UX Researchers, or Market Researchers, or Product/Design/User Researchers, I’ve recently just started calling it “Research”, and saying it’s my team’s job to help the company make better decisions by gathering insights about the people we’re interested in providing solutions to. Sometimes this takes the form of “Market Research”, where we’re focusing on overall trends and the habits, behaviors, interests, and needs of people who don’t even know our company exists yet. Sometimes it’s analyzing the behavioral and motivational patterns exhibited when experienced users interact with a new design (“UX Research”). What if we expanded this further? We could research our partners, sales leads, even our own coworkers, and provide intelligence to the entire organization.

By broadening the scope of the work we see ourselves capable of and responsible for, we lay the groundwork for deepening the impact we can have at an organization.

While this may sound like a panacea, it won’t happen until we control our own destiny. Intelligence gathering needs to move out of Product, Marketing, Customer Success, Engineering, or any other discipline it’s historically been part of. It’s time to create Intelligence functions, made up of (User) Researchers, Data Scientists, Competitive Intelligence, Growth Marketers, Pricing Research, etc. It’s time to start thinking more holistically about the questions we’re trying to answer for the business, including the methods we use and how the research we do can empower multiple teams within the organization.

How do we get there

There are already too many lofty, visionary essays out there that don’t provide any practical considerations. I won’t add another — here are a handful of things you can start working on right now regardless of your role or standing within your organization.

Establish a cross-functional working group

You don’t need to be in a management/leadership position to build relationships with your peers in other intelligence-gathering disciplines and get their buy-in to establish a working group. At Auth0, alongside leaders from Data Science, Product Marketing, Competitive Intelligence, Pricing & Strategy, and Customer Success, I established the Customer Intelligence Alliance (also known as the CIA 🕵️‍♀️). This biweekly meeting helped us stay on top of work that was happening in virtually all parts of the business (Data Science lived in Engineering) and identify opportunities for our teams to collaborate for deeper insight and impact.

Think outside the UX box

Once you stop thinking of yourself as just a UX Researcher, you can start to identify ways you can help other teams. Talk to Marketing, Sales, and Customer Success to understand their needs, challenges, and questions. Build trust across the entire organization and earn allies by providing value for them.

You can then spend that trust and social capital later: despite reporting up through Product, Auth0’s 2023 recruiting and incentives budget was funded by Marketing because we were able to fulfill part of the research they planned to outsource.

Another thing that will help: we all need to become better mixed methodologists. Brush up on your qual/quant skills, take a statistics/survey design course, and design more complex research. This will make it easier for you to identify ways for your research to serve more than just your immediate need and increase your credibility among colleagues in other departments.

We can’t get there alone

Until “Researcher” is the most common origin story of CEOs, we won’t be able to make these changes ourselves. We need support from leadership, and unless you have particularly visionary and forward-thinking leadership, they need our help to understand the value we can provide. Researchers need to get better at managing up, as it’s often more important than managing sideways or down. The larger and more entrenched an organization becomes, the more likely they are to do the conservative thing — what everyone else is doing. Help your company break that mold.

Part of managing up is understanding the impact you can have on the business. What are your company’s Key Performance Indicators (KPIs)? How can Research support business objectives and have an impact on the top or bottom line? There’s no shortage of existing discussion about the ROI of UX Research, but the more teams your work impacts, the less anyone ever calls that into question. 

Final thoughts

The transition from academia to industry is a challenging one, whether that be for individuals making the jump into a UX career in or the entire field as a whole. If Researchers are being called upon to provide greater impact and more value than we have historically, expecting the current way of working to get us there is questionable at best. We need to evolve, not only as individuals, but as a discipline, and that evolution means moving Research beyond Product or Design.