User experience (UX) researchers do a lot. It’s the dynamic field behind product, design, and marketing decisions at the world’s most innovative companies, like Google, Figma, and Spotify. But they don’t do it all alone. In today’s hypercompetitive landscape of efficiency and scale, a specialized practice and role has emerged to help researchers execute effectively: research operations.
"Somewhere in the last five years, research operations has started to be recognized in the same way design operations is – as a strategic function with a totally distinct set of skills from research and design. And it's really about strategically enabling research to operate efficiently and impactfully in organization,” said Jane Davis, Principal Researcher at Great Question.
In our complete guide to research operations, we’ll cover:
Along the way, we’ll help you assess and optimize the various areas of ReOps in your organization and share helpful resources to learn more.
Let’s get started.
UX research operations – ResearchOps or ReOps for short – is a specialized part of design operations that focuses on the processes, tools, and strategies that support the execution of research. The goal of an effective research operations program is to magnify the impact and value of UX research in an organization, giving researchers a seat at the table to ensure the voice of the user is at the center of every product release and ultimately help improve business outcomes.
The ResearchOps Community offers an official definition:Â
“ResearchOps is the people, mechanisms, and strategies that set user research in motion. It provides the roles, tools and processes needed to support researchers in delivering and scaling the impact of the craft across an organization.”
The community’s founder, Kate Towsey, who is also the author of Research That Scales: The Research Operations Handbook, said during her keynote presentation at People Who Do Research 2023:
“Research operations is a combination of two things: giving people ways to find out and giving people ways to understand.”
Research operations is a young field nested inside the already young field of UX research. So young, in fact, it started with a tweet.
At the forefront of the field is the ResearchOps Community, founded by Kate Towsey five years ago. This group of research leaders began holding workshops in 2018 and has since grown the community to more than 16,000 strong across over 100 countries.Â
The community studies best practices for the evolving research operations function, led by an active Board (known as “The Cheese Board”) that host workshops and conferences. At the center of it all is a thriving Slack community and even an up-and-coming podcast.
To map research operations and give shape to discussions about the field, the ResearchOps Community created this now widely-used framework below. (You’ve probably seen it before.)
A well-run research operations program can be the difference between making informed company-wide decisions and guessing in the dark. While ReOps pros typically don’t conduct the research itself, they do play a pivotal role in setting researchers up for success every step of the way.
"Every research operations person I have ever met genuinely just wants to make things work better,” said Davis. “They want systems to work for humans in them. It's not that they want the system to work perfectly, it's that they want the perfect system for the people that they're dealing with.”
What does this look like in practice? On any given day, you might find a research operations manager:
"The misunderstanding of research operations is people still assume it’s just recruiting and coordinating,” said Davis.
To best support UX researchers, ReOps pros also need to have a thorough understanding of research methods and best practices. (More on methods soon.)
Related read --> How Canva’s Research Operations Manager builds user-centricity securely and at scale
Without the right research operations infrastructure in place, building a panel and recruiting participants can easily go sideways, overrun by tedious tasks and manual labor. It’s ReOps’ job to make sure this doesn’t happen.
In this section, we’ll cover the various phases of the panel management and recruiting process – and where research operations fits in.
The purpose of a panel management tool is to optimize the process of building a panel and recruiting participants. Adding a participant management tool should result in faster turnaround and lower costs while using fewer tools altogether. It should make it easier for researchers to recruit for themselves or to have someone recruit for the team. Â
Benefits of a panel management system include:
Maybe you want a participant management tool that could be used by non-researchers like designers and project managers. Maybe you don’t have the budget and need to establish a system with tools you already use. Whatever your situation is, assess what your organization needs most and prioritize accordingly.Â
Once you establish a participant management system, you’ll want to create a panel with high-quality candidates. You’ll need to make some decisions: Will your database be your own customers, pulled from Snowflake or Salesforce, or an external panel of people who are relevant to your work? Either way, you’ll want to set parameters about who to invite, give people the ability to opt-in and out, and be thoughtful about privacy and data storage.Â
Great news – you’ve got research participants. But how do you keep them happy and engaged?
You'll need a quick, easy way to schedule and conduct research, and send incentives. You also need to consider how to securely handle customer information. Answering these questions can help you see where you stand and take action to improve as needed.
Both you and your participants will benefit from a centralized ReOps system. Look for scheduling, research, and video hosting tools that work together to ensure no one slips through the cracks. Ideally, your scheduling tool will automatically update your database of participants so you can see what’s happening at a moment’s notice.
Your panel likely already has attributes you can use to filter and find your target audience. You can further narrow your audience by inviting eligible candidates to take a screener survey. It’s just as important to consider who you are excluding, so make sure you are including diverse perspectives in your participant pool.
Avoid reaching out to too many people by understanding how engaged your audience is. Look at past email metrics to get a better idea of your average response rate. The higher the response rate, the fewer people you will need to contact. For example, if your typical response rate is 20% and you want to schedule 10 people, email 50 people to start.
The last thing you want to do is exhaust your pool by overcontacting and annoying participants. Your response rate may change from study to study so be conservative. Some topics will be more appealing than others and the time when outreach occurs can affect the number of responses as well. If your engagement is initially low, you can always reach out to more people. With Great Question, you can send outreach in batches, and if your study fills up, it will automatically stop sending.
You’ll need to keep track of your candidates including who you have reached out to, who has signed up, who ultimately participates, and who needs to receive incentives. For interviews you may need to reschedule, cancel, mark as complete, and record no-shows. To avoid using the same people in your studies you may need to exclude participants for a set amount of time. You also may want to exclude recent contacts so they don’t feel bombarded by requests.Â
What happens when your huge research goals are met with crickets from participants? There are a number of causes behind recruiting struggles. Let’s assess.
Tools and templates are your best friends during research recruitment. Having email templates handy (like these) streamlines outreach and ensures a consistent experience.
You can also use recruitment software like Great Question to create branded recruitment landing pages and manage participant contact info and permissions. Having a single place to log who has opted in to research and when you last contacted them helps you cycle through your audience for a diverse, representative sample.
Incentives help increase research participation and reduce no-shows. They also let your participants know you value their time and feedback. Rewards should reflect the time and effort involved in participating but should not be used to coerce participants to engage in research.Â
Some research teams offer gift cards, discounts, or physical goods like swag bags. Other companies may have a very engaged audience that’s happy to share feedback without compensation. Ultimately, you will need to decide what incentive type and amount is right based on your study type, length, budget, and engagement.Â
Some standard recommendations:
The harder it is to find a user, the greater the incentive should be. You’ll also want to offer more for high earners and in-person sessions as they are more inconvenient. Even for a shorter session, a $20 minimum is ideal to make it worth most participants’ time.Â
Short surveys typically offer no reward. Longer surveys may provide a direct reward or an opportunity to be entered into a raffle. When raffling prizes, be aware of the legal implications. Many states and countries have specific laws related to gambling that may affect your raffle.Â
A key part of incentive management is tracking spending and staying within budget. If you are in the United States, payments exceeding $600 in a calendar year require a W-2. Tracking spend per participant will avoid overpayment resulting in a need for tax documentation.Â
Many participant management systems offer incentive distribution. There are also incentive distribution services available that offer various reward types in different countries.
Related read --> How do incentives impact bias in UX research?
UX researchers are trained to run a wide range of UX research methods to study user behavior. No matter the method, the end goal remains the same – to build a better user experience, and ultimately, improve business outcomes. Part of achieving this goal is leveraging the power of research operations.
Now, you don’t need to be a researcher to understand ReOps. In fact, you don’t even need to be a researcher to work in ReOps. But you do need to understand researchers. How they work and how they spend their time. What’s hard for them and what motivates them. The part of work they love and the part they loathe. And of course, where research operations can help.
So, let’s get comfortable with the types of research methods you’ll encounter.
Throughout the various methods we discuss below, you’ll hear two overarching schools of research referred to time and time again: qualitative and quantitative.
Qualitative research is the process of gathering data by directly observing participants while they use a product or service. The purpose of qualitative research is to discover the why behind a user’s thoughts, motivations, pain points, and attitudes. In turn, these findings can be leveraged to make smarter decisions across product, design, marketing, and other areas of impact within an organization. Because qualitative research is based on observing the behavior of a small sample of users, it’s more subjective and open to interpretation than quantitative research. Examples of qualitative user research included moderated customer interviews, focus groups, field studies, and card sorts.
Whereas qualitative research is after the why influencing user behavior, quantitative research zeroes in on the what, where, and when. Quantitative research is the process of indirectly gathering numerical data – typically from a research software tool – to study how users interact and engage with a product or service. This allows researchers to collect large, statistically significant samples of data that can be used to set benchmarks for performance and forecast growth. Examples of quantitative user research are surveys with close-ended questions, product analytics, and usability metrics like time-on-task and conversion rates.
A user interview is an in-person or remote one-on-one conversation between a researcher and a participant. Moderated user interviews are a great way to learn about a participant’s habits, views, emotions, and daily routines, or walk through a prototype or website.
Examples of interviews that UX researchers may conduct with users include:
Ready to run your own interviews? Swipe free user interview templates from our library to get started.
Once a product concept is defined and the user interface (UI) is being built, UX research teams often conduct usability testing. It gives them a chance to identify design issues and opportunities for improvement before the product is released. Usability testing is also used to continuously optimize products post-release.
There are two main types of usability to know:
Unmoderated usability testing is a research method in which the participant performs a set of tasks without the guidance of a moderator. Typically, the research participant is asked to walk through a prototype or website to demonstrate how they would naturally use it.
To run unmoderated user tests, researchers use software that prompts the participant with tasks and records their behavior. This may include follow-up questions after completing the tasks. Because unmoderated testing doesn’t include direct researcher-to-participant interaction, it’s best for quantitative insights, like measuring task completion times or error rates.
Related: The complete guide to prototype testing
Like unmoderated user testing, a moderated user test requires a research participant to use a product or service to perform specific tasks. The key difference? This time, the researcher observes the participant as they use the product or service in real time, absorbing organic feedback and watching for lightbulb moments.Â
Moderated testing allows researchers to easily ask users questions while they experience the product firsthand and better understand why a user behaves a certain way. This method is ideal when interaction between a researcher and participant is needed, like gathering qualitative insights about user behaviors and the thought processes behind them.
A focus group gathers a group of people – typically five to 10 target users – with similar experiences or backgrounds to discuss a specific topic. Focus groups require a moderator to guide the conversation by asking broad questions.
What makes focus groups so effective for UX researchers? In a group setting, people are often more likely to feel comfortable sharing thoughts, opinions, and behaviors they may feel uncomfortable sharing in a one-on-one research interview.
This research method is powerful, which is why they must be used appropriately. Focus groups are ideal for exploring big-picture concepts to inform the strategic direction of a product or service. Focus groups should not be used to gather feedback on intricate design details.
Ready to run your own focus group? Use this free focus group template!
Researchers use surveys to collect quantitative data on a specific topic and screen participants in the recruiting process. Surveys tend to be cost-effective, good at providing information about large populations, and flexible as they can be administered online, on paper, over the phone, or in-person. Anonymous surveys also allow people to be more candid and open with their responses.Â
Common types of surveys to know include:
When done well, surveys can deliver game-changing insights for organizations of all sizes and industries. When done poorly, they can provide bad data that leads to bad decisions.
If you need survey help, we’ve got you covered. Browse 24 free survey templates here.
Understanding best practices for survey design will result in better data and insights. It will also give you the ability to spot a survey that needs to be edited, or worse – a bad survey in the wild that shouldn’t be used.
When writing survey questions:
When programming surveys online:
When analyzing survey data:
Dive deeper --> The complete guide to survey research
Card sorting is a research method that studies whether or not users are able to find what they’re looking for when using a product. In a card sort, participants are given note cards labeled with topics and asked to organize them into groups based on which topics are related.
There are two types of card sorts:
Observing how users group related topics helps researchers better understand how they think about finding information, like an answer to a FAQ or the cost of upgrading to a plan with more features. It helps product teams identify points of friction and remove them from the user experience.
Card sorting is ideal for the earliest stages of the design process when a product’s information architecture and navigation is being developed – before visual components like wireframes come into play.
A diary study asks research participants to keep a regular journal about their behaviors, experiences, and attitudes involving a product over a period of days, weeks, or even months. Diary entries are self-reported by participants in the form of text, images, videos, or audio.
This method provides researchers the opportunity to learn about a person’s usage habits, pain points, and motivations. For example, a food delivery app like Doordash may ask its power users to record all interactions with its delivery drivers over the course of a month to evaluate their quality of service.
Because diary studies help provide context of a problem space, they are best suited for the early days of product discovery and exploring different use cases of a product.
Read --> The complete guide to UXÂ research methods for 2023
Research teams produce mountains of hard-earned data, insights, and documentation. Understanding what knowledge is already available can help enrich future research and avoid duplicating past projects. That’s why every organization doing research needs a repository.
A research repository centralizes all insights and artifacts under one roof, making it easier to store, find, and share within an organization. A well-managed repository can empower stakeholders to engage more frequently with research and access findings to inform work and strategies. It’s a home for your research.Â
There are many approaches to creating a repository. Some teams use a combination of tools they already have to develop a system. Others will build out a proprietary internal system, or use research repository software. Building a repository requires thoughtful planning, including the careful consideration of cost constraints and goals, before deciding how to best design a system.
One of the most important roles of the research operations team is managing the research repository. Without one, a research practice can’t reach its full potential.
To assess where you stand today, consider the following.
The best repositories are easily accessible across teams. No matter where you choose to store past research, ensure everyone knows how to access it. The more you’re able to collaborate cross-functionally, the more impactful your research will be.
Great Question offers a built-in research repository alongside other participant and research management tools.
Research teams need to understand the ethical and legal implications of collecting and storing data about people. You must establish policies and procedures that limit unnecessary data storage, comply with laws, and respect people’s rights to privacy and to decline participation in research.
Informed consent is a crucial component of research ethics. It’s vital that participants are able to decide whether to participate in research before it begins. Informed consent requires the person to know what the research entails and consent to being involved. You will want to establish a system to get written consent, oral consent, or both from participants.
General Data Protection Regulation (GDPR) is a European Union regulation, designed to improve the data security and privacy of European citizens. Its primary aim is to "give control to individuals over their personal data and to simplify the regulatory environment for international business within the EU". If you’re running research with your users, make sure you understand the implications of GDPR.
The California Consumer Privacy Act of 2018 (CCPA) gives consumers more control over the personal information that businesses collect about them. CCPA regulations provide guidance on how to implement the law.
Personally Identifiable Information (PII) is any information that can be used to identify a specific person. Examples include full names, social security numbers, and emails. It’s best practice to establish guidelines for storage and deletion of PII, as well as to create systems that avoid storing or sharing PII from the outset.
ReOps is responsible for helping maintain research ethics and compliance within a company by ensuring all communication and permissions meet national and local privacy laws. Creating steps in the research workflow to gain participant consent is essential.
You can find templates or contract builders online to kickstart your compliance toolkit. It’s always wise to consult with your legal team to ensure you have everything in order before using forms during research. The ResearchOps Community has a consent form you can use, and you can find non-disclosure agreements (NDAs) here.
Training is crucial for any team. Guides, templates, and training sessions are extremely useful in onboarding and set people up for success. A training library can be a great reference for cross-functional team members at every level.Â
Map out processes to complete specific tasks and create templates for documentation. Templates will create consistency for your team while guides will help people become familiar with process best practices and steps. These tools will give teammates something to refer to and ensure every necessary element is included. While one teammate might be an expert in a particular tool or process, there may be a day when they are unavailable or need someone to step in and help on a particular task. Having guides and templates can save a lot of time and allow others to step in as needed.Â
UX Research teams frequently work closely with non-researchers. These teammates may not have the same background or understanding of research processes and tools. You may want to set up onboarding sessions for all new teammates who may work with research. Introduce them to important concepts and let them know how they can work with research, what tools may be available to them, and where they can review existing insights. Templates and guides may be created specifically for them with their use case in mind.Â
There are many creative ways to build awareness, create buy in, and get people in your organization excited about research. Organizations that understand the value of research are more likely to take action on research recommendations and provide ongoing support for the research team. Here are a few ideas:
While the previous steps focused on running the research process smoothly and continuously, sometimes a new person joins the team. The ReOps lead can help new researchers understand the process and advocate for research across the company.
If the thought of documenting every single thing about your research program is overwhelming (understandable), take it one step at a time. Create notes about your efforts to complete a particular task and repeat them for every research stage. As frequently asked questions about research from stakeholders emerge, you can get out in front of them by adding these to your training.
Templates are a great way to save time and create a consistent experience for your participants. Amanda Gelb, Research Manager and Area Lead at Asana and Professor at NYU, said it best at People Who Do Research 2023:Â
“Everything that can be a template, should be a template here. Remember, you’re the expert. This program lets you share that expertise, so it’s on you to set everyone else up for success.”
Create a library of templates to share with teammates and use as a reference guide for yourself. Consider saving email signatures with templates you use frequently. Here are examples of templates you can create:Â
Email invitations are one of the most used methods for recruiting participants. Basic templates can be modified to best suit your needs, but remember: there are many other considerations for sending an email, from the subject line, sender address, and email design, to the target audience and number of people you contact.
As we mentioned earlier, the ReOps community offers a free consent form generator for those looking to build a written consent template. The form covers major issues like expectations, data collection, privacy, recording, and rights. If you are receiving consent orally, build all the necessary language directly into your script so you don’t forget anything. Â
Sometimes research will require a participant to see sensitive information that is unavailable to the public or considered a trade secret. Create an NDA template that you can edit and send to participants before engaging in research when necessary. You can use a freestanding template or include it in your written consent form. Partner with your legal team if possible to ensure the language is correct and the contract is binding.Â
A research plan template provides the framework for all the essential information needed for a research project. Make sure to include the project name, date, lead researcher, stakeholders involved, background, objectives, methods being used, and a participant section. This template will be particularly helpful for non-researchers writing their own plans.Â
Create a script template that includes sections for an introduction, session details, consent request, and questions. Similar to a research plan template, this will be particularly helpful for non-researchers writing their own scripts.Â
Steve Krug, renowned usability consultant and author of Don’t Make Me Think: A Common Sense Approach to Web Usability, shares some fantastic, free usability templates on his website, including:
Many of these templates can be edited and re-used for other methods, making them a great addition to any research toolkit.
If you want to make continuous UX research the standard for your organization, research operations is a must.
It’s time to buckle down and make a change if:
If your current research process is inefficient (or nonexistent) you have two options—create a ReOps role or integrate ReOps principles into your workflow.
If you want to invest in research, you could create a ReOps role. Chances are you already have a researcher who would work alongside the research operations person.
Creating a research operations role could be good for you if:
Want to go this route? It’ll likely take a few months to hire someone for your ReOps role. Here are your next steps:
Suppose you’re still at the early stages of your company or want to work more efficiently with your resources. In that case, you can use tools and processes to incorporate ReOps into a small team. It’s never too early for startups to begin conducting research, and you can build good operations practices now.
Optimizing research operations in your current team might be ideal if:
Want to go this route? Here are your next steps:
Related read --> How Blinkist’s ReOps Manager streamlines and scales research
Research that seems simple on the surface, such as talking to customers about how they use features, becomes complex once you consider all of the steps that go into it. That’s why some companies rarely conduct research or even skip it altogether.
Great Question believes that every company can benefit from UX research and that the process should be continuous, not cumbersome. Customer-centric teams at dynamic companies like AppFolio, Nutrisense, and Opal use Great Question to recruit participants, manage incentives, conduct research, store insights, and share what they learn – all in one place.
Ready to see how Great Question could transform your research operations? Get started free today.
Jack is the Content Marketing Lead at Great Question, the all-in-one UXÂ research platform built for the enterprise. Previously, he led content marketing and strategy as the first hire at two insurtech startups, Breeze and LeverageRx. He lives in Omaha, Nebraska.
.png)
