Some argue that the key is to get insights quickly; any research is better than none. Others believe that doing UX research correctly requires extensive training and education, and that results generated by quick and dirty approaches lacking quality can steer teams in a completely wrong direction.
While the actual state of UX research in many companies may lie somewhere in the middle, this article explores the question of how to make something like more ‘rigorous’ or ‘scientific’ UX research tangible. What can we learn from people who have shifted from academia into corporate UX jobs, and why is this relevant? As someone who’s doing both hands-on work in the UX research field and co-authoring scientific research papers, I will explain a rather unusual take on how to level up your practice as a UX researcher. Alongside that, I talked to Maximilian Speicher, Director of Product Design at BestSecret, who has a PhD in Computer Science and went from academia to corporate UX years ago.
UX researchers might wonder why they should care about the art and science of writing research papers.
Before disregarding scientific research papers as unrelated to their work, UX researchers can discover a lot of hidden value in exploring this territory. Writing research papers can not only refine your craft as an influential UX professional, but also help position yourself as a credible voice within the UX community, driving change through insights and rigor. I will explain how by exploring five key benefits of writing research papers.
One aspect that often gets overlooked in UX research is to review and evaluate existing insights first before deciding to run a round of primary research. This is an integral starting point of writing papers, not only to find a topic to work on that hasn’t been covered already, but also to make sure readers understand its background and the landscape of existing knowledge of the topic. Going through this process hones your ability to analyze and evaluate existing information, identify gaps in knowledge, and develop new hypotheses from there — a vital skill for any UX researcher who wants to be efficient and focus on the work that adds the most value.
“Similarly to something that’s really common in the academic world, we should all focus more on reviewing and building on existing insights in UX," said Speicher. "Most people focus too much on new topics and always want to do something new — although replicating, and building on existing knowledge would be a lot more useful.”
An effective research paper is structured around solving a problem. Writing papers requires researchers to identify problems, develop solutions, and test those solutions. Anything that does not pay into these steps is left out and the paper is considered successful only when there’s an answer (whether positive or negative) to the initial questions asked. The key takeaway from this is to apply problem-solving thinking to UX research, too, and prioritize only those questions, tasks, and activities that will help you solve the problem at hand.
Ever heard the infamous “this would be super interesting to know” from a stakeholder who wants to add more questions or tasks to a study? Then this is your sign to push back and protect the focus of your work on the initial problem.
In UX research, one thing that can be quite overwhelming, especially for those new to the profession, is that there seem to be a million different ways to actually run research. It can be tricky to decide which method to choose, how detailed your planning has to be, and how to eventually prepare and run your research. While the decision on how to approach a project is far too individual to quickly break down in this article, we can learn a thing or two from successful research paper writing:
“Essentially, the basis of writing a paper is to describe everything you’re doing in a way that all relevant info is in there and it’s easy to replicate by others," said Speicher. "This is something that’s often not done so well in UX research. Ideally, in UX there would also be that step to replicate results so we can be sure we can rely on the results and that we’re making the right decision. Just as an example, I’ve seen a lot of A/B test results where the numbers were actually super random or, in other words, straight out wrong (I’ve written about this issue) and this goes for a lot of methods being used out there.”
If we compare classic UX research to writing research papers, the two do have things in common. But there are also some big differences, from study design to execution to analysis. Some key elements of a good paper are:
While we don’t have to do all of the above every single time, not doing them can make you vulnerable to criticism from stakeholders. Most UX researchers have had stakeholders doubt results, or reject or even ignore outcomes because the results aren’t favorable to them. While we can’t entirely avoid this, one way to reduce the chances of it happening is to make your research truly bulletproof — something that successful research papers absolutely and ‘officially’ are, due to the above standards they meet.
“The right level of rigor is even more important for qualitative research because you can’t just rely on some standard statistical tests," said Speicher. "I’d estimate that 80% of UX researchers can’t do proper thematic coding for unstructured qual data and that can really become a problem. Fortunately, it’s not as hard as it used to be because more and more tools are accessible for UX researchers. A great example is statisty.app, a free tool that helps you find and run the right statistical test for your data which can eliminate a lot of the typical mistakes, like using the wrong t-test.”
Mastering communication is one of the most underrated skills in UX research. It’s much less tangible compared to the many step-by-step instructions that exist for research methods, analysis tips, templates, etc. but all the more important if you want your work to have an impact. To hone and expand communication skills as a UX researcher, there are some powerful principles we can learn from writing scientific research papers:
For research papers, your audience is usually a specific publication or conference relevant to your project, and your task is to adjust the writing style to those who will read it.
“You have to know which people you have to convince of your research," said Speicher. "Often it’s not the other UX researchers in the company but a director or even the CEO. You have to understand how to write it for them. In the world of writing papers you always get peer feedback, and you can look at other papers that have been successfully published in the past. You can look at them and copy the style to be successful.”
In a research paper, you “sell” your results to the audience but you also have to explain everything in enough detail (methods, analysis process, limitations) so people could replicate it. In UX research, you “sell” your results, too — but the audience is your stakeholders.
“In many teams, people basically create the reports for themselves and their peers," said Speicher. "Then they complain that nobody reads it. But that doesn’t always have something to do with a lack of interest in research (then businesses would stop spending money on it) but more with the fact that most who write reports and present them don’t really think about who it has to resonate with in the end.”
In the corporate context, people often overestimate what their coworkers already know. This can become tricky if your work touches different areas, or the company is adding new employees who have to make sense of everything.
If we compare this to research papers, the requirement is to never assume; always communicate findings comprehensively, clearly, and critically. Doing so, we make sure the audience can properly understand and evaluate what has been done, why, and what the implications are.
Oftentimes, publications or conferences where you can submit papers have strict page or word limits. Even if it’s an extensive study about a complex topic, a paper will be rejected if it exceeds the allowed maximum length. This is of course not the case for corporate UX research and that can be a problem. Ever gone through a 50-page slide deck to find a simple insight? Or have busy stakeholders share that they are overwhelmed by the amount of results generated by a project? Right! Most of the time, this is neither digestible nor really needed as key insights (with few exceptions) can and should be presented in a more condensed way.
“Writing papers, you always have to strictly prioritize what you include," said Speicher. "For things that aren’t a priority but are still relevant, you can have an appendix if needed. This concept is really powerful and should be done more in UX research or presentations generally.”
Writing is one of the most powerful ways to not only improve your communication skills, but also the way you approach tasks that require deep thinking. By honing your ability to articulate your thoughts well, you’re also strengthening your ability to structure and express ideas more coherently and persuasively.
“I’ve read somewhere that there’s no better way to evaluate potential new hires for senior leadership positions than to let them write a six page essay. And that's because all skills that have to do with leadership also have to do with writing a good, coherent essay. There’s also this story about Jeff Bezos not letting people do slide decks for meetings but instead asking them to prepare papers of maybe six pages. Then everyone reads it at the start of the meeting and the rest of the time they discuss the content, which seems to be a much better way to keep everyone’s focus and also test the communication skills of the presenter.”
UX researchers often know their way around factors that influence human behavior and biases we are all prone to. Factoring this in certainly makes designing studies and analyzing them more challenging, but we can also use this to our advantage in an ethical way. If you’re familiar with the concept of ‘Ethos’ when introducing yourself or doing your elevator pitch to someone in the company, you know it essentially means creating authority, trustworthiness, and status. In short: whatever follows will be received in a more appreciative way. It can be as simple as “I am a UX researcher in team X working on Y and I’m also a published researcher.” The goal is not to be boastful but to let people know that, well, you know your stuff.
“Being able to say you have successfully published papers makes you a lot less vulnerable as a researcher. When a paper gets published, there is literally nothing left to criticize: it is written so well and so transparently, including limitations and sources, that different independent people have given it the seal of approval to say ‘this is truly good research.’”
Another nuance of building credibility is that through writing and publishing research papers, you are demonstrating that you’re serious about quality research and not just scratching the surface. Of course, you can be a brilliant UX researcher without ever writing a paper. However, it’s without a doubt a strong sign that someone actually cares deeply about mastering research.
“Writing and publishing papers also improves your skills to critique the work of others and spot things that aren’t well done which is a really underrated skill in UX," said Speicher. "We need to understand when an agency we’re working with or a colleague has used the wrong method or interpreted data in a misleading way. This can be trained really well by writing your own papers and reviewing others’.”
It’s common to write research papers with other experts in the field or in transversal teams. This naturally prevents operating in a silo — something that corporate UX research teams can easily fall victim to. Most teams I’ve been a part of tend to assign one researcher to one topic but rarely together with other researchers. Meanwhile, writing scientific papers alone is very rare.
“Working together is always good!" said Speicher. "Writing papers together is certainly not like those projects back at school where only one will do all the work in the end but it’s usually a team of people with different specializations and interests so the end result is really rich and multi-layered.“
Speaking from experience, writing papers can help expand your network in many wonderful ways. For one, this practice can help connect and exchange with more experts in the field to grow your skill set and knowledge even further. It can also lead to ideas, conversations, and opportunities you might have not even thought of before, e.g. doing a PhD in an area you are really passionate about, or studying something out of interest that might advance your career in surprising ways.
To wrap up, I’d like to reflect on what I have personally changed in my practice as a UX researcher since co-authoring and publishing research papers. I now put an even greater priority on research activities that will have a major impact, and doing them properly and rigorously, like thematic coding of unstructured qualitative data. I’ve also worked to better understand statistical methods and how to apply them mindfully, considering the consequences it can have to choose the wrong one — something I still see happen frequently in UX research teams I work with.
When designing and creating deliverables, I always imagine that what I deliver will be read by someone not at all familiar with the topic, who could have their doubts about it or simply a critical mindset. Then I make sure to cater my storytelling to the audience who will consume the results. I also tend to exchange more with others about research projects I’m working on and limitations I spot to discuss how to overcome them.
I’d like to encourage UX researchers to think about how they can apply these universal principles of successful paper writing to improve their skills in the field. This can be through starting to write research papers yourself, or to just take away what can be integrated right now into your daily work, like bouncing a study design off a colleague, or using one of the many research tools out there to check whether you’re on the right path with your analysis
I am a freelance Senior UX researcher and UX advisor, co-founder of UX consulting firm Jagow Speicher, and a researcher at heart. Working with diverse UX teams, I help them mature, run impactful research, manage and optimise their UX practice, design powerful personalisation campaigns, and tackle change management challenges. Outside of work, I'm writing about all things UX Research, UX Management, and ResearchOps. Feel free to reach out here or go to https://johannajagow.com to learn more. 👋🏼
