One of those dimensions seems to get a lot more attention than the rest of them: Qualitative vs. Quantitative. These two have seemingly split the entire industry in half, with the mythical âMixed Methodsâ researcher being seen as a unicorn-like ideal.
Recent discussion about whether any one person is truly a âmixed methodologistâ underscores a potentially ugly truth about the state of Research: most people come from (and are more comfortable with) a Qualitative background/approach.
This guide is meant to meet you where you are with quantitative familiarity to help you level up your Quant knowledge.
Before we get too deep into the weeds, letâs make sure weâre on the same page as to what exactly âquantitativeâ and âqualitativeâ mean. There isnât a precise definition of this, and a quick search of Merriam-Webster and the Oxford University Press suggests that theyâre basically defined as ânot the other thing." đ
For our purposes, though, weâre going to use the following definitions:
Nielsen-Norman Group has a deeper dive on qual vs. quant if you want to read that, with a caveat that itâs more focused on usability testing than on research overall.
Surveying is easily the most popular quantitative research method. However, methods like A/B Testing, Card Sorting, Product Analytics, and Multivariate testing also fall under the quant umbrella. Common qualitative research methods include interviewing, usability studies, contextual inquiry and ethnographic observation, and diary studies. You can, of course, add qualitative elements to your quantitative study (e.g. asking follow-up questions during a moderated card sort), and vice versa (e.g. measuring time-on-task and success/error rates for usability studies).Â
Now that we have a working definition of these two terms, and a hint at how you might combine them, letâs discuss why mixing methods matters.
Ignoring our work as researchers for a moment, statistical literacy is a critical skill in todayâs world. The sheer amount of data created, processed, and consumed has exploded in the past decade, and if we learned anything over the past few years, statistical literacy may be more important than ever. How many times have we all seen people mix up correlation and causation?
Jokes aside, having a solid understanding of the âother sideâ of research that youâre less familiar with is crucial. First, if your only tool is a hammer, itâs tempting to treat everything as if it were a nail. Too many researchers, when faced with a tough problem, will fall back on the methodologies theyâre comfortable with instead of using whatâs actually best for the questions theyâre being asked to answer. If youâre uncomfortable working with large data sets (or donât even know how youâd collect that much data), it can be far too tempting to conduct a handful of interviews and then start reporting percentages based on your findings.Â
âIf your only tool is a hammer, itâs tempting to treat everything as if it were a nail.â â Silvan Thompson? Abraham Maslow? E.S. Dallas?
I know I just said 30 is the magic number, but remember: thatâs per segment. If you have hundreds of thousands of users, itâs virtually impossible to capture all of that with 30 interviews, and turning 16/20 into â80% of usersâ can create a crucial failure in your findings.
Iâve seen research plans that call for 10 interviews, thinking theyâre going above and beyond the typical 5 to 8 users â except theyâre covering 8 subsegments. That puts their minimum number of interviews at 40. An understanding of quantitative reasoning and analysis is required to recognize that flaw and to understand the dangers of sampling bias.
Improving your understanding of quantitative research also makes you a better teammate, especially when it comes to cross-functional collaboration.
Whether youâre in a critique session with fellow Researchers or youâre working alongside colleagues in Marketing (especially Growth Marketing), Data Science, Engineering, or Product, having a solid grasp of the core concepts in quant allows you to make meaningful contributions.
This could be as simple as calling out potential sources of bias, or finding ways to weave some quantitative metrics into a study design (bonus points for leveraging existing data, whether proprietary to your company or publicly available for your industry). If youâre trying to find ways to work together across disciplines, understanding quantitative research will make it easier to see opportunities for collaboration.
For many of us, the building blocks of qualitative research are things we learned through school or socialization. Active listening, follow-up questions, note-taking, and affinity diagramming are concepts that are familiar to most, though mastering them to the level required for Research is its own challenge.
Depending on how far you are into your career, you might need to think back to the earlier days to understand what itâs like to learn a new skill. Take that same beginner mindset, and go through the same motions you did back then: focus on learning the basics, then start to build proficiency with execution, analysis, and synthesis. After that, you can build deeper proficiencies and start to become more T-shaped.
âšď¸ If youâre feeling uncertain about research design in general, check out this excellent introductory textbook.
Now that Iâve convinced you itâs important, you may be thinking âGreat, Iâm convinced â but Iâm equally convinced I donât know a mean from a mode, and I have no idea how or where to get startedâ. Intimidation and fear are powerful forces and often prevent people from even trying. Before you even worry about quantitative research methodology and analysis, it's helpful to build a basic understanding of statistics. Most educational systems prioritize calculus over statistics, so itâs entirely possible you went your entire academic career without taking much of any statistics coursework.
Start with the basics:Â
These are the building blocks of any quantitative research study. Think of them as a new language you hope to speak with colleagues. When discussing any data set (say, the results of a survey), being fluent in what the standard deviation means in relation to the mean or median is the building block of a fruitful critique.
Once you have a solid grasp of the core concepts, you can decide where to focus your attention and dive deeper. While it might be fairly easy to pick up the basic concepts, it can take years to build proficiency in any of the following topic areas, and doing so will open up new career paths for you:
Of course, you can also stay in the Research world and use these skills to drive better outcomes by delivering better insights. đ Deepening your understanding of what Research can do and the value it can deliver helps you advocate for yourself and your team, raising the profile of your work within your organization.
If youâre already comfortable with one or more of the above topics, itâs time to get your hands dirty. By improving your technical expertise, you move beyond being able to recognize the application or design the study, opening up more flexibility in your analysis. Itâs a very different experience to reuse an existing analysis provided by a vendor or a teammate, versus being able to get into the data tool yourself and follow whatever piques your interest. That interest might again lead you into an entirely different career arc, which proficiency in these skills would certainly open up if you wanted to pursue.Â
And certainly, when I come across Researchers doing things like writing Python, SQL, or R alongside their diary studies or interviews, I do not hesitate to call them Mixed Methods Researchers.
No matter where you are in your career development, what company you work for, or what your role is, having a deep understanding of and comfort with quantitative research and analysis will improve the quality of your work and the interactions you have with your colleagues. Finding novel ways to triangulate your findings with qualitative and quantitative data leads to higher levels of confidence and stronger recommendations. The path to quantitative excellence may be a long and scary one, but it's worth it.
Brad (they/them) is a UX Leader, User Researcher, Coach, and Dancer who's been helping companies from early-stage startup to Fortune 500 develop engaging, fulfilling experiences and build top-tier Research & Design practices since 2009. They have helped launch dozens of products, touched hundreds of millions of users, managed budgets ranging from $0 to $10M+, and coached hundreds of Researchers. Born in Buffalo and currently based in Brooklyn, NY, Brad dances with the Sokolow Theatre Dance Ensemble and Kanopy Dance Company, co-organizes the NYC User Research meetup, and served on the Board of ResearchOps from 2018-2021.
.png)
