Have you ever wondered how many people are needed for a UX research study?
It is important to know the right size of your sample because it can make all the difference in reaching accurate conclusions.
In this blog post, we will discuss what factors influence the number of participants needed and how to calculate the right amount.
What Is User Experience Research?
Why Do Companies Conduct User Experience Research?
What Are the Parameters Influencing Sample Size?
Our Best Suggestions for the Right Sample Size
Final Thoughts
What is user experience research?
User experience (UX) research is a way of understanding how people feel about a product or service. It’s the process of collecting and analyzing data about users, with an emphasis on measuring usability, how easy it is to use something for its intended purpose (a website, app, and so on).
UX research includes:
- Generating ideas and brainstorming solutions
- Conducting market studies and competitive audits
- Evaluating user satisfaction through surveys and interviews
Why do companies conduct user research?
- To validate their assumptions before investing in development. This helps minimize wasted time and money due to incorrect decisions later on.
- To get early feedback from real users so they can iterate based on actual needs rather than imagined ones. This speeds up the design cycle by up to 50% which reduces costs.
- To conduct market research on their product or potential new products within the same category (features, user groups, and more) so that they can improve and optimize what is already there before releasing something brand-new that may not meet expectations.
One of the most common questions people ask when it comes to UX research is: “how big should my sample be?”
What are the parameters influencing sample size?
Many factors are influencing this decision but one thing remains true regardless — if you want results with statistical validity & significance then your study needs at least 100 respondents! It sounds like a lot but keep in mind that these numbers come from real-world studies. These studies are conducted by professionals who know what they’re doing.
Do you mean at least 100?
No way.
You can only work with a handful of users right now, you don’t need that many for my study.
That’s ok but keep in mind you’re sacrificing valuable insights by not including enough respondents to reach statistical significance (95% confidence level).
If you don’t have that many people on your team or you’re new to research then consider partnering with an experienced agency that can guide you through the process.
You may also consider using the product, Loop11 (a cloud-based platform for visual usability testing). It allows researchers to test with just five respondents. Also contains features that will help simplify your research while saving time & budget!
Our purpose is to provide an affordable solution that unlocks the full potential of UX Research by allowing companies to perform studies. It would be without breaking their budgets or becoming data scientists to get quick insights from customers.
The consequences of major actions are far-reaching. That is to say, the individual executing the decision tries to ensure they are making the best choice possible.
By declaring only a certain amount of people out of the total population as test subjects, we are limiting the number of people who can use your system. As a result, you miss out on valuable information and/or data points which could lead to greater insights or even changes in what should be tested.
When examining two or maybe more innovative solutions, it’s common to compare them based on behavioral characteristics such as accuracy, speed, and ease of use.
It’s difficult to establish who the champion was without a full understanding of what success involves. The sample size you use is determined by the magnitude of the variation you want to discover.
This is just a sample size calculator. It helps you determine the number of test subjects needed to achieve statistically significant results and it can help you find out how many participants are required for your particular situation.
The ‘right’ answer depends on several factors including budget, resources (time), type of product being tested (fast vs robust) and what success looks like. The more complex the design space becomes with multiple variables involved, the more participants you need to test.
In this case, statistical significance is very important. Because we want results that will have a large impact on how our product evolves in the future.
Failure to meet these criteria will result in a weak signal which can be misleading when making decisions about design changes or even whether it’s worth pursuing further.
Better safe than sorry.
Our best suggestions for the right sample size
The sample size depends on various factors as we discussed above. Here are a few steps to determine the sample size for your UX research study.
The first step is to clearly define the problem or research objective.
The second step is to determine what data points you need for your study and how these will be used in formulating a solution.
The third step is to decide on the statistical significance of results that meets your budget constraints.
In addition, it also helps if there are more participants from various geographical locations as well as demographic backgrounds such as age groups, cultural diversity, and so on, which help improve generalization of results across users resulting in better insights.
In other words, sample size depends on many factors with commonly cited thumb rules being:
- For quantitative studies collect at least 30 responses per variable (independent/dependent) – rule proposed by Jacob Cohen.
- For qualitative studies aim for at least five participants per each interview – suggested by Miles and Huberman.
- For usability studies aim for around fifteen users to get enough data points (30 is ideal) – suggested by Rubin, Chisnell & Jared Spool.
- More participants often lead to better data.
- Including more people who are representative of your target users is always a good idea. It’s important to keep in mind that the best number for you will depend on certain criteria. As in, what you’re trying to accomplish with user research, how much time and money you have available, and more.
There isn’t one perfect sample size but ultimately there need to be enough participants so that results can be generalized across all your customers/users if possible.
Usually, 30 responses per variable should provide adequate generalization (in other words the minimum needed to make this assumption).
But it depends.
Sometimes qualitative studies need fewer participants than quantitative because certain types of insights require more in-depth understanding.
If you need to test a hypothesis, then power analysis can be used to determine how many participants are required.
One of the most important factors in determining sample size is what your resources allow for. If you don’t have enough money or time, then testing with fewer people will need to suffice.
Doing this may introduce more risk into testing and it’s unlikely that results will generalize which could limit future studies.
This would mean using other methods like triangulation (testing multiple solutions against each other) instead of A/B tests for example.
Testing smaller samples also means running fewer tests overall because there isn’t enough data to get statistically significant findings. You should work out all these numbers yourself along with everything else before you start testing.
The risk of doing UX research work with smaller samples is that results won’t be generalizable.
This means it might not apply to other people and the same types of problems in different contexts.
It would mean using triangulation instead which involves testing multiple solutions against each other, as opposed to just A/B tests for example where only one solution needs to be tested at a time.
Testing smaller sample sizes also means running fewer studies overall because there isn’t enough data to get statistically significant findings. Hence, think about everything involved very carefully beforehand.
To address concerns with usability testing, the sample size needed would be less, usually around 30-40 participants for a good experience.
For remote usability testing, where you have multiple locations being tested at once, test with 15-20 users per location to ensure that the data is representative of their behavior and feedback.
You should be able to get statistically significant results from smaller sample sizes. However, it would take longer so think about your research aims carefully before proceeding.
Additionally, these tests run over one day rather than two which can affect what happens during them if people are tired they might not give useful feedback, and so on.
Therefore, again just bear this in mind when running small studies like this or consider testing with more users instead.
As always, let’s remember that no matter how many users we talk to there is never 100% certainty about what we find out. If you want to be confident in your results then running a larger study is your best option.
Final thoughts
The right sample size for a UX research study is one that provides the right amount of data to answer your research question.
With the help of this article, you will now be able to determine the right size for your next UX research study.
Related Reading:
How to set up your first remote usability test on TryMyUI
Ready Researcher One: A guide to quantitative UX research & user testing
The key to more usability testing success
