UX Benchmark: Compare your website or app with the competition

For many companies and researchers, this is one of the most fun studies to carry out. In this article, we explain to you why conducting a UX Benchmark is so exciting. On top of that, we provide you with a step-by-step guide on how to conduct a UX Benchmark yourself. 

What is a UX Benchmark?

UX Benchmarking - also known as user experience benchmarking - is a process of evaluating and comparing the user experience of a product or service with that of its competitors. Both quantitative and qualitative methods can be used for this purpose. The aim is to understand the strengths and weaknesses of the user experience and to identify and implement concrete improvements.

Characteristics of a User Experience Benchmark:

• Focus: A UX Benchmark focuses specifically on the user experience of a product, service, website, or app. It focuses on understanding how well the product or service performs in terms of usability, efficiency and satisfaction of individual users.
• Measurement method: UX Benchmarks often use qualitative and quantitative measurements, such as user studies, surveys, and analyses of user behaviour. They focus on aspects such as navigation, ease of use, performance and overall design.
• Purpose: The main purpose of a UX Benchmark is to understand how users experience a specific product or service and to identify and implement concrete improvements in the user experience.

What is a CX Benchmark?

CX Benchmarks - also called customer experience benchmark - are focused on mapping the customer experience on a large scale. Here, customers from different companies are asked to participate in the study, after which the results can be compared.

Characteristics of a Customer Experience Benchmark:

• Focus. A CX Benchmark focuses on the broader customer experience with an organisation as a whole. It includes all interactions and touchpoints a customer has with an organisation throughout the customer journey, including before and after the product or service is used.
• Measurement method. CX Benchmarks use a range of measurements, including customer satisfaction scores (CSAT), net promoter scores (NPS), complaint rates, response times to customer requests, and more. They often include feedback from customers through surveys and other channels.
• Purpose. The main purpose of a CX Benchmark is to measure and improve customers' overall satisfaction with an organisation. This can cover customer service, marketing efforts, product quality and other aspects of the customer experience.

The difference between CX and UX Benchmarks

The main difference between UX and CX Benchmarking lies in their focus. UX Benchmarking concentrates on the user experience of a specific product or service, while CX Benchmarking focuses on the broader organisation-wide customer experience. Both approaches are valuable and can be complementary, as a positive user experience (UX) contributes to a positive customer experience (CX). It is important for organisations to determine which approach is most relevant to their specific goals and needs.

The benefits of a UX Benchmark

• Objective measurements. Performing quantitative UX Benchmarks provide objective measurements, meaning that the results from the study can be generalised. Various usability metrics can be used for this, such as the SUS, NPS, SEQ and SUPR-q.
• Understanding strengths and weaknesses. By conducting UX Benchmarks, you compare your product with those of the competition. This immediately reveals which facets of your product perform above or below expectations.
• Comparison with competitors. You immediately know where you stand in the market and what needs to be done to become or remain market leader.
• Identification of trends and changes. By carrying out UX Benchmarks with some regularity, you can compare the results. This way, you can objectively determine whether your organisation is moving in the right direction. 
• Basis for decision-making. Carrying out UX Benchmarks creates a lot of data. The insights from the studies can be used as a basis for decision-making, so no major decisions have to be made based on gut feeling or assumptions.

UX Benchmarking methods

To perform UX Benchmarks, you can use both qualitative and quantitative methods. Below, we discuss both methods and explain their advantages and disadvantages. Of course, it is also possible to combine both methods.

Qualitative UX Benchmark

In the case of a qualitative benchmark, you can use user testing. User testing can be carried out both with and without a moderator. In most cases, it is advisable to perform unmoderated user tests, as more can be measured through and the moderator cannot influence the results. In a qualitative UX Benchmark, the focus is not just on establishing which website is preferred, but rather on why this is the case. 

Advantages

• In-depth insights. Qualitative UX Benchmarks can provide in-depth insights on how users experience a product or service. This includes detailed feedback on user interactions, frustrations and successes.
• Realistic feedback. By having respondents participate in the user test online, you can ensure that tasks are entered in a realistic environment. This makes the feedback more accurate compared to staged scenarios.
• Identify specific problems. User tests can reveal specific problems that users experience, such as navigation problems, difficulties in finding information, or unclear user interfaces. Using recordings, you can easily look back and see what users are up against. 

Disadvantages

• Limited scalability. Qualitative UX Benchmarks using user testing are conducted in smaller numbers, about 5 to 10 per device type. The metrics collected cannot be generalised in this case. 
• Potential bias. User tests can be influenced by participants' personal preferences and backgrounds, which can lead to subjective results. To reduce the chances of this, it is important to prepare a good briefing for recruiting respondents.

Quantitative UX Benchmark

In most cases, UX Benchmarks are carried out by using quantitative studies. Here, large numbers (100+) of respondents are invited to perform tasks on different websites, after which they are asked to complete a questionnaire. For this purpose, the use of the SUPR-q is ideally suited. The SUPR-q is a standardised questionnaire to measure the overall user experience and compare it with other companies.

Advantages of quantitative UX Benchmarks:

• Scalability. Quantitative data collection can be carried out on a large scale, making it possible to collect feedback from a large number of users. This has the advantage that the statements can be generalised.
• Objective measurements. Quantitative data are generally objective and less prone to personal bias, making results more reliable.
• Trend analysis. Quantitative data collected over time allows organisations to identify trends and changes in user experience.

Disadvantages of quantitative UX Benchmarks

• Lack of depth. Quantitative data can lack important context, which can make it harder to make targeted improvements.
• Potential oversimplification. Relying too much on quantitative data without qualitative insights can lead to (too) superficial solutions to complex problems.

UX Benchmark Template and Case Study 

Want to know more about conducting UX Benchmarks? Then check out our questionnaire (the SUPR-q) or read more about it in our customer case study.

Step-by-step-guide

Interested in conducting a UX Benchmark? Follow these steps to get the most out of your efforts. 

Step 1: Setting objectives
Set clear objectives for the research project. Decide on which points you want to be able to compare the UX of your website or app.  

Step 2: Choose the right methodology
Based on your objectives, decide whether you want to use a qualitative approach (such as user tests) or a quantitative approach (such as the SUPR-q). Sometimes a combination of both is the best approach.

Step 3: Collect data
Conduct the benchmark according to the chosen methodology and collect all relevant data. Make sure you involve representative users in the process. Are you conducting quantitative studies? Then start recruiting respondents on time. In most cases, you will need at least 100 respondents. 

Step 4: Analyse the results
Analyse the collected data thoroughly and identify trends, patterns and problems. Use both qualitative and quantitative insights to get a complete picture of the user experience.

Step 5: Implement improvements
Based on your analysis, work on identifying concrete action points for improvement. Prioritise these points based on their impact on the user experience and implement the necessary changes.

Step 6: Repeat the process
UX Benchmarking is not a one-off task. Repeat the process regularly to measure progress, set new benchmarks and continuously strive to improve the user experience.