9 Surprising Things I Discovered After Conducting 127+ Usability Tests

Many small businesses today conduct usability testing on their website to improve user experience – or to provide better service to their website visitors. It is a simple test conducted to find errors and problems with a system (a website for example) in order to fix it.

If the website offers poor user experience, people will leave immediately, which will instantly drop their search engine rankings and reputation. This is why usability testing has become even more important than before.

But, most usability test takers are doing it all wrong. In fact, almost 90% of usability test takers are conducting usability tests the ‘wrong’ way that makes their results look even worse than the original.

Here are 9 common yet deadliest usability testing mistakes that most people are making today, and how to avoid it:

1. People are testing elements that look exactly the same

Can you tell me the difference in the usability test screen below, within 3 seconds? Well, I could not tell, until I had to give it another closer look.



(Image source: KissMetrics)

Only when you really look closely, you’ll discover that both variations have a similar sidebar – with the same grey background colour. But, the right screenshot image differs slightly from the left, as the grey background colour expands right across the top column.

The difference isn’t so obvious, is it? That’s what I was asked to find out. I wasn’t prepared as I was expecting a bigger difference in the layout to make an easy choice.

These types of usability tests frustrate a lot of users because most test takers are expecting more dramatic difference than what you see above.

2. People are testing with the “wrong people”

Here’s what I found most people are doing: they’re running a usablity test with people who aren’t their targeted audience – or people who aren’t slightly interested in their product or service.

This is a wrong way of conducting usability testing. Usability testing with a “random stranger” leads to colossal damage to your bottom line.

The right way is to test with your target audience – your existing customers or people who are genuinely interested in your offers.

Why is this a mistake?

You see, when you run a usability test with your intended audience, (or your customers) you’ll know you’ll get better results because these people know what the product does for them. In other words, they know what they can expect from the product. And if they don’t get it, then they can provide appropriate feedback through usability testing.

On the other hand, conducting a usability test with a random stranger – who is not even slightly interested in your product or service – isn’t helpful, as they can only assess your products and offers from what they can see.

For example, let’s say you’re trying to test “a blackjack system” (an eBook) and wanted to discover if people would be interested in reading the book to improve their skills and rake in profits. Testing this system with someone who has never played a blackjack game before, or has little or no interest whatsoever in the game would mean you’d get very poor feedback. Why? Because these people would not know what ideal readers (people who are interested in blackjack game) actually expect to learn from the book.

On the other hand, if you’re testing it with a beginner, or even an intermediate blackjack player who is looking for secret tips that can help them become a better blackjack player, then they’ll be interested in simple playing rules such as knowing card values of each card, how to play (players play against the dealer, not against one another), special bets, among other things.

So, the bottom line is this: make sure you select the people who really care about your product or service – and not just with a random people – to get better feedbacks from your usability tests.

3. People are testing “background colours” using usability tests

The whole point of running a usability testing’s to find “problems” that creates frustrations (barriers) for people to complete a task.

Sadly, the usability test (below) doesn’t state the task clearly for the user. Rather, it asks which “background colour” is likeable.



The usability test (above) doesn’t state the task clearly for the user. Rather, it asks which “background colour” is likeable, which isn’t the purpose of a usability testing. (Image source: Kissmetrics)

This is a question related to brand identity, which helps answer things such as “Does the colour go well with the website theme?” or “Does the colour blend well with the emotion the offer is trying to conjure.”

This sort of questions should be handled by a creative guy who looks after the brand identity. Or the web designer needs to work closely with the existing clients to figure out whether the colours represent the brand properly.

4. People are being “unprofessional” when gathering feedbacks from users

When conducting a usability test, a usability tester must adopt the attitude of “professional detachment.” This means that the tester must NOT encourage a user to take the desired action by using encouraging words such as “Awesome” or “Job done well.” Doing this will imply that the users are being evaluated, and not the system.

Here’s what to do instead: Instead of finishing sentences for your users, maintain your silence, listen, and pay attention.



When conducting a usability test, a usability tester must adopt the attitude of “professional detachment.” (Image source: Usability Testing Fundamentals)

5. People aren’t conducting usability tests ‘properly’

Framing the situation is important so that usability test takers know exactly what the situation is all about. This can be done easily with a statement like “Imagine you’re doing something in an occasion/state of mind/location.” For example, you can frame a situation with a statement like, “Imagine you’re cruising in a big white yacht for a vacation.’

And you state a task for your users so that they know exactly what they should be doing. For example, you could ask, “Can you tell, without looking, what service this company offers?

For a click usability test, the task is more focused towards what a designer thinks a person should click on. Here’s an example: Click where you would find the “order” section.

In the usability test below, these two vital test elements are lacking: framing a situation and stating a task. How will the user know which form is easier to see if they don’t even have a clue what they’re supposed to be doing in the first place? And I really don’t understand what they are trying to accomplish with this test. Test such as this doesn’t help your business.



Can you spot any improvement above? I don’t see any. (Image source: Kissmetrics)

6. People are revealing way too much during the tests, voluntarily or involuntarily.

Here’s another deadly mistake testers are making: revealing way too much information during usability testing. For instance, I’ve seen tester use keywords such as “search box” or “navigation bar” in the task questions like, “Can you see the search box on the screen?”, which obviously reveals the user that the screen is meant to be used for search section.

Avoid doing this. Rather, maintain your silence, listen, and pay attention to their response to get “accurate” data that you can count on to find “real” problems with your products and offers.

7. People are testing to design their logo from scratch

People are asking random strangers, like me, which company logo design they’d prefer. Oh, la la! That’s like trying to hit an arrow in the bull’s eye, completely blind.

As I’ve mentioned earlier, tests such as these would yield better feedbacks when conducted with existing customers or people who are at least interested in your products and offers – not with a random stranger.

Here’s what you can do: Create as many logo designs as you want, but show them to your target audience and see how they react. Doing this is very important – because you’ll instantly know what your existing or potential customers think about your company logo.



Asking a random stranger for feedbacks about the company logo design is like asking a blind man what’s the colour of the sky. Rather create as many logo designs as you want, but show them only to your target audience and see how they react to get better feedbacks. (Image source: Kissmetric)

8. People are making it difficult to take usability tests for their users

The usability test below asks the user: “You are looking to buy blue Men’s straight fit 54 jeans. Where would you click?”

But after looking at the test screen, I couldn’t decide where to click, because there is no clear indication of it. And the texts are way too small to read. I would prefer a zoomed in screenshot of each jeans to be able to accurately and quickly provide the tester with the data they’re searching for.

Even though I know what I’m supposed to be doing here, I’m still out of my wits because it’s too hard for me to see what I’m clicking on. (I can’t even read the text after squinting at it.)


Make usability testing easy and simple for your users. If they can’t read the text, they won’t be able to complete the required task – click on the link, for e.g. (Image source: Kissmetric)

9. People are not “re-iterating” to test a potential solution

The main purpose of conducting usability testing is to find problems, not to find a better solution. If you want to improve your offers, you may quickly talk with your in-house design team for quick solutions to the problems you identified with usability testing. But, in case you don’t have a design team to fix your problems, relax and do this simple thing: run the test few more times to find the best solution for your problems.

Don’t just rely on the first few results you get. Keep on getting more feedbacks from people who care about your product or services. Don’t cut corners, for god sake –it’s your product and it’s going to rake in huge profits if you improve it.



Test iteratively. (Image source: Melody Chou)

Cutting corners means inviting “bad karma” into your life. The solution actually becomes worst than the original design.

Here’s what you can do instead: Set up few rounds of usability testing and run until you get the best solution to the problems your product or offers have. And if don’t find any problem, you don’t have to run the test. It’s that simple.

So, what did you learn?

One of every three usability tests had serious issues. Successful usability testing is still far from being perfect. The key thing is to know two things: why you’re doing usability testing and whether you’re testing with the right audience.

Logos are sensitive items, so make sure you conduct your logo design tests with your targeted audience rather than with a random user.

Usability tests are a great tool to find problems with your products and service so that you can fix them quickly. So ensure you take the test few more times to be sure the solutions you implement actually solve the problems.

Over to you: Did you conduct any usability tests for your website in the past? What exactly did you test and what was the result? Please share them in the comment box below.


We will be happy to see your thoughts

Leave a reply

Login/Register access is temporary disabled
Compare items () compare