How to create a valuable product on the basis of usability testing? Case study

When growing business, people don’t realize how much data there is to consider. When you don’t know how to use the data, you tend to rely on your intuition or emulate others. What can you do to introduce smart changes utilizing research knowledge?

More and more frequently, not only in the industry media, you can hear that you ought to test, monitor, listen to and use all the empathy you have towards your Clients. However, how should you approach this issue when you have little to no research experience?

Based on the example by I will show you how, thanks to research, we created a product – nomen omen – a research tool. Making data driven decisions and continuous work with the users has brought us to create a tool that is a serious competitor to the top products on the market.

Where did the idea to start a business come from?

When we first started creating the usability testing tool, we did realize that we were not the only ones on the market. The decision was strongly encouraged by the need to observe user behavior, which would lead us to identify their problems when we were working on projects for our Clients.

The tools that were available at the time would only show us the overview, leaving questions that were crucial to the project team unanswered (Do users open the site in many tabs? What resolution should we design for? Is the browser always in full screen? Do the users need to zoom content when using mobile devices? I know there’s a user on my site right now, but what is he doing there?).

We wanted to have the possibility to monitor current projects, but available heat maps wouldn’t make it possible to have a quick peek at two versions of the same website. The conclusion was simple: if we want to know more about end users using what we design, we need to create our own custom mini research tools.

It’s the mini-tools we created that now form In our case, the created product is the answer to a real need to observe the broad context of user behavior.

What questions did we ask ourselves and where did we look for answers in the first months of creating CUX?

  1. First of all, does anybody else need it?

We knew we wanted to create a usability testing tool. If there’s a few of them on the market, it must mean there’s a need for such tools. But in which direction should we develop it? How should it be differentiated? Which functionalities should we focus on in the first run of the product?

We decided that our focal point should be conversations with people who might really benefit from this kind of data in their daily work, and whose work without the knowledge about users is fruitless. We wanted to learn what their daily tasks are, if and where are the loopholes or gaps, and what could be done to improve the tasks. This is why we interviewed UX Designers, Service Designers, Product Managers, and people on other decisive positions in a company to better understand their work.

I did not ask whether they’d like to use my tool, or how do they like my idea in a range from 1 to 10. But I did tell them what we were creating, emphasizing that it was not the main aim of the research. The main objective was, on the other hand, to better understand the environment at which we target the product, and finding spots in their daily work where the research tool might actually be beneficial.

Our main findings were:

  • UX Researchers/Designers didn’t use data all the time, oftentimes relying on their own hunches and opinions, which was the result of lack of time for the project, or lack of budget for research, because e.g. their superiors wouldn’t see value in data,
  • UX Designers/Researchers lack strong arguments, which would prove their theses and help put forward ideas during conversations with decisive staff,
  • UX Designers/Researchers are forced to spend a lot of time drawing vital information out of different tools, and preparing reports.

In my opinion, before we speak to the users, we should think about the context, and not ask direct questions. In our case, we focused on the context of work of the people who already use research tools, or it’s required in their daily work to use such tools.

  1. What is the first impression our product makes?

When our product was developed enough to present it to the audience, we invited over our potential Clients for a demo showcase. We conducted interviews mainly via Skype to save time and money.

Thanks to the interviews, we were able to:

  • Observe reactions to the product and its features,
  • Listen to users’ questions and follow their lead,
  • Collect other needs for use in further development of the tool.

However, you need to draw attention to a certain issue: while talking to your potential customers, you might feel tempted to introduce all the changes they suggest. If a client says that he might find a certain functionality useful, why wouldn’t you actually introduce it? But it’s worth validating the information you received against product roadmap and checking the usefulness of the proposed feature for the general audience. Essentially, you need to keep in mind that feedback is not a solution, but only a valuable insight from which we can draw conclusions.

Striving toward gathering the most precise information possible to learn how customers use our product, we need to remember that this information can’t be translated one to one to the product.

  1. Can you use the product at all?

The next important phase when we worked closely with the users was the moment when we decided to make available for our potential Clients to introduce and use it in their companies. What’s interesting, the testers were not our acquaintances. They were people who, knowing what stage of development we’re at and what we can offer, decided to trust us.

It was a breakthrough for us, especially because we decided to release the demo just four months after writing the first line of code. Because we wanted our Clients to use just like any other analytics tool that was available on the market, we intentionally did not create any test scenarios. We would go back to our Clients more or less every two weeks to ask for feedback. We talked via Skype and communicated over emails.

At some point, Michał Sadowski from Brand24 mentioned in his presentation as one of valuable analytics tools. From that moment, new potential users would contact us themselves and ask if they could test We treated this phase a bit like presales. The currency wasn’t money though. We cared about the product being tested in real environment, about valuable feedback and, as a result, recommendations and willingness to use the product continuously. Priceless.

  1. How will potential Clients perceive the product?

Three months testing phase and ongoing work on the product gave us confidence that what we create does satisfy real Client needs. Thus, we decided to enter the beta phase by offering the product for sale. Obviously, at this stage we needed something else besides the product – a marketing website.

Oftentimes, when creating a product, we get so wrapped up in its development and improvement that we tend to forget about the first point of contact between Client and brand, which is a website. It’s too obvious that a website should sell. But I do know what it looks like in practice – you put all your heart into a product, its functioning, usefulness, and existence. But that’s not enough. You need something more. Something that will encourage people to get interested in the product and, as a result, in testing or buying it. Thanks to the observation of users’ behavior, we acquired information what should be changed.

For example:

  • Users felt lost when accessing the tab “pricing” – we could see in the preview that they didn’t find there what they’d expected.
  • Even though the whole website is in English, when hovering over forms, the hints are in Polish.
  • We have multiple CTA’s, but users only use the ones on the main page.
  • Miscommunication: not everybody understood Demo as a Skype conversation but rather as creating a trial account – we should send a clear message before we open registration.


Having in mind all the missing information on the website, we constantly improve it. Why? We noticed that in our case, users act instinctively. They are used to certain patterns of marketing websites of SaaS products and look for concrete pieces of information.

  1. What do numbers tell you?

At the stage when a product is available for multiple users it’s the easiest to check the statistics in order to monitor which parts still need improvement or modification.

Interviews with users or observations of individual user behavior act as a fantastic supplement by suggesting ready solutions. There are certain “must-analyze” indicators.

These include conversion rate, sources of traffic, or bounce rate. We’re not measuring conversion because we haven’t yet introduced open registration on our website. However, it doesn’t mean that we don’t use Google Analytics! Quite the opposite: we improve our knowledge continuously because, despite having all the information from, we believe in collecting additional data from various sources. At this moment, we’re using Analytics to acquire information about the effect of our marketing activities and to learn about demographic data of our users.

We constantly place heavy emphasis on contact with our users. We want them and their needs to shape the tool but we also remember about our objectives and targets. Up to date knowledge about market and real user needs allows us to effectively adapt development roadmap of to the expectations. It’s worth noting that on the Internet you can’t find ready analytics tools or research patterns that would fit a particular business. A wise move would be validating your own website usability research and data against what could be improved and how, based on that data, you can create a product developed for users based on their needs.

Originally published in polish here 
Translation: Magdalena Sobieszuk Translation: Magdalena Sobieszuk