Interview questions for non-biased mobile app user testing

March 27, 2020
Vlad Averin

Follow these quick tips to end up with objective feedback from your test users.

Preparing for mobile app user testing is no mean feat. You've got to gather materials, set up a testing environment, find participants, prepare prototypes, and much more.

With so much to think about, it's easy to forget that the questions you ask — and the way you ask them — can dramatically impact the results of the testing process.

And, if you’re not careful, a little bias will start to sneak in.

Bias can affect how users respond to your app, or it could influence what you take from what they’re saying. Either way, the result is the same: you’ll end up with misleading data, rather than useful, objective feedback.

So how do you avoid research bias while doing mobile app user testing? Follow these quick tips...

The best interview questions are open-ended ones

In user research, there’s one golden rule: never ask a question that could be answered with a simple yes or no. Instead, ask open-ended questions that encourage the test participant to express their opinions in detail.

To make a closed question open, swap words like did, was, would, which, or when for words like how, what, and why. This allows the participant to fully express their thoughts and allows for more natural follow-up questions.

Closed Questions ❌

  • Was the feature easy to use?
  • Did you like the homepage of the app?

Open Questions ✅

  • How would you describe your experience of using this feature?
  • What three words would you use to describe this homepage?

Open-ended questions eliminate the risk of bias by empowering users to share their unbridled views. After all, ask someone “Did you like my app?” and they’ll likely tell you what you want to hear.

If, on the other hand, you ask “What worked well, and what worked not so well in this app?” — then you’re more likely to hear the truth.

Use a neutral tone

When doing user testing, you'll speak with strangers and you might want to break the ice with some friendly humor.

However, you want honest and critical feedback from your test participants, which is difficult to give to someone you’ve instantly warmed to. So try to keep your questions and conversation as neutral as possible, without seeming too stuffy either.

Essentially, keep your language clear, concise, and free of jargon (or flattery!).

Avoid leading questions

How you ask questions can encourage the test participant to answer in a certain way. So be sure to phrase your questions in a way that doesn't influence the participant’s answer.

Some examples of leading questions to avoid:

  • Can you see how our app would save you time?
  • Does our app make you more productive?
  • Was that feature too slow for you?

Not only are these questions closed, but they also betray what you’re looking for, which will bias the response.

Create realistic tasks

When planning user testing, you'll have a specific outcome you want to evaluate — for example, testing the function of inviting new users to help with editing a document in your application.

So whilst you want to make sure this task gets adequate testing, you don’t want to be too transparent with your objective. This’ll only make users hyper-aware of what you’re testing; eroding the realism of your test conditions, and increasing the risk of bias.

The solution? Write your discussion guide in a way that encourages users to discover the desired task organically. Instead of specifically directing participants to the “invite” button, ask them to perform an activity that would naturally lead to inviting others, such as collaborating on a document.

Need more help with mobile app testing?

We hope these tips help you minimize bias in your mobile app user testing process. After all, non-biased research helps you learn, anything else is such a waste!


If you’re looking to develop a mobile app, we'd love to hear from you. Get in touch with the Orbitum team today and we’ll help turn your app vision into a reality.