User research is hard, but shouldn’t be complicated

By João FerreiraOn June 23, 2020

User research is hard, but shouldn’t be complicated

When I ask someone why they don’t do User Research regularly, usually the arguments revolve around it being very time consuming, extremely complicated to organize, and very hard to get actionable insights out of it.

Like everything that you do, the first few times are always a bit harder and require more energy to pull off, but doing it regularly will dramatically improve your understanding of what makes it work and you’ll soon be more focused on perfecting it than on making sure it happens. The same phenomenon happens if you do it a bunch of times during a project and then stop for quite some time, getting back on that horse will feel like a mountain climb. That’s a natural deception of your mind, to try to stop you from actually getting in an uncomfortable position where you’d need to reconsolidate some knowledge and oil up those rusty wheels, creating room for mistakes. Now, no one likes to make mistakes, but it’s an essential part of evolving and improving at something: you need to fail to see where you went wrong. Knowing where you went wrong will guide your future efforts to stop it from happening again.

I made a lot of mistakes when planning and preparing interviews. I wanted to have a perfect script to read to the participant, so I used to read Steve Krug’s intro with minor adjustments. It was so formal and long, that it just didn’t feel natural to me and to the tone I’d feel comfortable with. Not only that, but we also used to do ALL the steps he suggested in his book “Rocket surgery made easy”, which would take so much time and energy that by the time you were done, you felt like you never wanted to do that again. I’d prepare the recruitment survey and then go to specific forums or sites where I assumed I’d find the target audience I was looking for to share the link. I didn’t use a recruitment platform because I wanted to save the company some money, and ended up spending hours doing the same task, which would actually cost more than just paying a recruitment fee on such websites.

I used to kick myself in the head for being lazy and delaying the research reports because I didn’t know how to communicate all the insightful findings I got to stakeholders, then I’d be utterly frustrated because they didn’t seem to care or even understand them. Indeed, they didn’t fully understand what I was showing them but that was because I was trying to do the impossible task of sharing insights of over 5 hours of interviews and overwhelming them with information, when instead I could digest the info and share as actionable bullet points. I remember thinking that all stakeholders would go through the interview recordings and take notes, then being terribly disappointed when I found out no one besides me had actually seen those. The truth is, making sense of the interviews and highlighting the main insights was part of my job, not theirs. I wouldn’t want to spend hours watching them have technical or business discussions, I’d rather have someone briefly explain to me why this and that decision was made and where we go from here. That’s it.

inglourious basterds

You know how you get to Carnegie Hall, don’t you? Practice.

Aldo Raine, from “Inglourious Basterds”

I used to get to the end of 5 user interviews and feel like I’d just run a marathon and I’d never want to do that again. At the time I’m writing this, I’m helping with User Research on 3 different projects, with totally different goals, and I always raise my hand when someone asks for help with research. What changed? Since then I ran over 100 interviews, with different methods of preparation, in completely different areas and for all sorts of purposes. I started with User Testing on prototypes, just following a set of steps, and now I feel comfortable doing Generative Interviews, which scared the shit out of me back then. I watched professional researchers preparing, running, and analyzing interviews, and they made it look simple. Simple doesn’t mean easy, though. The process should be manageable and streamlined in a way you can repeat it without feeling exhausted halfway through, but there’s nothing easy about guiding a stranger to tell you about their feelings and motivations for doing something. You can’t write down all the questions that will get you there, you need to have a clear concept of the goals of those interviews, be on the lookout for learning opportunities that you may not have considered before, and get the interviewee to feel like they’re having a conversation, instead of feeling interrogated. It’s not so much about finding the “perfect question” but rather how you start a conversation around each goal, how you follow up on what they say so you can slowly dig deeper.

Set the goals and talk to the right people

Goals should be agreed upon before beginning your research. What do we want to learn in these interviews? What are the grey areas that we need to color up? These goals may vary depending on a multitude of aspects, but there’s a few that will always be present:

  • What Job are people who use X trying to do?
  • How do people get the Job done today?
  • What are the pain points of that process?

Now, “people” is too abstract for us to get anything valuable from this. We need to understand who are we targeting first, who are the personas from whom we’ll learn more. These will most likely become your product’s early adopters, the ones who will benefit the most from having a better solution for getting the Job done. Say you’re trying to understand what Job “people” are trying to do when they use something. There’s a lot of users and a lot of features there, so you should define the profile you’re looking for going into these interviews:

  • Do you want to talk with users who use it every day or a few times per week?
  • Do you want to talk with users who use it in a specific way or for a specific purpose?
  • Do you want to talk to newcomers or experienced users?

Start a conversation

The first thing you should keep in mind when preparing an interview is that people want to be helpful, but they don’t really know how they could do that because they probably never thought about the things you want to know about. People use a product because it solves a problem for them, but they don’t rationalize it unless you guide them. That’s why instead of jumping into “What do you like in app X?” or “Why do you use it?” it could be more interesting to break it down into a conversation about how they went from zero (don’t know about the app) to one (using the app):

“When did you first hear about app X?” “Did you download it immediately?” “Did something happen that lead you to do it?”

These 3 questions should teach you about what made them interested in the concept of app X, if they were sold on it from the get-go or did something happen that pushed them to try it, and will get your foot in the door regarding their expectations about app X and how it could help them get the Job done.

You can then move on to figuring out how and why they use it:

“When do you use app X?” “When was the last time you used it?” “Can you walk me through that?”

It’s all about focusing on their lives and constraints, getting to facts instead of speculative scenarios where they’ll try to find the right words to describe why they use app X. Some questions don’t carry on much value by themselves, but they help the flow of conversation and trigger the questions that may lead you to the golden nuggets of truth you’re looking for. By going through their memories and following their journey up to becoming a user for app X, you’re helping them help you.

There’s also a lot of learning opportunities in talking about how they used to get the Job done before using app X. This could uncover unexpected competitors to your product, which could shed a brighter light into what makes app X valuable to them and if there are any missing pieces in it. You may even find that they use some other tool or method to complement app X, whether because it doesn’t cover all their needs or they simply find it easier or faster to get a part of the Job done in some other fashion.

“How did you deal with this before using app X?” “Did you stop doing that completely?”

If the answer is “No”, it usually is followed by the reasons why. You can then follow up on it, tap into why app X isn’t fully solving the problem for them. This could happen for multiple reasons, it could be that app X doesn’t cover all their needs or maybe it does, but it just isn’t good enough for some of them. Maybe they just got their own streamlined system and find it easier to do it that way. Knowing what makes them use it and why they prefer it can provide powerful insights about how to differentiate from the competition and stand out in the crowd.

Digest the information and share insights

A key part of the research process is sharing what you’ve learned with the rest of the team in a way they can understand and act upon. I found that it works really well to prepare a simple presentation where I show some clips of the most relevant quotes or moments during the tests, focusing on the actionable insights instead of the curiosity factors. Not everything you learned during those interviews is inherently interesting to everyone else, in fact most of it will only resonate with you or your fellow designers because it’s in your nature to be more sensitive to the user's needs and pains. Everyone else is only focused on the big picture and trust you to fine-tune all the other details.

Start by presenting the demographics of your recruitment, as well as their profile regarding your product. This will allow you to segment the findings by the different personas and clear out the noise from the results. After setting the stage, go through the goals and the key insights you learned for each of them, stating clearly how many interviewees (and if relevant, which personas) share that feeling or pain. Highlight the positives and the negatives but don’t force a balance if it isn’t there, just stick to the facts. If you’re presenting the results of user tests and you made some changes between testing sessions (known as the RITE method), make sure you show them the iterations you made and how they affected the results.

At the end of digesting each goal, you should share your interpretation of what you’ve learned as well as any ideas you might have to deal with it. These will be your “next steps”, the things you’ll work on to solve the problems you’ve learned about.

After doing all this, open the floor for questions and thoughts from everyone in the room, repeat the next steps of the research efforts and pat yourself on the back: you’ve done the work and you’re now a few steps closer to the ultimate goal - create a product people care about.

Recommended Resources

To record interviews, I recommend Loom. It uploads your recording to their platform and you can instantly share it, as well as trim, make comments or even record a video reply. Quite fantastic.

If you’re running the interviews with a colleague (and you should), use any online tool that allows you to write notes collaboratively, such as Google Docs or Paper.

To share the results with stakeholders, the good o’l Keynote is probably the best choice.