The GV research sprint: Finalize schedule and complete interview guide (day 3)

Updated June 2021

At Google Ventures (GV), we have a four-day process for answering questions and testing assumptions without the time or expense of launching. We call it a GV research sprint. This is the fourth in a series of five articles on running your own research sprint. (You can also watch a 90-minute video about research sprints or see GV’s Guide to UX Research for Startups.)

Research sprint checklist:

– Create a recruiting screener
– Post recruiting screener where the right people will see it
– Select and schedule participants
– Start creating interview guide
– Confirm participants
– Complete interview guide
– Review prototype with your team
– Set up test devices and recording system
– Interview five customers!
– Summarize findings and plan next steps with your team

Finalize schedule and confirm participants

If you have any interview slots open, go back to your screener responses and look for more people who fit your criteria and your schedule. As explained in Day 2, call and email them to confirm.

You should have a full day of five interviews planned. Call each participant to confirm the time and place (unless you just spoke with them, of course). Send out the schedule to your team. Buy the gift cards or other incentives you’re planning to give participants.

Complete interview guide

Yesterday you started writing your interview guide, adding an introduction and context questions to learn about participants’ experiences and behavior. Today you’ll complete the interview guide by adding tasks, follow-up questions, and debrief.

Part 3: Tasks
Your team should have a complete (or nearly complete) prototype by now. Now’s a great time to get together with your team, review the prototype, and talk about which tasks you need to test to answer your questions or validate your assumptions.

Task-based testing is critical for getting research findings you can trust. Many startup founders are used to pitching their vision, but in the real world, your product will have to stand alone. People will find it, evaluate it, and use it without your explanation of why it’s great.

And task-based testing is much more useful than “so what do you think?”-style testing. Asking people for feedback doesn’t reveal what they really think, or what they really do. In a research sprint, you should structure interviews around realistic scenarios — you’ll get real reactions from participants and research findings you can trust.

You can do task-based testing with almost any kind of prototype, as long as it looks real: a few mockups, a clickable prototype, your actual product, or even a competitor’s product. Check out our guide to GV design sprints for more info on prototyping.

For a research sprint with Nest, which has been acquired by Google, I asked people to think aloud while scheduling a temperature change on the thermostat. I just described the goal, without hinting at how or where to look in the UI. I wanted to see how people would figure it out on their own. This allowed us to validate our assumptions about the industrial design and interaction design of the thermostat.

What features and tasks do you want to test?
Make a list of the important features and tasks in your product. Which ones are critical to test in this research sprint? You can also think of these as the specific parts of the product or prototype that you want users to visit during the interview. For example, these might include:

• Understanding the value proposition
• Creating an account
• Getting started with the product
• Answering questions or getting unstuck
• Completing checkout
• Changing a setting or configuring the product

Task instructions
Write a brief instruction for each task. Since you want to see how participants do (or don’t) figure it out, write instructions that don’t give any hints about what to do. Good task instructions are like clues for a treasure hunt — it’s no fun (and not useful) if you just tell them where to go and what to do.

For example, rather than asking someone to find the “Help” link, ask them what they’d do if they needed assistance.

During the research sprint with Nest, I didn’t ask, “How would you turn on ‘Away’ mode?” Instead I asked, “Let’s pretend you’re going out of town for several days, and you want the thermostat to save energy while you’re gone. Please put the thermostat into an energy-saving mode.”

Like a game of Taboo, avoid saying specific words that appear in the UI itself (e.g. “Help” or “Away”) during interviews. Use generic alternatives instead. It can be tricky to avoid saying these words when you see them right there on the screen. This is another place where an interview guide comes in handy — you can think about unbiased task instructions ahead of time and write them down.

Part 4: Follow-up questions
After each task (or series of related tasks), you’ll want to ask follow-up questions. This part of the interview is somewhat improvised — for example, you’ll need fewer follow-up questions if everything makes sense and the participant completes the task easily. But it’s worth adding important follow-up questions to your interview guide so you don’t forget to ask them.

Here are some follow-up questions that come in handy:

What is this? What is it for?

What did you think of that?

So what happened there?

Was that what you expected? Why or why not?

So what goes through your mind as you look at this?

Did you find what you were looking for?

What would you do next? Why?

Is there anything else you would do at this point?

Is there any other way to do that?

In what ways would you want this changed to make it better for you?

What additional info would have helped?

The best follow-up questions are easy to answer and not intimidating. These are not hard-hitting Barbara Walters-style interview questions. These are more like the questions that Terry Gross asks on Fresh Air — just a way to put people at ease and get them talking.

Part 5: Debrief
After the context questions, tasks, and follow-up questions, your participants will probably have some strong opinions. Don’t let them leave before asking them to reflect on the prototype, its usefulness, their likes and dislikes, and how the product might (or might not) fit into their life.

Here are some sample questions you can ask during your debrief:

Can you describe to me what you see on this page?

Which parts of this page are most or least important to you?

What do you think this [point to UI element] might do?

What does this [point to UI element] mean?

If you wanted to _______, how would you…?

What do you like or dislike about this?

If you had three wishes to make this better for you, what would you wish for? Why?

How would you describe this to a friend?

If you’re testing two or more prototypes in your interviews (which I strongly recommend!), you can use these questions:

How would you compare those different versions? What are the pros and cons?

Which parts of each prototype would you combine to create a new, better version?

Which one worked better for you? Why?

How is A different from B?

What does each of these do well? Poorly?

What types of people does each of these versions seem to be designed for?

Build an arc

Plan your interviews as a conversation that will flow smoothly from the introduction right through the context questions, tasks, debrief, and closing. Without a smooth arc, the interview is bumpy and awkward, and you’ll have trouble establishing rapport with the participants.

Add transitions between your context questions and tasks. For example:

Now, I’d like to show you some rough prototypes of ideas we’re experimenting with. These are just prototypes, or in some cases just pictures of screens. Even though they look real, they won’t work completely. That means some links or buttons or features may not work quite right. You can still click anywhere you like to do the tasks. When you run into something that’s not working, I’ll let you know. You don’t have to worry about breaking anything.

As you work on the tasks, please think aloud. This means that you should try to give a running commentary on what you’re doing as you work through the tasks. Tell me what you’re trying to do and how you think you can do it. If you get confused or don’t understand something, please tell me. If you see things you like, tell me that too.

Since I didn’t design this, you won’t hurt my feelings or flatter me. In fact, frank, candid feedback is the most helpful.

If you get stuck or confused, it’s not your fault. It helps us identify the problems in the product that we need to fix.

If and when you do get stuck, I’m going to try not to answer your questions or tell you what to do. I’m just trying to see what you would do if you were using it on your own. But don’t worry — I’ll help you if you get completely stuck.

To begin, please just take a look at [this screen]. What are your first impressions of what this is?

Put tasks in an order that’s natural for someone who’s unfamiliar with your product. For example, don’t ask participants to try the checkout flow before they’ve found a product to buy.

Add a closing after the final debrief. An abrupt ending or dismissal when the time runs out can seem rude. Take a minute to summarize the conversation before thanking and paying participants. For example:

This has been incredibly helpful. [Briefly summarize key points.] Your input is really valuable for me and the team as we think about the next steps for these ideas. I really appreciate you taking the time to come in, and answering all of my questions. Thanks so much!

See “How to Test Prototypes with Customers: The Five-Act Interview” for a short video demo of this kind of interview.

Final review: time budget and leading questions

Your interview guide should contain all of the questions you want to ask. Now it’s time for one last review before you conduct your interviews on Day 4.

Take a few minutes to add time budgets to your interview guide — 60 minutes can fly by during an interview. Review your interview guide and assign time limits for each section and task. Be realistic with your estimates and postpone lower-priority topics to your next research sprint, if necessary. (You can also consider running a pilot interview to test your guide and the timing.)

Finally, give your interview guide one last review to check for leading questions. Make sure your questions and tasks don’t contain clues about the “right” things to do or say. Watch out for yes/no questions, which are more likely to reveal your preferred response.

Here are some good and bad examples:

Bad: You program your home thermostat to save energy, right?
Good: Do you program your home thermostat?

Bad: Would you rather use the old version or this new, improved design?
Bad: Is this version better?
Good: How would you compare these two?
Good: What are the pros and cons of these prototypes?

Bad: If you wanted to register, would you click “register” or “sign up”?
Good: How would you create an account?

With a full schedule of recruits and a complete interview guide, you’re ready for Day 4!

GV research sprint series
Previous: Schedule participants and draft interview guide (day 2)
Next: Interview participants and summarize findings (day 4)