UX Watch Parties

How to help your team get the most out of research interviews

Source

There’s nothing quite like visiting a foreign city for the first time — especially with a great tour guide. That eye-opening, first-hand experience just doesn’t compare to reading a description or looking at photos a friend brought back from their trip. The same is true for UX research interviews. No second-hand research report or highlights video alone can deliver the same jolt of insight and impact as when the core team actively observes a full set of customer interviews. As a researcher, I try hard to help teams see and hear their customers (and the research process) themselves. I try to be startups’ UX tour guide instead of just their “report-er.”

This guide includes tips and the templates we’ve used to facilitate hundreds of successful UX watch parties with GV startups.

Top reasons your team should gather to watch UX research sessions live

Gets consensus faster on ideas based on their merits.

Research participants react to designs and concepts without knowing which came from your CEO or your intern. A stubborn Decider (usually the CEO or PM) might still override customers’ feedback on their pet idea, but it’s a lot less likely when the whole team watched it bomb in interview after interview. And an unpopular or unlikely idea can win a team’s support by performing well in research. Watching sessions together creates a shared understanding on a team, and makes it a lot harder for any individual to deny the results when a team has witnessed it together.

Accelerates understanding of users and motivates team.

More than most sales or customer support calls, good UX research interviews accelerate teams’ understanding and empathy for their customers’ goals, circumstances, and difficulties. Watching and listening to these sessions turns abstract “users” into real people with unique stories. And it consistently rallies GV startups’ teams to work even harder to improve their product or service for those customers.

Saves time and streamlines communication.

After a team has watched a day of research interviews, you don’t have to spend nearly as much time trying to document and communicate all the details and nuances from the study. It shortcuts a ton of the reporting and subsequent communication work you’d have to do otherwise. Observers are usually very eager to share the stories, findings, and next steps (as well as their excitement about UX research!) with the rest of the company.

Get your team to show up.

• Make sure you’re planning a study that’s directly aligned with the team’s and stakeholders’ top priorities. If they’re not interested in your research, you might be working on the wrong thing. (See Questions to ask before starting user research and Start at the end: How to do research that has real impact.)
• Get stakeholders’ input on your study goals, plan, interview guide, and recruiting criteria. Those are good chances to make sure they’re invested, and eliminates their excuses for dismissing results.
• Schedule interviews in a clump. Whenever possible, I schedule five one-hour interviews in one day. And we ask (require, really) teams to reserve the whole day and attend all sessions, preferably from the same room. Reserving a whole day allows them to focus on the study without getting distracted by other meetings and demands. Observing all five interviews in a day also makes it MUCH easier to spot the patterns, and to agree on key takeaways and next steps. When stakeholders or team members attend only one session, there’s always a chance that’s the session that goes sideways or is an outlier.
• Talk it up! Promote the study to your team. Remind them how it will help the project. Ask team leads to encourage others to attend. Add it to their calendars. Send reminders.
• Stoke FOMO. Ask an observer to send the team catchy updates during the interviews.

Make it easy for them to watch.

• Build a simple UX lab to make it easy to stream and record from almost anywhere.
• Reserve a good room for observing together. Lure people with lunch, popcorn, cookies, or whatever special goodies are popular in your company. Provide plenty of drinks and caffeine!
• Stock the observation room with supplies, like pens, paper, sticky notes, white board, extension cords and trash/recycle cans (snacking teams create a lot of detritus in a day. You might even want an air freshener!).

Coach your team to be good observers.

Unless your team are experienced interview-watchers, they’ll need explicit instructions about what to look for and how to listen. Remind observers of the study goals and big questions you’re trying to answer. (And write them in the first column of your Summary Sheet.) Provide a brief description of the participants, along with your recruiting criteria. Review the schedule and plan for the day. Then outline the “house rules” for the observation room:

• Try to see and understand the world through participants’ eyes.
• Focus on your observations. Don’t jump to conclusions. (And for Pete’s sake: Don’t make premature changes to the product or prototype during the sessions!)
• Be careful about taking participants’ comments too literally. Watch what they do vs. what they say.
• Don’t dismiss feedback from participants you disagree with, or who don’t match your assumptions or expectations.
• Respect participants’ privacy, and protect their PII. Don’t go digging into their accounts without their explicit permission.

As an example, here’s the Observer Instructions template we use for GV research studies. It includes links to templates for a Running Notes Doc, Summary Sheet, and Big Takeaways Form.

Give them jobs.

Assigning teammates specific jobs (and rotating roles throughout the day) keeps them engaged, and helps them resist the siren song of their Slack or email.

• Primary “Running Notes” Notetaker: Captures notes and quotes in your Running Notes Doc. (As the researcher, knowing others are taking detailed notes frees me to focus 100% on the participants.)
• Secondary “Running Notes” Notetaker: Adds accuracy and depth to notes. Highlight important observations, and capture noteworthy quotes and time stamps.
• Interview Technique Notetaker: Captures specific interview techniques the interviewer
  uses, so you can use them when you run our own interviews in your Running Notes Doc.
• Facilitator: If possible, it helps enormously to enlist a partner to facilitate the observers. While I conduct a day of interviews, one of my partners, Vanessa or Kate, joins the product team to keep them on task, highlight key observations, communicate with me, and manage debriefing between sessions. I like to check in with the observers after each session (especially the first ones) to get feedback and answer any questions about how I’m conducting the interviews.

Debrief together while it’s fresh.

After each interview, help the team fill out one column of your Summary Sheet. Discuss each of the key questions (the rows in the Sheet) while the Facilitator or researcher types the team’s answers.

After a full day of interviews — before your observers leave! — capture the team’s big takeaways. Review the Summary Sheet together to look for patterns and to counter recency effects. I like to ask, “What surprised you?” Ask each observer to complete a copy of this Big Takeaways Form. Before anyone leaves, ask the Decider to weigh in on next steps. In the coming days the team will have time to reflect and continue discussing what they saw and what to do next. But those initial, fresh reactions are a valuable snapshot that teams refer to again and again.

What if team members can’t attend in real time?

• Record audio and video of your interviews (with participants’ permission).
• Throw a viewing party. If team members can’t attend sessions live, schedule a viewing party to watch a recording of at least one session of your choice. Lure them with lunch, special treats, or highly coveted “I’m a customer champion” badges. While watching the session together, point out key moments, observations, and specific interviewing techniques you want them to notice. (“At this point, I was building rapport. They were nervous at first, but see how they relaxed as we chatted about their dogs.”) Reserve time at the end to discuss what everyone observed and what was interesting or surprising.

Some startups resist watching their own UX interviews. They say they’re too busy, and will just read the report later. Maybe they don’t like hearing frank feedback about their ideas or testing their assumptions. The best startups value these opportunities to learn about their customers, test new ideas, and accelerate their progress. As a great UX tour guide, you can give your team the memorable, first-hand exposure to their customers they need to build great products. After they experience it that first time, they’ll be hooked.

Thanks to Kristen Brillantes, Kate Aronowitz, Vanessa Cho, Anshu Agarwal.