How to evaluate a designer with a design exercise

Now that we’re past hunting for unicorns, and we know what design roles are needed at a startup, it’s time to get into the gritty details: the design interview.

I wasn’t really good at interviewing designers until I had a few dozen interviews to compare. So if this is your first time interviewing a designer, I highly recommend finding a design advisor to help interview (and source) candidates for your startup. I’ll often do this for startups as part of my role at Google Ventures, and we’ve found it useful in nearly every case. For teams without designers, a design advisor is just about the only way to get an accurate read on a candidate’s design skills. And for teams with designers, it’s nice to have a second opinion from someone who’s done a bunch of interviews recently.

Even with help, you’ll still need to give design interviews yourself, and that’s what this article is about. There are three common parts to a design interview: a portfolio review (discussing a candidate’s previous design projects), a product critique (tearing apart a product you both understand), and the design exercise — my favorite!

The design exercise

It’s simple in concept: You’re going to set up a well-scoped design problem and ask a candidate to solve it on the spot. It can take anywhere from 15–40 minutes depending on depth and complexity. It’s such a good technique because there’s no faking (like showing portfolio work from a big team effort) and when moderated well, it can simulate working together.

But crafting a good design problem is the hard part.

Ask for the impossible

The trick is to add constraints until the problem can’t be solved perfectly. If you’ve interviewed as a designer at Google, you’ve probably had to wrestle with designing an alarm clock with way too few buttons. The point of the design exercise is not whether someone can get the right answer; it’s to see how people think. And the best way to keep people thinking is to invent a problem that’s impossible to solve.

Create a level playing field

You don’t want someone to hit a home run just because they’re an expert in an area. So I ask candidates to design interfaces for either general audiences or niche audiences that the candidate is not familiar with (e.g. doctors, pilots, 3rd grade teachers). And I avoid domains where I’m an expert — because I’d be a bad judge of what’s easy or difficult.

Focus on a small set of skills

After looking at a candidate’s résumé and portfolio, you’ll get an idea of where they’re strong. Fill in the gaps by getting specific with a design exercise:

• Product design exercise — Can they get beyond the interface and think holistically? Example question: How would you design an ATM for kids? Do candidates start with parents’ needs for teaching children about money, or do they dive into the interface?
• UI Design exercise — Can they use existing widgets appropriately and invent their own when needed? Example question: Design a signup form with some easy data types, and some challenging ones (date ranges, colors, image uploads, etc.)
• Information design exercise — Can they communicate difficult concepts clearly and layout a page? Example question: Design better MTA transit timetables, maps, and signage.
• Interaction design exercise — Can they understand user goals and structure an interaction flow with the right feedback? I like focusing on interaction design because it’s needed on nearly every product, and because other design skills can often be seen in the portfolio. The only way I’ve found to judge interaction design skills is with a design exercise.

An interaction design exercise you can use

So you’re in a meeting room with a design candidate — just grab a whiteboard marker and say:

Let’s do a design exercise. Imagine we’re designing a kiosk at a transit stop. Its purpose is to let regular commuters refill their transit cards. We have an engineer coming in 20 minutes and he needs a spec. In that time, we need to explain exactly how this kiosk should work.

We’ve provided a design goal, user description, and time limit. Now start drawing the machine on the whiteboard and explain as we go.

• This machine lets regular commuters re-fill their transit card with cash.
• There are four push buttons, and a 40-character text display next to each button.
• There’s a card reader, bill acceptor, and dollar-coin return.

Then just hand over the whiteboard marker, say something encouraging, and pay attention to what they do next.

Do they uncover constraints?

Experienced designers will ask a bunch of questions before suggesting a solution. They’re going to uncover and push on technical limitations. It’s great if candidates do this right away, before drawing a single thing. So you’ll need to have the details worked out in advance:

• Actually the max allowed on a card is $50.
• That dot on the bill acceptor? Oh yeah, that’s a light we can turn on and off.
• The machine is right next to a station agent, who can help with anything.

I try to wait until candidates ask before revealing these details. But if someone is stumbling through solutions because they didn’t ask the right questions, I’ll jump in to help. (Don’t let a little mistake ruin the whole session.)

Do they define tasks?

Interaction design isn’t about the screens — it’s about supporting tasks through action-feedback loops. Good designers will clearly define the tasks, often by writing them out on the board. For this problem, the tasks might sound like this:

• Refill card with all cash inserted (common case)
• Refill card with some of cash inserted, issue change
• Issue change when maximum value reached
• Redirect to station agent on error

Some designers are deliberate thinkers, and list out all of the tasks before beginning to draw. Others are explorers — they’ll uncover tasks as they play with ideas. Both ways are fine. Occasionally I’ll meet a designer who doesn’t talk about tasks at all, but still comes up with a great solution. I actually worry about this last type. Even though they’ve found a good solution, it may be difficult for them to work with teams — if they can’t describe tasks and goals to others, critique sessions can get ugly fast.

Are they a visual thinker?

Writing an email or blog post is a great way to crystallize thoughts. Writing helps us think. And the act of drawing is the same — it helps us think through visual problems. You want a designer who is in the habit of thinking visually and feels comfortable drawing.

In this interview you’ve already drawn something on the board and handed the marker to the candidate. If they still don’t get up and start drawing, they’re probably not going to draw much on the job either, and that’ll make solving problems harder.

Are they full of ideas?

Great design requires generating lots of ideas, and then picking out the best ones. So look for designers who find several different approaches to the design exercise. Find people who can see a dozen crazy ways to solve the problem, and aren’t afraid to talk through each one. Avoid people who get stuck and can’t brainstorm their way out.

Can they critique their own work?

Look for people who are never satisfied. They should be able to see what’s wrong with a design and then try to find something better. Some people do this naturally — they’ll come up with a solution and then, without hesitation, list all its faults. If people don’t do this, I’ll gently prompt by asking, “What could be better?” And if a candidate ever gets too sure of any given solution, shake things up a bit. Say something like:

Imagine we put your design into the field, and we found people were scanning a card, inserting a bill, and then walking away without seeing any follow up screens. What problems does that create? How would you solve them?

Feel free to use this design exercise in your interviews, and let us know how it goes! You might want to prepare by giving the exercise to a few designers and non-designers. That way you’ll have a good reference for judging candidates.

Don’t worry about people having seen this exercise online. Just ask, “Have you seen this before? Because it’s really obvious when someone is solving this for the second time.” I’ve found people are honest and will say if they’re familiar with a design exercise.

For designers out there, how do you run a design interview? Do you have any favorite questions? What do you look for?