giffconstable

in the name of god Note: also see my 11 Customer Development Anti-Patterns post. Each time I give a talk introducing people to qualitative “customer development” conversations, I try to revisit my points. A few months ago, I gave this talk to an entrepreneurship class at Columbia Business School, and once again the list and messages evolved. Below you can find my latest thinking (here is version 1 and 2) 1. One person at a time Focus groups are a group-think, distraction-filled mess. Avoid them and only talk to one person at a time. If desired, you can bring someone with you to take notes — some UX designers like this approach. Personally, I tend to do one-on-one interviews because I think people loosen up and thus open up a bit more, but it can be nice to have a note-taker, which allows you to focus entirely on the conversation and body language. 2. Know your goals and questions ahead of time Have your assumptions and thus learning goals prioritized ahead of time. Decide who you want to talk to (age, gender, location, profession/industry, affluence, etc), and target interviewees accordingly. Prep your basic flow and list of questions. You might veer off the plan to follow your nose, which is great, but go in prepared. 3. Separate behavior and feedback in discussion Decide up front if your focus is going to be on learning a user’s behavior and mindset, and/or getting direct feedback or usability insights on a product or mockup. Do not mix the two in the discussion flow or things will get distorted. Put “behavior and mindset” first in your discussion flow. During this part, don’t let the interviewee go too deep in terms of suggesting features, but keep them focused on if they have a problem, how they think about the problem space, and if and how they have tried to solve it in past. If you want to get feedback on a product, whether on paper or digital, do this after digging into behavior and mindset. 4. Get psyched to hear things you don’t want to hear If you don’t do this, you might find yourself selling or convincing, or even hearing what you want to hear. This is called “confirmation bias” and we are all very susceptible to it. Your initial goal should be learning. You might, however, run an experiment where you test the market, test pricing, and do try to close a sale. That is great, but keep this part of the conversation separate. As with product feedback, keep this out of the behavior/mindset portion of the discussion. 5. Disarm “politeness” training People are trained not to call your baby ugly. You need to make them feel safe to do this. Ask them up-front to be brutally honest, and that this is the very best way they can help you. If they seem confused, explain that the worst thing that could happen is to build something people didn’t care about. 6. Ask open ended questions Do not ask too many yes/no questions. For example, minimize such questions as “do you like Groupon?” Instead ask “what kinds of deals do you look for, if any?” “What motivates you to hunt for deals?” “How do you discover deals?” “Do you get frustrated with the deal sites out there?” Sometimes it is hard not to ask a yes/no question, but always follow up with an open-ended question like “why?” or “tell me more about that experience.” 7. Focus on actual behavior, not speculative or abstract feelings To emphasize #3: people are not very good at predicting their actions, knowing what they want, or knowing their true goals. Your job is not to ask the person for the solution. It is *your* job to figure out the best solution, and then validate that your solution is actually right. People *love* to talk about features and solutions. When you are in learning mode, don’t let that dominate the conversation. Try to keep things factual. Get them to tell you stories about how they previously experienced a problem, if they tried to solve it (or why not), and what happened. Get them to tell you stories about using other products that are in the same domain space. You do want to dive into their emotions, but you can trust a discussion of historical emotions much more than one speculating “what ifs”. Some people like to ask the question, “if you could wave a magic wand and make this product do whatever you want, what would it do.” Occasionally interesting things can come from this, but I would advise that you take the answers with a grain of salt. 8. Listen, don’t talk Try to shut up as much as possible. Try to keep your questions short and unbiased (i.e. don’t embed the answer you want to hear into the question). Don’t rush to fill the “space” when the customer pauses, because they might be thinking or have more to say. Make sure you are learning, not selling! (at least not until that part of the conversation, if relevant) 9. Follow your nose and drill down Anytime something tweaks your antenna, drill down with follow up questions. Don’t be afraid to ask for clarifications and the “why” behind the “what”. You can even try drilling into multiple layers of “why” (see “Five Whys”), as long as the interviewee doesn’t start getting annoyed. 10. Parrot back or misrepresent to confirm For important topics, try repeating back what the person said. You can occasionally get one of two interesting results through this. In the first, they correct you because you’ve misinterpreted what they said. In the second, by hearing their own thoughts, they’ll actually realize that their true opinion is slightly different, and they will give you a second, more sophisticated answer. Another approach is to purposefully misrepresent what they just said when you parrot it back, and then see if they correct you. But use this technique sparingly, if at all. 11. Ask for introductions At the end of every interview, see if you can get leads to another 1 to 3 people to talk to. If it is not obvious to everyone by now, let me just be clear that you want to avoid doing these interviews with friends and family. There are lots of creative ways to recruit interviewees (the tactics vary depending on who you need to get to), but getting referrals will make the process a lot easier. 12. Write up your notes as quickly as possible The details behind a conversation fade fast, so if you haven’t recorded the session, write up your notes and color commentary as soon as you can. I brain-dump into a shared Google Doc so the rest of the team can see it. (Note: I typically have not recorded sessions for fear of making interviewees more self-conscious or careful, but other entrepreneurs have said to me that, while it takes some rapport-building at the start, pretty soon people forget about a recorder.) Afterwards: Look for patterns and apply judgement Customer development interviews will not give you statistically significant data, but they will give you insights based on patterns. They can be very tricky to interpret, because what people say is not always what they do. You need to use your judgement to read between the lines, to read body language, to try to understand context and agendas, and to filter out biases based on the types of people in your pool of interviewees. But it is exactly the ability to use human judgement based on human connections that make interviews so much more useful than surveys. Ultimately, you are better off moving fast and making decisions from credible patterns than dithering about in analysis paralysis. http://giffconstable.com/2012/12/12-tips-for-early-customer-development-interviews-revision-3/

in the nam of god Steve Blank always liked to say, “In a startup, no facts exist inside the building, only opinions.” The lean startup movement encourages that you get out of the building with a mixture of experiments and qualitative research. Doing qualitative work gives you several benefits. It helps you learn how others experience and think about your problem space. It helps you uncover evidence about your assumptions, or lack thereof. My post “12 tips for customer development” tries to help entrepreneurs and product designers understand how to do qualitative work more effectively. But people struggle with this area. Here are some anti-patterns to watch out for and defeat. 1. You treat speculation as confirmation Here are some question types that I don’t like — and if you ask them, you should heavily discount the answer: “would you use this?” “would you pay for this?” “would you like this?” I can’t say that I *never* ask these questions, but I always prefer behavioral questions over speculation. As contrast, here is a behavior-focused interaction: “Tell me about a time when you bought airline tickets online.” “What did you enjoy about the process? What frustrated you about the process?” “What different systems or methods have you tried in past to book tickets?” 2. You lead the witness Leading the witness is putting the answer in the interviewee’s mouth in the way you ask the question. For example: “We don’t think most people really want to book tickets online, but what do you think?” Examine both how you phrase your questions and your tone of voice. Are you steering the answer? Ask open-ended, neutral questions before you drill down: “what was that experience of buying online tickets like?” 3. You just can’t stop talking Some entrepreneurs can’t help themselves — they are overflowing with excitement and just have to pitch pitch pitch. There is nothing wrong with trying to pre-sell your product — that is an interesting experiment unto itself — but you should not mix this in with behavioral learning. If you do try to pre-sell, don’t just ask “would you pay for this?” but rather ask them to actually pay, and see what happens. Some people ask the question, “how much would you pay for this?” but I do not. Instead, try actually selling at different price points. I much prefer having the potential customer experience something, rather than speculate over something. 4. You only hear what you want to hear I see some people go into interviews with strong beliefs about what they like and dislike. When you debrief after their custdev conversation, it is magical how everything they heard aligns perfectly with their opinions. Our brains are amazing filters. Leave your agenda at the door before starting a custdev conversation. One way to solve this is to have two people for each interview — one person to ask questions, and the other to take notes. 5. You treat a single conversation as ultimate truth You’ve just spoken to a potential customer and they have really strong opinions. One instinct is to jump to conclusions and rush to make changes. Instead, you need to be patient. There is no definitive answer for how many similar answers equals the truth. Look for patterns and use your judgement. A clear, consistent pattern at even 5 or 10 people is a signal. 6. Fear of rejection wins out This is one of the biggest blockers to people doing qualitative research, in my experience. Both fear of a stranger rejecting your advance or rejecting your idea. Many excuses, such as “I don’t know how to find people to talk to”, are rooted in this fear. JFDI. Customer development isn’t just about street intercepts. You can recruit people on Craigslist, Facebook and LinkedIn groups, and good old fashioned networking. 7. You talk to anyone with a pulse I see some teams taking a shotgun approach. Instead, define your assumptions around who your customer will be, and who your early adopter will be. You might even do a lightweight persona (see the book Lean UX for examples). Zoom in on those people and try to validate or invalidate your assumptions about your customers. It is ok to occasionally go outside your target zone for learning, but don’t boil the ocean. Focus, learn, and pivot if necessary. 8. You wing the conversation If you go into a custdev conversation sloppy, that is how you will be perceived. Instead, write up your questions ahead of time and force-rank them based on the risks and assumptions you are worried about. To define your assumptions, you can answer the questions we ask in www.neoincubate.com, do a business model canvas, or tackle the product points Marty Cagan lays out. Your exact method doesn’t matter as much as the actual act of prioritizing your risk areas. During your actual interview, do not literally read your questions from a piece of paper, but rather keep things conversational (remember, you are getting the subject to tell you stories). If you uncover something interesting, follow your nose and don’t be afrad to diverge from your initial priorities. 9. You try to learn everything in one sitting Rather than trying to go as broad as possible in every conversation, you are actually better off zooming in on a few areas which are critical to your business. If you have a huge range of questions, do more interviews and split the questions. 10. Only the designer does qualitative research It is ok to divide and conquer most of the time, but everyone on the team should be forced to get out and talk to real people. Note: you will probably have to coach newcomers on #5’s point about not jumping to conclusions. 11. You did customer development your first week, but haven’t felt a need to do it since It is always sad to see product teams start things off with customer development, and then completely stop once they get going. It *is* perfectly fine to let custdev work ebb and flow. If your learning curve flattens, it can make sense to press pause on custdev or change up how you are doing custdev. However, you want to build a regular qualitative cadence into your product process. It will provide a necessary complement to your quantitative metrics, because it will help you understand the reasons why things are happening. Bonus 12 (added). You ask the customer to design your product for you There’s a famous line attributed to Henry Ford, “If I had asked people what they wanted, they would have said faster horses.” Remember, it is not the customer’s job to design the solution. It is *your* job. It is the customer’s job to tell you if your solution sucks. Get feedback, yes. Remember that the further away you are from a working product, the more you have to filter what you hear through your judgement and vision. p.s. as with all tips on lean and agile, there are always places and times to break the rules and do what is right for your context, and your business. p.p.s. if you want to learn more about lean product design practices, check out the book Lean UX.