The Startup Real Book
Preface
i
Copyright
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. ii
To everyone that has ever failed and had the audacity to admit it.
iii
Foreword “When all you have is a hammer, everything looks like a nail.” – Unknown
My first encounter with the term lean
sneakers, we need a great toolbox. And
startup introduced me to the concept of a
we need to know how to use it.
smoke test. The idea was elegant, place a value proposition onto a landing page with the intent to gauge customer demand. So of course I applied it immediately. When I heard about Sean Ellis’ “How disappointed would you be” survey, I used that. When I heard about concierge testing, Wizard of Oz testing, paper prototyping, I used them. I used them without thought. I used them because that’s all I knew. I used them because someone whose name I recognized used them and wrote it down on a blog, in a book, on a slide. I used them with the same flawed logic that says, “Air Jordans makes great basketball players.” After 6 years of living lean, I’m starting to recognize that to build something great, to build something that will last, to be a great carpenter, we don’t need a great pair of
iv
What's a Real Book?
Go to any jazz jam session and you’ll see
Of course I learned much later that the
one book one stage. The Real Book.
Real Book was illegal.
Every serious musician has it.
First published in 1971, it didn’t pay any royalties to the musicians who wrote the
It’s a large, tome with text that looks like
songs. But it was the standard book that
someone took handwritten music charts,
everyone learned out of because it was the
photocopied them, and then stuck them in
most complete and accurate.
a binder. It was also published in multiple keys so Each page is a jazz standard. All Blues by
that musicians with varying instruments
Miles Davis, Autumn Leaves by Johnny
would be able to refer to the same song,
Mercer, A Night in Tunisia by Dizzy
with even identical page numbers, and not
Gillespie...even some more esoteric tunes
miss a beat.
like Peaches in Regalia by Frank Zappa. Nowadays, The Real Book is legit, and is Unlike various “fake books” out there
published by Hal Leonard who does pay
which provided details on how to play
royalties as appropriate.
each note and where to put your fingers, the Real Book was minimal. Each page
-
had the key signature, time signature, chords, melody line, and not much else. It
The Real Book answered a simple ques-
was just enough detail to play the song. It
tion for musicians, “How do I play that
was vague enough to allow for the exten-
song?”
sive improvisation that makes jazz jazz. It was heaven.
Where a musician’s job is play songs, the startup product manager’s job is to run experiments and do research. Our job is to
-
5
challenge assumptions and test hypothe-
-
ses. More importantly, please write on it. ScribWe have unanswered questions and the an-
ble, scratch, and change it.
swers lie out in the real world with real users. We gather these answers with re-
If you think it’s wrong, submit a change to
search and experiments.
[email protected].
The goal for the Startup Real Book is to
If you have a suggestion, send it in. If you
provide a single reference material to help
learning a better way, let us know.
answer a simple question: “What should we do now?”
This book is creative common licensed and is meant to be continually improved
Based on the question we ask about our
and shared. So please help us by finding
startup, there is an experimental or re-
problems and fixing them!
search method to answer it. This book should help us find those answers by showing us what methods are most appropriate and give us just enough information to craft an appropriate experiment for our situation, in our industry, in our country, in our business model. It should not be overly dogmatic and should leave su!cient room for interpretation such that depending on our unique circumstances, we can still improvise. This is not a text book, it’s not a “How To” guide, and it’s not a “fake book.” It’s a reference book. Keep it, refer to it, and toss it to the side when you need to. 6
Who is This Book For?
If you’ve ever recognized a giant hole in
It’s ok if you haven’t heard of any of these
your business model and had no idea
terms, but you’ll get a lot more out of it if
how to go about filling in the unknown, this
you have.
book is for you. You may be asking: " " " "
Who are our customers? What are the most important features of
our product? Why are people doing that? Will people actually pay for this?
This book is for managing innovation pro-
In particular, you should already be bought into the idea that there are parts of your business model that are unknown and the way to figure that out is to do research and experiments out of the building and in the real world with your customers.
jects where the business model is partly or
Innovation at Scale
completely unknown. Your job title might
This book is particularly useful for those
be:
managing or assisting large numbers of in-
" " "
Product Manager CEO of an early stage startup Entrepreneur in Residence
Previous Experience You should be familiar with concepts such
novation products. This includes: Chief Innovation O!cer " VP of Innovation " Accelerator Manager " Lean Startup / Innovation Coach "
as: If this is you, you’ll find this book serves as "
Lean Startup
a versatile and handy quick reference
"
User Experience
guide for almost any startup you’ll be deal-
"
Human Centered Design / Design Think-
ing with. It can also be used to diagnose
ing
typical startup problems.
"
Business Model Canvas It is also a Creative Commons project which means you can use it as training ma-
7
terial for your startups without any additional cost.
Business Model This book is best used to answer questions about certain business model elements such as the Customer, the Value Proposition , Channel, Relationship, & Revenue. In other words, the critical ele-
Warnings: Academics and Existing Businesses This book is not for students trying to learn in an static academic environment. You’ll have to go out and use this book in the real world with real customers to get anything out of it. So if your teacher just handed this to you, get ready to get kicked out of the building to go talk to customers.
ments in Product / Market Fit . This book is also not for companies executWhile many of the methods listed here can
ing on an existing business model. While
be used to investigate other business
some of the techniques listed here may
model elements such as Partners & Re-
work quite well for optimizing an existing
sources, it may take some interpretation
product or service, it’s not designed for
on the part of the reader.
that and traditional product management
Stage and Industry
methods might be more appropriate.
The methods here work fine for small, early
Continual Improvement
stage startups and equally fine for teams
Future improvements may change or
in large companies trying something new,
broaden the focus on this book. This is a
risky, and outside the normal business
living document which will be updated
model.
regularly.
It also works for any industry, but there is a bias towards providing examples and case studies from the technology industry. Other industries will be included as case studies become available. Please send any such case studies to
[email protected] to be included.
8
How to Use This Book
Do not read this book straight through.
It is highly recommended that you read the
Read this Index thoroughly, then reference
index thoroughly. You will not get the major
other pages as needed.
benefit of this book without it.
-
When faced with an unknown aspect of your business model, first figure out what
This book is not a step 1, step 2, step 3
you need to learn. What’s your learning
guide to building a startup. Startups don’t
goal? What question are you trying to ask?
work like that. Once you know what you need to learn, Think of this book as a toolbox. Like any toolbox, it’s organized to help us find what we’re looking for when we need it. When we need a way to test market demand, there’s a section on Evaluative Market Experiments. When we’re looking to
consult the index to find a list of research and experimental methods that will help. Then read each method and determine which will work best for your situation and resources. -
prioritize our ever growing feature list into a Minimum Viable Product , there’s a sec-
Each method section, you will find the fol-
tion on Generative Product Research .
lowing headers:
The index for navigating this book is not al-
In brief
phabetical, chronological, or ontological.
A quick 2-3 sentence description.
The index is ordered by what you’re trying to learn. Are you trying to learn about your
Helps Answer
customer? How to price your product?
A list of common questions that this
What will make your users come back?
method helps answer.
-
Tags
9
A list of terms that can also be used to navigate through the book such as B2B (for methods commonly used for Business-to-Business models) and qualitative (for the type of data used by this method.
Method A more detailed description of the steps normally taken to run this research or experiment method.
Potential Biases A list of common errors that may distort the results of the method and lead to bad conclusions based on incorrect data.
Field Tips Tips for startup practitioners who have used this method.
Case Studies Links to various case studies that might serve as examples.
References A list of additional material or resources for those who want additional reading.
10
Contributors
"
Tristan Kromer, TriKro LLC, Blog
"
Kenny Nguyen, TriKro LLC, Linkedin
"
Jan Kennedy, Academy for Corporate Entrepreneurship, @innovationmojo
"
Luke Szyrmer, LaunchTomorrow, @launchtomorrow
"
Phyo Pine, LinkedIn
" Austin
Elford, Linkedin
"
Casey Sakima, Portfolio, Linkedin
"
Jason Koprowski, E#ortless Growth, Linkedin
"
Dharanidhar Malladi, Linkedin
"
Gian Tapinassi, ArtDigiland, Linkedin
"
Hameed Haqparwar, Linkedin, @haqparwar
"
Rammohan Reddy, LeanMantra, Linkedin
xi
1 The Index
“An index is a great leveler” – George Bernard Shaw
13
What Are You Trying to Learn?
“If I had an hour to solve a problem I'd
“Right there in front of you,” answers our
spend 55 minutes thinking about the prob-
teacher.
lem and 5 minutes thinking about solutions.” - Albert Einstein
“Is there a right answer?” we hesitantly inquire.
In school, we’re constantly taking tests to gauge how well we’ve learned last week’s material. We cram geographic boundaries, the dates of battles, and multiplication tables into our heads and spit out the re-
“Yes there is,” assures our teacher. “What are the questions?” we plead. “That’s what you have to figure out.”
sults. As an entrepreneur (or intrapreneur) we Sadly, those rote memorization skills and
can’t just guess at the answers without
the ability to answer pre-formulated ques-
first identifying the right questions to ask.
tions doesn’t help us when we embark on
If we just guess by building a fully function-
entrepreneurship. When building a new
ing product, it’s very likely that the market
business model, there is no test or quiz
will judge us wrong and punish us with
which we can cram for. It’s as if we sat
zero sales and bankruptcy.
down for our final and open up the exam book only to find a blank piece of paper in
Our job, as an entrepreneur, is to first ask
front of us.
the right questions and only then to find the right answers.
“Where is the test?” we ask.
14
What is the Right Question?
For example, let’s say we’re selling a new type of shoe that cures plantar fasciitis. We
The questions we must answer are funda-
put up a landing page test (a type of
mental holes in our business model. Ques-
smoke test) with our value prop and a “buy
tions like:
now” button. Then we put $1000 in to Goo-
"
Who is our customer?
"
What job do they want done?
"
What channels can we use to reach
gle Adwords for “shoes” to drive tra !c and sit back to wait for the money to start rolling in…
them? "
Which features should we build for our first product?
"
Is our solution good enough?
If we can identify the right question, there is a corresponding method or methods listed in this book to help answer that question. Depending on the resources and time constraints, one of the methods may be
…and our conversion rate is 0%
simpler to execute. Should we give up? That’s what the land[insert pic of questions with lines drawn
ing page test says. There is insu !cient de-
to specific methods.]
mand for this product. But what is plantar fasciitis?
If we attempt to execute a method in this book without first identifying the right ques-
Exactly. All those people coming to our
tion to ask, the results of that experiment
site are asking the same thing.
are typically very di !cult if not impossible to interpret correctly.
Is our test failing because customers aren’t interested? Or do they fail because they simply can’t understand the value proposi-
15
tion? Or are we just focused on the wrong
Based on this, if we have a clear hypothe-
channel?
sis of who our customer is and what we think they will pay for, we can conduct an
In this case, we were asking “Does anyone
Evaluative Market Experiment such as a
want my product?” when we should be
smoke test. If we don’t have a clear idea of
asking “Does our customer understand
who our customer is, we can do Genera-
what plantar fasciitis is?” or even “Who is
tive Market Research like data mining.
our customer?” Similarly, if we have a clear hypothesis of
Focusing The Question To simplify our search for the right method, we’ll ask two questions: 1. Do we need to learn about the market or the product?
which features will solve the customer’s problems, we can do an Evaluative Product Experiment such as Wizard of Oz testing. If we do not know which features will lead to an acceptable solution, we can do Generative Product Research such as a Concierge Product to try and come up
2. Do we have a clear hypothesis to evalu-
with new ideas.
ate or do we need to generate a clear idea? Mapping the intersection of these two questions gives us a 2x2 matrix:
Warning: “Any framework is an over simplification of reality! This Index is a quick way to navigate to the correct method, but doesn’t mean you don’t need to think.” The Index of Questions and the Index of Methods show the complete list of questions and their corresponding methods. But first we’ll look at the details of Market vs. Product and Generative vs. Evaluative.
16
Market vs. Product
Do we need to learn about the market or
"
the product? To narrow down the large list of methods
How much will this customer segment pay?
"
to something actionable, we can first sepa-
How do we convince this customer segment to buy?
rate our questions into those about the market and those about the product.
"
What is the cost to acquire a customer in this customer segment?
Market Product " " " " " "
Who is our customer? "
How can we solve this problem?
"
What form should this take?
"
How important is the design?
"
What’s the quickest solution?
"
What is the minimum feature set?
"
How should we prioritize?
What are their pains? What job needs to be done? Is our customer segment too broad? How do we find them? Is this customer segment really willing to pay?
17
"
Is this solution working?
side of the customer using it, but in this case, we are again simplifying for naviga-
" Are
people using it?
"
Which solution is better?
"
How should we optimize this?
"
What do people like / dislike?
"
Why do they do that?
In this case, “market” refers to any element mostly or completely on the identity of the Customer segment. This is a neces-
tion. “If using the Business Model Canvas, market’ questions are those on the right side of the canvas including: Customer, Channel, Relationship, & Revenue. ‘product’ questions are those about the Value Proposition and everything left of it including: Key Activities, Key Resources, Key Partners, & Costs.”
Where should we start?
sary oversimplification that makes it easy to find the right method.
This book is agnostic about where we start. We may already have a product and
For example, market questions include
are investigating who to sell it to or we
those about which Channels we can use to
may have a customer segment with a
reach the Customers. We cannot use tradi-
strong pain point and be trying to find a so-
tional broadcast television advertising to
lution. However, when in doubt, start with
target customers who don’t have a televi-
the customer.
sion set. If the customer segment changes, then the “Product” (or service) is simplified to mean
Product usually must be adapted to the
anything regarding the Value Proposition
Customer. However, if the product
or the production of it. This includes Re-
changes, Customers may simply use a dif-
sources needed to produce the Value
ferent product. Human behavior is notori-
Proposition, as well as any Key Activities,
ously di!cult to change although it is not
Partners, or Costs.
impossible.
The Value Proposition really sits at the intersection of the Market needs and the Product itself. The Product has no value out-
18
Generative vs. Evaluative
Do we have a clear hypothesis to evaluate
Yet these types of flawed hypotheses are
or do we need to generate a clear idea?
common. Here is a slightly more subtle ex-
This distinction depends on our ability to
ample:
understand what makes a clear hypothesis.
“If 250 Los Angeles teachers were asked to treat minority students with more re-
“Our customers really want our product.”
spect, then at least 50 teachers would do so.”
This hypothesis is clearly bad for a number of reasons. The most obvious is that it’s
While not as flawed as the first example, it
tautologically correct and not worth investi-
has fundamental problems that would pre-
gating. If they are our customers, then tech-
vent us from designing a good experiment.
nically they have already purchased our
If we force an experiment, we will most
product and that is a good sign they actu-
likely have ambiguous data or be unable to
ally want it.
interpret it correctly.
It is roughly equivalent to, “If the piece of
In this case, several things are unclear:
paper is flammable, it will burn when ignited.”
"
Which teachers? Teachers at schools with a number of minority students? How many minority students are su !cient for this test?
19
"
"
How should we ask the teachers? Will
Defining good hypotheses can be a chal-
we ask each teacher di#erently? Will we
lenge, so here are some things to look for
let the principals ask them?
and a short checklist.
What is respect in this context? What be-
Simple and Unambiguous
haviors would indicate “more respect”? The hypothesis should be clear and unamWithout defining the hypothesis very
biguous so that anyone reading it will un-
clearly, we might let the principals of
derstand the context and be able to clearly
schools ask the teachers on our behalf
interpret the results.
and they might ask them with varying degrees of persuasiveness.
“If 250 Los Angeles teachers were asked to treat minority students with more re-
We might also argue about the results. Is
spect, then at least 50 teachers would do
calling a student “Mr.” instead of their first
so.”
name a sign or respect or a sign of sarcasm?
In this case, we may have di #erent opinions as to what “respect” means. In order
When we do not have a clear, well defined,
for us to agree that someone is being
and falsifiable hypothesis we are best
treated with “more respect,” we must
served by doing generative research in-
agree on what behaviors would indicate re-
stead of an experiment. In this case, our
spect.
learning goal could be “What teacher behaviors indicate teacher respect to minor-
“If 250 Los Angeles teachers were asked
ity students?”
to treat minority students with more respect, then at least 50 teachers would re-
Given this goal, we are better o # doing cus-
fer to their students using an honorific.”
tomer discovery interviews (a.k.a. speaking to the students) rather that testing our
While this is more specific, not everyone
vague hypothesis. The outcome of the gen-
knows what an honorific is, so we should
erative research should be a clear, well de-
avoid using any specialized vocabulary or
fined, and falsifiable hypothesis that we
jargon.
can then go and test with an Evaluative Experiment.
20
“If 250 Los Angeles teachers were asked
“Students at Dalton High School that
to treat minority students with more re-
study less than four hours a week get a C
spect, then at least 50 teachers would call
or lower in at least one class per year.”
their students using ‘Mr./Ms.’ and their last name instead of their first name.”
There are still more issues, but the hypothesis must relate two or more variables to
Measurable
each other.
“Our customers have a strong desire to do-
Cause and E!ect
nate to charitable causes.” “During the summer, ice cream consumpThis hypothesis may be true, but it is not
tion increases and more people drown per
observable. At least not until we invent te-
day.”
lepathy. This is a true statement, but does not tell “Our customers donate to charitable
us how those two variables relate to one
causes twice per year.”
another. Are people drowning because they ate too much ice cream? Or are they
This new hypothesis has some other is-
eating more ice cream because they are
sues, but it is at least something observ-
sad about all the drownings?
able. “During the summer, people who eat ice
Describes a Relationship “50% of students at Dalton High School
cream will drown at a higher rate than people who do not eat ice cream.”
get a C or lower in at least one class per
This specifies a clear relationship and the
year.”
causal direction of that relationship. Simply
This again may be true and it is observable, but it doesn’t tell us anything about the cause of the low grades. A good hy-
using an IF _______, THEN _______ sentence structure can help make sure cause and e#ect are apparent.
pothesis should allow us to change one
“If we feed ice cream to people, then the
thing and observe the e #ect in another.
average # of drownings per day will increase.”
21
Achievable
When it doubt, we can ask ourselves, “What evidence would prove this hypothe-
“If an astronaut in a stable orbit around a
sis incorrect?”
black hole extends one foot past the event horizon of a black hole, then they will be
If there is no amount of evidence that
pulled in entirely.”
would prove our hypothesis is invalid, then either the hypothesis is flawed or we are
There are many theoretical physicist who
very stubborn.
create a number of hypotheses which are not testable now, but may be testable at
Other Frameworks
some point in the future. While this black hole/astronaut hypothesis is theoretically
There are a number of frameworks and
testable, it is not testable today.
checklists for forming hypothesis, one of which is popular enough to comment on to
Unfortunately, as entrepreneurs, we should
avoid confusion:
restrict our hypotheses to ones that can be tested within the immediate future or
We believe
will result in
within our current resources.
and we will know we have succeeded when
[warning call out: Many things seem untes-
signal>
table today but clever application of lean thinking can simplify the hypothesis into a
The entire sentence is not the hypothesis.
testable first step.]
Let’s break this into it’s parts:
Falsifiable All of these conditions add up to a hypothesis being falsifiable. If a hypothesis cannot
will result in That is the hypothesis.
be proven incorrect, then it is not relevant
we will know we have succeeded when
to run a test on it.
“There is an invisible, intangible tea cup
That is the data we will collect including
floating between the Earth and Mars.”
any information about sample size, margin of error, success conditions, or fail criteria.
22
We believe That section just confirms we think the hypothesis is correct. It is not part of the hypothesis and there are many situations where we may test a hypothesis that we believe is incorrect.
23
Index of Questions
24
Generative Market
Generative Product
"
Who is our customer?
"
How can we solve this problem?
"
What are their pains?
"
What form should this take?
"
What job needs to be done?
"
How important is the design?
"
Is our customer segment too broad?
"
What’s the quickest solution?
"
How do we find them?
"
What is the minimum feature set?
"
How should we prioritize?
Evaluative Market " Are they really willing to pay? "
How much will they pay?
"
How do we convince them to buy?
"
How much will it cost to sell?
"
Can we scale marketing?
Evaluative Product "
Is this solution working?
" Are
people using it?
"
Which solution is better?
"
How should we optimize this?
"
What do people like / dislike?
"
Why do they do that?
25
Index of Methods
26
Generative Market
Generative Product
"
Customer Discovery Interviews
"
Solution Interview
"
Contextual Inquiry / Ethnography
"
Contextual Inquiry / Ethnography
"
Data Mining
"
Demo Pitch
"
Focus Groups*
"
Concierge Test / Consulting
"
Surveys* (open ended)
"
Competitor Usability
"
Picnic in the Graveyard
Evaluative Product Evaluative Market "
5 Second Tests
"
Comprehension
"
Conjoint Analysis
"
Data Mining / Market Research
"
Surveys* (closed)
"
Smoke Tests "
Video
"
Landing Page
"
Sales Pitch
"
Pre-Sales
"
Flyers
"
Pocket Test
"
Event
"
Fake Door
"
High Bar
"
Paper prototypes
"
Clickable Prototypes
"
Usability
"
Hallway
"
Live
"
Remote
"
Wizard of Oz
"
Takeaway
"
Functioning Products
" Analytics "
/ Dashboards
Surveys* "
Net Promoter Score
"
Product/Market Fit Survey
27
Tags & Other Frameworks
There are many great methods, books
"
and frameworks out there on how to iden-
2-sided market - For a business with buyers and sellers.
tify and prioritize risky assumptions, hypotheses, and questions. This index will work in conjunction with any of them through the use of tags.
"
Etc.
Using these tags to navigate the methods is not as simple as using the Index and
All the methods are tagged so as to be eas-
may result in a large selection of methods
ily searchable depending on any other
not entirely suited to the learning goal, but
frameworks we might use. This includes
can be helpful to further narrow down the
simple tags such as qualitative or quantita-
methods, so we’ve included them.
tive used to denote the type of information that the method produces. It also includes tags related to the type of business model, such as: "
B2B - For Business-to-Business
Using the Business Model Canvas The Business Model Canvas is a very popular framework that identifies 9 basic building blocks of any business model and asks us to make assumptions as to what our business will be. Those blocks are:
"
B2C - For Business-to-Consumer
"
B2G - For Business-to-Government
28
Based on our completed canvas, we
For example, if the Customer is the big-
choose the area of greatest risk to our suc-
gest risk to our customer segment, then
cess. Sometimes this is the Customer seg-
we are asking “Who is our customer?” or
ment, but in the case of an existing market
“Is this our correct customer segment?”.
it may be the Value Proposition, Channels,
Based on that, there are several tools avail-
or even Key Partners.
able to learn more about our customer, including:
Each method in this book tagged with these blocks. If we can identify the great-
"
Customer Discovery Interviews
"
Ethnography
"
Data Mining
"
Surveys (close ended)
est risk to our business model via the Business Model Canvas, we can search the tags for a complete list of experimental methods relating to that building block.
29
"
Focus Groups
“Warning: This won’t di!erentiate between Generative Research and Evaluative Experiments, so you’ll still need to take that extra step.”
30
2 Generative Market Research
“Lorem ipsum dolor sit amet, donec ornare vitae.” – Leo Venenatis
Customer Discovery
In Brief Customer Discovery is the early stage of Customer Development - an overarching methodology to discover and validate the right market for your product or service by applying a scientific method to better understand your customers. Discovery should be used at the very beginning of developing new ideas in order to gain a deeper understanding of how customers behave when confronted with a problem you may be looking to solve. It is
very much focused on observing customers and can be likened to “ethnography”.
Helps Answer •
•
•
Who exactly is our customer? How might we profile and segment our customers? What are the characteristics or personas of the customer?
32
•
•
•
•
•
•
Do we understand the context surrounding the problem? What “workaround” solutions do they currently use? What are their pains and how deep is that pain? Do you know if you are solving a real problem? What are the jobs to be done? Can we sell it?
Tags •
B2C
•
B2B
•
Qualitative
tomer is likely to be an “innovator” or “early adopter” – someone who is interested to solve a known problem that they already experience. Therefore, zooming in on an “extreme user” can help you identify the right person / people to observe. This is someone who is confronted with the problem more than most, or is more heavily impacted by it than most. Assume a Beginner’s Mindset We all enter situations with assumptions that may be misconceptions and stereotypes and that may restrict the amount of empathy and understanding you can build. It’s therefore important to “assume a beginner’s mindset” – just how a child would and start your observation experiment from scratch to approach the challenge with fresh eyes. Some Tips from IDEO
•
Customer
•
Problem
•
Value Proposition
1. Don’t judge. Just observe and engage users without the influence of value judgments upon their actions, circumstances, decisions, or “issues.”
Customer Observation
2. Question everything. Question even (and especially) the things you think you already understand.
Your experiment should be focused on observation techniques in order to understand what the current situation is, through the eyes of your customer. As with most new ideas or new technologies, your first cus-
3. Ask questions to learn about how the user perceives the world. Think about how a 4year-old asks “Why?” about everything. Follow up an answer to one “why” with a second “why.”
Description
33
4. Be truly curious. Strive to assume a posture of wonder and curiosity, especially in circumstances that seem either familiar or uncomfortable.
searcher. Some researchers have ethical misgivings with the deceit involved in this approach. •
5. Find patterns. Look for interesting threads and themes that emerge across interactions with users. 6. Listen. Really. Lose your agenda and let the scene soak into your psyche. Absorb what users say to you, and how they say it, without thinking about the next thing you’re going to say. Options Make an informed decision on whom to observe, when and in what type of situation to observe them. It does not have to be “spying” but could take the form of shadowing, getting users to conduct a video diary over time, tracking people’s movement and location via GPS etc. A mixture or combination of 3 approaches can typically be used for observational research (Wikipedia): •
Covert observational research - The researchers do not identify themselves. Either they mix in with the subjects undetected, or they observe from a distance. The advantages of this approach are: (1) It is not necessary to get the subjects’ cooperation, and (2) The subjects’ behaviour will not be contaminated by the presence of the re-
•
Overt observational research - The researchers identify themselves as researchers and explain the purpose of their observations. The problem with this approach is subjects may modify their behaviour when they know they are being watched. They portray their “ideal self” rather than their true self. The advantage that the overt approach has over the covert approach is that there is no deception (see PCIA-II; Holigrocki, Kaminski, & Frieswyk, 1999, 2002). Researcher Participation - The researcher participates in what they are observing so as to get a finer appreciation of the phenomena.
Time Commitment From 2-3 hours for an initial study to several days or weeks for a longer study.
How To When you are in the field observing your customers, record what you see and apply “what”, “how” and “why” to the behavior. •
Start with concrete observations (What): •
What is the person you’re observing doing in a particular situation (when confronted with the problem)? Notice and
34
write down the details. Try to be objective and refrain from making assumptions.
to one at 7pm – 9pm or when comparing different days of the week. •
•
Move to understanding (How): •
How is the person you’re observing doing what they are doing? Does it require effort? Do they appear stressed or rushed or annoyed? Does the activity or situation appear to be impacting the user’s state of being either positive or negative? Use descriptive phrases packed with adjectives.
Interpreting Results •
Field Tips Got a tip? Add a tweetable quote by emailing us: [email protected]
Case Studies •
Step out on a limb of interpretation (Why): •
Why is the person you’re observing doing what they’re doing, and in the particular way that they are doing it? This step usually requires that you make informed guesses regarding motivation and emotions. Step out on a limb in order to project meaning into the situation that you have been observing. This step will reveal assumptions that you should test with users, and often uncovers unexpected realizations about a particular situation. Your next step may be to pursue a “problem interview”.
Keep to a natural environment as much as possible, rather than an artificially created situation for accurate situational observation of your user.
Steve Blank’s “Lean Startup” Works Brilliantly, Even If You’re Building Robots
Got a case study? Add a link by emailing us: [email protected]
References & Contributors •
•
Rob Fitzpatrick: The Mom Test Stanford D-School: Method - Extreme Users
•
Wikipedia: Ethnography
•
Wikipedia: Observational Techniques
Got a reference? Add a link by emailing us:
Potential Biases •
[email protected]
Timing of Observations: An observation at 9am – 11am may produce different insights
35
File Not Found
In brief Not finished! We’re writing this book as fast as we can. Tweet us and we’ll write more chapters: Hey @trikro please write more chapters for “The Startup Real Book”
36