The solution we’re proposing will not feel familiar. It also may not seem — at first — to be particularly easy to achieve. But it can be done and, once achieved, it will fundamentally alter the perspective of your entire organisation and its ability to do new things.
At its heart, the challenge is this: when attempting to make something new which is revolutionary, you must not just be creative with the product, you must have the ability to recalibrate the company to think about, and operate successfully in, a totally new market, or to think about an existing market in a totally new way. And, unlike your existing market, the new market may well not be well formed yet, and it almost certainly doesn’t suit many of your company’s existing operating procedures, practices or success metrics.
The identification of successful new products must, therefore, be based on the ability to actively understand new markets.
Think of all of the companies that have attempted to enter the digital music distribution market. Why is it that Apple succeeded where brands as diverse as HMV, Nokia, Tesco and numerous startups have so far failed to capture the imagination of consumers? Perhaps it was through the force of a very particular will. Perhaps it was through an incredible focus on the value of the change. Perhaps it was because Apple loves music. Or perhaps it was the opposite: it doesn’t care terribly much about music and so avoids the traps previous music obsessives have fallen into. Either way, that company, which was previously a computer manufacturer, managed to completely reorient itself to a new market.
We know now of the success Apple achieved in doing this. And so it is less easy to understand quite how big a deal that transformation really was.
Apple perceived that, in the future, there would be a massive market for digitally downloaded music. It created a hardware and software product for the market which required the negotiation of extremely complicated rights and the establishment of a server infrastructure which was significantly beyond anything the company had previously achieved.
Apple brought the product to market with the same slick marketing it had exhibited for years in its other product lines, having identified a distinct consumer benefit of digital music — your entire music collection in your pocket.
Imagine what could have gone wrong. How many times do you think the product was challenged internally for not being a core business? How many staff questioned that consumers would ever learn to act in this way? How many suggested that the technology could not be achieved. How many, do you do think, suggested that the already very successful company should stick to its knitting?
Could your business do this? Would your staff let you? Would your customers?
We know the ingredients for new product success. We know the shape of a product that takes us beyond our current market. We know, too, the behaviours and factors which hold us back, and stop us making the necessary leap. How do we break those habits and deliver the next product success?
7.1 Our five principles for revolutionary innovation
1. Get the right idea before getting the idea right
Don’t let the potential execution of your genius idea trample over identifying your genius idea in the first place.
It’s tempting to have an idea, visualise how you think it would work as an end product, and get on with designing and building it, assuming that you have all the answers you need. However, by doing this, you may well not be capitalising on the opportunity or insight that led you to the idea in the first place. Or worse, your original insight might be completely invalid.
This is the homework mentioned earlier that is so important to the success of new products — ensuring you understand the essence of the idea, that the problem you are trying to solve, or the opportunity you are trying to exploit, actually exists — and then getting it into shape as a proposition that delivers on the promise of that idea. This means a proposition that meets the needs of customers, that is feasible to make, and viable to operate. Only then can you really think about the best possible execution.
Determining the essence of a product involves unravelling a proposed solution until you get you back to the core insight that led you to the idea in the first place. Once unravelled, you can clearly see the difference between idea and execution, and you can formulate that original idea as a hypothesis you can later test.
When you do this, your ideas about the execution usually change dramatically as you discover more about the value that the proposition has for end consumers and the feasibility and viability of the execution. The desire to skip the proposition development phase and get straight into the production of that idea is a recipe for disaster. Desirability (will customers want it?), feasibility (can it be made?) and viability (can it make a profit?) must all be understood before you can decide on the execution and start building in earnest.
The proposition must, at this stage, tell you who you believe you are going to sell the product to, and how you are going to sell it; what are the distribution networks, who are the partners, how will your product be presented? You must have convincing hypotheses on whether or not customers will buy it, and what their choices and motivations are when they do.
2. Stop making predictions and start experimenting
Without a doubt, the number one cause of missed targets is the setting of targets.
The rule here is that if you’ve been asked by your boss to predict how successful your innovation will be, you are no longer in the innovation game.
You must avoid being pulled into predictive behaviour too early. Companies that require this sort of forecasting in the early stages of product development are doing nothing more sophisticated than asking to be lied to, albeit with complex-looking spreadsheets and graphs to support those lies.
We’re not averse to a litmus test of viability in the shape of a rough calculation where you work out what constitutes a significant product in the world of your normal business, and working back to see whether it’s feasible to build it at the right cost or to reach enough customers. But we stop short at making grand predictions of future success.
At Fluxx, we call this the ‘Numbers Game’: someone sets an arbitrary target of profit we want to achieve, and we work out how many sales or customers we would need to achieve this random — but compelling — number. What this process does not do is make any predictions; rather, it highlights what we don’t know and shows us what things would have to be true to make us successful. It also allows us to sanity check our customer or sales requirements against other known businesses. This activity is not intended to be a predictive exercise or generate a business case; it is simply an activity that highlights the potential weak links in the chain we know we’ll have to look at in more detail later. We are always careful not to allow these numbers to turn into targets or predictions.
As described in The Other Side of Innovation (p. 146), there is an interesting cognitive bias when targets are missed in business to assume that outcomes were too low, rather than the predictions too high.
What experimenting does is bring increasing levels of certainty to some of the numbers that underpin the calculations in our Numbers Game. A series of small experiments will steadily evolve your understanding of what something will actually take to deliver, how customers will react to it and, therefore, the likelihood of achieving the numbers you need to make the business stack up.
As our good friend Shed Simove says, ‘experiments don’t succeed or fail, they merely have outcomes’. Experiments will have hypotheses to test, for sure, but whether a hypothesis is validated or not should not be seen as a business target hit or missed but, rather, simply as something you’ve learned that will improve the product you’re working on.
The ‘big bets’ culture has businesses agonising over, say, a £2m investment for a new product based on no actual evidence. Rather than take this one huge bet, the experimenting method proposes we take a series of much smaller bets, say twenty £100,000 bets, or even 200 £10,000 bets and, in the process, learn a massive amount about the market, the product and what is likely to work; and we do, of course, have the ability to stop at any time without losing face.
With these much smaller budgets, we free up more investment to explore more areas of opportunity, and also reduce the level of predictive promises that business teams require from each other in return for releasing that capital. Remember, the smaller the ‘I’, the less we care about the ‘R’ in ROI…
3. Learn from what people do, not what they say they will do
We know that we are bad at predicting how other people will behave. Why then would our customers be any better than us at predicting how they themselves will behave?
There is a fiction persisted by researchers that we can find out what people will do in the future, or indeed discover why they did something in the past, by simply asking them. What these methods don’t account for is that people simply don’t always tell the truth. They don’t do it maliciously, but they craft responses to market research based on a complex set of very human considerations. How will my answer affect someone’s perception of me? Will my answer prejudice some benefit to me or to my fellow humans at some point in the future? How would other people want me to answer? How can I look clever to the rest of the group? How does the researcher who paid me money and put a glass of wine in my hand want me to respond?
So we need to find new ways to find out what customers care about, and how they will actually react to the products we are developing.
4. Build a team to learn, not to ‘succeed’
Probably best phrased by Jeremy Clark in Pretotyping@work 18 when he says: ‘Wake up, Pollyanna: MOST NEW IDEAS FAIL.’ Clark and his colleague, Alberto Savoia, also coin the brilliantly reversed catchphrase: ‘Failure is an option.’
When you are in the incubation phase, the results of any experiments or study should not impact on the wellbeing of individuals or the team itself. The reality is that this can be very difficult to do. Left to their own devices, teams are very likely to become emotionally attached to the ideas on which they are working, and they are likely to make a connection between this idea and advancement in their careers. But the outcomes must impact on the idea itself and not on the team that learned of the outcomes — in other words, we must thank the messenger, not shoot them.
The measure of success and, therefore, the basis of rewards and advancement for the team must be their ability to learn, and to generate learning. We must reward our staff’s own behaviours, and not the behaviour of the markets. If we incentivise our people to produce results, then they will bias themselves to find positive outcomes — regardless of what they have learned in the process about the suitability of the idea for our business. We should reward responsible, honest behaviour, as that’s the only way we will know when it’s right to start — or shut down — a project.
Get people used to moving on quickly with no stigma attached. As much discipline should be applied in de-funding projects as went into funding them in the first place. When shutting down a larger initiative, you need to take time to ensure the reasons are understood and retained, and that the team involved sees it as the right thing to do rather than as a reflection on its abilities. The advantage of having a centralised function that deals with new product development is that team members can retain the knowledge and learnings from everything, regardless of whether or not a product went to market. In addition, they can see every outcome, even a negative one, as acceptable, not as a personal failure.
And don’t keep it a secret. The innovation team must get used to giving bad news as well as good. And the board (or whatever executive exists) must get used to receiving it. Unless you’re incredibly lucky, there’s going to be more bad news than good news coming. Attempts to conceal the bad news or magically transform it into good news are futile. Regular grown-up conversations are critical to a sane innovation process. They also provide a platform for the innovation team’s work to be shared with the business which may of course make for some quite unexpected successes, through other parts of the business making use of what has been learned.
5. Do something
As you will discover very early on in any innovation programme you launch, the lists of reasons not to do something will always be longer than the list of reasons to do it. But not even attempting something is to admit defeat from the outset.
The number one characteristic of successful innovators is ongoing enthusiasm and tenacity. Without any success or failure, there is nothing to learn from, just a void — a total lack of knowledge or information. Doing something will start to fill that void with evidence rather than opinions and increase your confidence about what to do next. Leaving the void empty will paralyse you.
7.1.1 Getting the right product
Taken to its most extreme and somewhat unhelpful simplicity, creating an amazing new product or service has three key stages:
- Figuring out a great product which has an identifiable market and can be produced at a profit;
- Executing it flawlessly;
- Finding ways to make it better, incorporating feedback from the marketplace.
When we are in stage one, we are learning about a market, learning about technology and assessing business viability. That is all.
If you don’t do stage one right, then the rest of the process is irrelevant. Stage one is neither the most time consuming nor the most labour intensive.
But it is the most uncomfortable. Most people have no idea what it feels like to do it. The last time they had to work this way, they were in kindergarten, trying to build a picture out of bits of straw. And so they feel uncomfortable doing this sort of thing in the office.
Of course, for the most part, the instinct to not act like you are in a kindergarten in the office is the right one. But to do this first stage well means changing your understanding of what is appropriate in a corporate environment. We think this is why people find it hard.
If the idea is not evaluated then no amount of peerless execution or customer feedback at the end of the day will fix it. So we’re stuck with the need to do this uncomfortable bit.
Indeed, this is the bit that startups find so natural. When you’re starting with nothing, all you can really do is experiment to see what works and what doesn’t. You have limited resources and so you do what is within your power and budgets to do in order to advance your understanding.
Some have interpreted the ‘Lean Startup’ movement as being about maximum possible speed to market, and others have interpreted the concept of a ‘minimum viable product’ as meaning that we should go to market with the bare minimum of features in order to balance our investment with risk.
But this is not what Eric Ries actually describes in The Lean Startup — and both ideas are very dangerous. That’s not to say that Ries and the lean movement don’t emphasise pace. What Ries observed was that lean startups were unafraid to get into their markets quickly in order to generate some learnings. They got inexpensive experiments out really fast into the real world to see how people reacted, and it’s these experiments which are used to evaluate their hypotheses about what would make their product successful.
They did it this way precisely because they knew that to build a product worthy of launch to consumers would be expensive and time- consuming in order for them to get it right — no matter how bare its feature-set.
If we go to market now, even with a bare version of the whole product, then we are jumping to stage two. We have moved from evaluation to implementation. And if we then chose to launch something half- featured, we are not doing stage two properly either. Stage two says we should make the product as good as it can be. A product missing half the eventual features is not as good as it could be.
So, we agree with Ries; that we should first evaluate various hypotheses about the product as we proceed, with just enough investment to get the learning we desire about what will make a successful end product.
With this approach, our hope is that a whole series of features or facets will never have to be developed — badly or otherwise — as we discover that they are simply not required to drive the value implicit in our product, and instead we learn which elements of the product particularly matter to the customer.
Getting the right product is as much about learning about the viability of our product as it is about understanding its value in the market. Just how hard will it be to produce, or to operate? Could we make it better by adding more, or perhaps by taking more away?
We should be learning, too, about our routes to market. Who will help us to sell our product? Who will help us to distribute it? Can a partner make it easier or cheaper to produce? What other tools, partners or technologies would make it better?
7.1.2 Stop making predictions
‘History is merely a list of surprises. It can only prepare us to be surprised yet again.’
The most important single duty of a manager of a new product development team is to avoid the temptation to make bold predictions. And, similarly, as a director of an enterprise where innovation is critical, avoid creating a culture of arbitrary prediction making.
As we’ve said before, not all normal business is predictable. In 2011, a tsunami in Japan caused power and component shortages in the region that directly impacted upon industrial output and global corporate profitability. In October 2012, flooding in Manhattan caused the so-called capital of the world to shut down for three days in a row.
However, most businesses function on the basis of repeatability and predictability. It is when we are trying to learn about new things that we need to admit we don’t know what will happen. It is only habit that would have us do otherwise. If lack of knowledge is the problem, then learning has to be the solution, not guessing.
And how do we learn about things? Again, the answers are almost childishly simple:
- We find someone who knows the answers; or
- We experiment.
|Someone else has done it before||Someone else hasn’t done it before|
|You have done it before||Normal||Normal|
|You haven’t done it before||Hire the person who has done it before
Even in a scenario where it feels like others should know the answer to the question and, indeed, where you manage to find someone who has done something similar in the past, it is important to acknowledge differences that can exist because of brands, target markets, or even timing with the product you are trying to develop.
220.127.116.11 Testing commercial viability
Commercial viability is — in many ways — the easiest of the constraints to understand. It is certainly the area with the most established practice.
By the time you seek large-scale funding to bring your idea to life, you will need to have a very well-considered financial business plan for the new product or business. But that time is not now. Far too often, innovators put off making an initial plan because they are intimidated by the task, fearing they will be falling short by not producing a masterwork on first draft.
But this is not how it works. Like the product itself, and your level of knowledge of the market, the business plan will develop as you progress. It’s great to look back at successful ideas and companies we have seen come to market and see how the business planning evolved with them. A basic business plan can be constructed in just a few hours. Doing your first plan will make you feel a great deal more confident about building more comprehensive plans in future. And it will also force you to focus on some key factors which will inform every other aspect of the plan. Start here.
There are numerous approaches to building business plans. We typically use the business model canvas (see http://www.businessmodelgeneration.com/canvas). We’ll talk in a little bit more detail about actually carrying this out later, and how it feeds into a commercial model.
However, the most important part of doing this for the first time is to force yourself to answer the following questions:
- Who is the potential customer for this product?
- What problem do they have that this product will solve?
- How will my product solve their problem?
- What other solutions are there for this customer problem?
- How does my solution compare to these solutions (if any)?
- How much would a customer be willing to pay for this?
And now you’ve got your hypotheses, you can play the Numbers Game, and sketch out a plan for how you’ll make money from each customer. It doesn’t have to be rocket science. But if you can’t make it work on the back of a napkin, what chance have you got in the marketplace?
As we said earlier, if you have big holes in this plan (or significant costs that you can’t see how to overcome), what you’ve done is simply set out some of your risks and assumptions that you need to find out more about later.
What else have you learned?
In thinking about such a fundamental structure of your business, you have also begun to state the most basic assumptions you are making. These assumptions will prove to be the real questions you must understand to make a success of the idea. Begin to get into the habit of recording every one and then finding a way to address it before proceeding to the next step. Every assumption you keep unchecked as you move forward is a risk. It could be the thing that kills your progress later. So keep a list.
18.104.22.168 Modelling complex businesses
Complexity can arise in many ways. But often it comes from the need to try and understand the conditional usage of a product. So, for example, it may be easy to model the financial viability of selling an online storage service where a customer pays a fixed rate to gain access to the service, but more complex to model the business where usage impacts pricing, or some element of the service is available for free in the long term as a promotion for the service.
The disadvantage for the entrepreneur or new business team is that companies that have managed to make a success of these complex business models (such as the very darling of the lean startup world: Dropbox) do not talk about their model publicly. And, even when they do, these statements must be taken with a massive pile of salt.
The only way to learn to about these more complex, conditional models typically is to construct your own models and then conduct experiments to understand how people might react in them.
We won’t go into the detailed construction of models here, but we do have a few key pointers:
- Build models so that they have as many parameters as In working on one project, we were able to determine that certain styles of pricing would never work using the business model alone, by showing that even with the most optimistic customer forecast case, long-term usage growth would result in a negative margin.
- Be careful who the models are shared with. They could be taken as predictions, which could be potentially fatal for the project. Typically, it will be possible to adjust assumptions (parameters) to create positive and negative outcomes. If you must save or distribute the files, do so with the negative or break-even scenarios, lest your plans be misused as a yardstick of future
- It is easy to miss This creates excessively optimistic models. Address this by including an additional cost line which can be set to inflate costs arbitrarily by up to 100 per cent.
- It is easy to miss revenues or other positive For example, there may be opportunities to gain revenue from advertising in an online service. Make sure that these are included but take precautions to ensure that they’re not the elements that justify the investment.
- Forecast as far out as possible (typically five years). We have seen several unusual business cases that flip after 12–18 months. Very few of today’s most successful products made a positive return in the first 12 or even 24 months (including Facebook and Twitter) yet, bizarrely, very few companies are willing to even begin to invest in product ideas which don’t achieve positive returns in this This is where your company’s current attitudes towards investment and reporting might well be challenged.
- Beware of impossible growth forecasts. Anyone can build growth projects that do not have an effective limiting For example, imagine you start out with 1,000 customers in month one for a volleyball TV subscription service and then increase the users by a factor of 1.3 (i.e. month two is 1,300, month three is 1,690, and so on). This might seem reasonable. Bear in mind, however, that by month 60 (the end of year five), you will be forecasting 0.5 million subscribers. Are there really that many volleyball fans in your market? And are you going to be able to attract them all? Downward pressure on growth comes from market penetration. But this is the posh way of saying use your common sense when building models, and marry them up to real-world statistics about your market.
- Free earns you nothing and teaches you nothing. The behaviour of customers around ‘free’ has long been studied. Nothing you learn about customer behaviour with a free product will teach you anything about a paid-for product, so don’t expect that a product researched as free can be migrated to a paid-for price
- Market viability is not necessarily portable between For example, the economics of the top end of the market will be very different from the economics at the bottom. Geographically disparate markets, especially those with different competitor sets, will behave quite differently and need to be independently assessed.
We think of such models like lines of code in a digital prototype or the wires hanging out of a physical test device. They help us to understand how the product will live in the market, as we understand how a user will react to a visual or physical prompt.
Try to model so that you can understand dynamics rather than predict them.
22.214.171.124 Acceptable margins for big companies
A key question to ask when drawing up models to test ‘viability’ is: what would count as an acceptable margin or return for a product?
This question hides a more complex consideration, which is that the market and, therefore, margin for new products can evolve over time.
So early versions of a product may have a small market and low margins, but as these markets develop both margins and size can improve.
From this point of view, the business may potentially need to be able to operate in the market at a small scale and / or low margin in order to reap the benefits. The clearest example of this dichotomy is given in Christensen’s analysis of the Apple Newton. He makes the point that when Apple’s first products (such as the Apple II) were developed, sales of just a few thousand low-margin units were regarded as a major victory. Enough, in fact, to generate a very highly-rated listing of the company itself. By the time the Newton came to market, Apple had to achieve much more (despite launching the product into a market every bit as immature as the personal computing market had been 15 years earlier). So much so that the Newton was seen as a failure:
‘It was a market-creating, disruptive product targeted at an undefinable set of users whose needs were unknown to either themselves or Apple. On that basis, Newton’s sales should have been a pleasant surprise to Apple’s executives: It outsold the Apple II in its first two years by a factor of more than three to one. But while selling 43,000 units was viewed as an IPO-qualifying triumph in the smaller Apple of 1979, selling 140,000 Newtons was viewed as a failure in the giant Apple of 1994.’ 20
It is not just that companies are selling to early adopters; it may well be that the eventual purpose of the innovation is not its originally intended purpose. And, in fact, such clarity can come only from a time- consuming cycle of trial and modification in the market itself. Judged by the rules of the parent company, such programmes may be deemed to have failed too early as the energy needed to sustain them to fruition may simply be absent from an organisation that is used to greater predictability and larger day-to-day successes. And this is the reason that a new, hungrier startup may find it easier to prosper.
This game of comparisons is very clearly an example of why large companies often pull out of innovation ventures whilst startup companies continue aggressively pursuing the same market. And it is a core reason why we believe any innovation team must be isolated from the management of its parent company in a meaningful way. That’s not to say that new businesses should be massive loss-makers of course, but that they cannot operate to the margins honed over years by their parents, just as graduates can’t be expected to immediately earn a salary the size of their parents’.
7.1.3 Learn from what people do
Now we need to find out whether there is a market for our product, and what the dynamics of that market are. This will be the hardest and most important question to answer.
We have a very simple belief about this area of learning: when it comes to assessing whether consumers will purchase a product or service, the only strong predictor will be whether they have actually done it before. Therefore, to learn about whether a product is likely to be a success from a customer point of view means finding a way to put the product in front of potential purchasers and getting them to react to it as if it were real.
What we must not do is rely on asking customers to predict what their behaviour will be (without seeing something); neither must we try to infer likely customer behaviour by observing a different product in a different market.
As Christensen says:
‘Markets that do not exist cannot be analysed: Suppliers and customers must discover them together. Not only are the market applications for disruptive technologies unknown at the time of their development, they are unknowable. The strategies and plans that managers formulate for confronting disruptive technological change therefore should be plans for learning and discovery rather than plans for execution. This is an important point to understand because managers who believe they know a market’s future will plan and invest very differently from those who recognise the uncertainties of a developing market.’ 21
He continues: ‘Guessing the right strategy at the outset isn’t nearly as important to success as conserving enough resources (or having the relationships with trusting backers or investors) so that new business initiatives get a second or third stab at getting it right.’ 22
126.96.36.199 What people say, and what they do
‘In the mid-1990s Michael Moore’s TV show pioneered a post-modern playfulness with dumb research event polls (“46 per cent of Americans said they would rather be killed by a serial killer than by a mass murderer”).’ 23
We need to learn about how a potential market may react to a new product, under what circumstances consumers might buy it, how much they might be prepared to pay for it, and so on.
This is a very lucrative business. In the UK, £1.3bn is spent annually on market research. In the US, the number is over £11bn. In the UK, one study by the Department of Health alone cost more than £11m.
In his book, Consumer.ology,24 Philip Graves argues that all, not just a proportion, of this money is wasted. Asking people to describe what they have done in the past, do today or will do in the future is — he believes — very ineffective at actually understanding what people do or will do.
There’s a lot of evidence that he is right.
What we have seen is that there is a world of difference between what people say they will do, and what they actually do.
Why is this?
Because answering questions for a researcher is a fundamentally different thing to actually making decisions. Asking research participants to respond rationally is not asking them to respond as they will in the market place because we are very often not rational in our own decision making, and very often we simply do not understand how we have arrived at own decisions.
In a research context, respondents will try appear as if they have a clear and consistent decision-making process, even though the evidence suggests that decision making is — in reality — of a much more visceral nature.
To confuse matters, respondents will often post-rationalise their decision — explaining the ‘reasons’ for their actions to an attentive researcher, an effect that will be compounded in a group setting.
The respondent does not want to appear like someone crazily making decisions on impulse as if afflicted with a weird form of retail Tourette’s. And so the accounts of making decisions are sanitised and post-rationalised to create a disarmingly lucid account of a process which never actually occurred, a beautifully constructed fiction intended to impress the researcher and other participants, while helping the subjects feel as though they are capable of highly coherent thought rather than being a creature of uncontrollable urges and whims.
Graves cites a University of Virginia study where respondents were asked to select their favourite of four pairs of tights (pantyhose). Having made their selections, participants gave explanations ranging from sheerness to knit to elasticity, although — in fact — all four products were identical.
This is not the only serious flaw in traditional ask the customer research methods.
If the research has been commissioned to validate decisions already made, and the researcher is actively seeking evidence to move the project forward, just how much of what is being said is really being listened to?
When judging desirability, we are faced with a further and equally intractable challenge.
Humans are very poor at predicting how cost will affect their decisions. ‘I would like to offset my impact on the environment’ and ‘I am in favour of eradicating world hunger’ are both difficult statements to disagree with in the absence of an actual, tangible cost. Do you know anyone who is not for the eradication of hunger? Now try knocking on their door and asking for a donation towards the world food programme.
The ability for research to misjudge the likely commercial reception for a product works both ways. Over the years many products, including Bailey’s Irish Cream and the Aeron Chair were rejected by focus groups but embraced by real-life customers. Once again, customers are found to be unable to predict how they will actually respond to the product when they meet it for real, and when they experience not just the product itself, but its marketing and the reaction that others have to it.
Perhaps one of the most famous examples of product research failing to predict market reaction is New Coke and the challenge they were facing from Pepsi. Malcolm Gladwell points out that when people are asked to sip a cola, they are naturally inclined to prefer a sweeter taste but that they do not maintain this preference when drinking the cola regularly and in larger quantities. So Gladwell argues that this is why people favoured Pepsi in blind taste tests, yet consumers reacted badly when Coca Cola shifted their formula to be more like Pepsi.
Greeves has a more radical theory. His view is that the research stood as much chance as being wrong as it did of being right since those sampled were intrinsically unable to predict market reaction to the change itself.
In either case, it seems there is a case to be argued that those who liked the taste of New Coke would probably not prefer it as dramatically as those who disliked the changed formulation — either for taste reasons or because they are simply change-averse — and that this group were always bound to be more vocal. In fact, the research could have predicted this — since the same effect was seen in the groups — but Coca-Cola failed to interpret such a response as significant.
Another Gladwell example is coffee. In his 2006 TED talk on taste,25 Gladwell says: ‘The mind knows not what the tongue wants. […] If I asked all of you, for example, in this room, what you want in a coffee, you know what you’d say? Every one of you would say “I want a dark, rich, hearty roast”. It’s what people always say when you ask them what they want in a coffee. What do you like? Dark, rich, hearty roast! What percentage of you actually like a dark, rich, hearty roast? According to Howard [a consumer taste test specialist], somewhere between 25 and 27 per cent of you. Most of you like milky, weak coffee. But you will never, ever say to someone who asks you what you want — that “I want a milky, weak coffee”.’
For all the talk of Steve Jobs’ focus group of one at Apple, his attitude was in fact that you need to understand who your customer is, what their life is like and what they would love to be able to do with technology. Using this information allowed Apple to innovate and design phenomenally successful products. As he said in Fortune Magazine in January 2000:
‘This is what customers pay us for — to sweat all these details so it’s easy and pleasant for them to use our computers. We’re supposed to be really good at this. That doesn’t mean we don’t listen to customers, but it’s hard for them to tell you what they want when they’ve never seen anything remotely like it. Take desktop video editing. I never got one request from someone who wanted to edit movies on his computer. Yet now that people see it, they say, “Oh my God, that’s great!” ’
So, what are the key lessons for research?
- Don’t use research to design your product, do it to investigate your customer and understand them
- Don’t use research to prove you are It always will. Instead try to find a question or hypothesis you can learn something about and be very careful to remove as many biases as possible from the experiment.
- Don’t expect participants to predict the future or remember accurately why they did something in the
- Don’t believe the reasons people gave you in research to explain their decisions and actions.
- Beware research taken to delay an action or decision, often senior exec will call for research to be undertaken not to learn more but to confuse and slow down a process that they don’t buy If someone wanted to kill Jobs’ video editing software they would have commissioned research simply asking ‘How often are you likely to edit videos at home?’
- Be very careful about which kinds of research you carry out. In particular, there are two types of research of which we are extremely cautious, even though they are very commonly used in the development of new products;
188.8.131.52.1 Focus groups
The idea behind a focus group is pretty straightforward. You recruit a bunch of people who match a set of qualifying criteria. You get them together and you ask their opinions about the market segment you are interested in, or about a product you have developed. The participants in the study are able to interact with each other. Whilst this interplay can occasionally create issues with moderation of the group, it is typically seen as a positive, allowing participants to develop their viewpoints.
A traditional approach is to show early designs / prototypes of a product in such a focus group setting, and ask users to provide feedback.
While focus groups may be very valuable for other things, we are not confident in their ability to help us either understand potential customers or test new products. In terms of understanding customers, we find focus groups unreliable because of the impact the group dynamic has on people’s behaviour. Unless you are creating a product inherently geared to groups (for example, a restaurant format), customers do not make the decisions in such a group context. Yes, they are certainly influenced by friends and relatives but not in the quite the same, self- conscious way that people react with others in focus groups.
But it is in product testing that groups are particularly unhelpful. For the reasons outlined in the previous section, our experience is that customers do not really know how they feel about new products when presented with them in this context.
So, asking them this question is unlikely to create reliable results. Instead, you are likely to get results to the experiment What happens when you ask members of the public to act like product designers? There may be some useful outputs from the exercise but it will not answer the question — will people buy or use this product?
As Steve Jobs himself said: ‘In the end, for something this complicated, it’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.’
We would go one step further than that. Simply showing people the product and asking for a hypothetical reaction may provide different results from putting the product in their hands in a non-research setting. Such an approach also lacks any of the influence of marketing and prior information which customers are presented with in the market, making it — at best — half a test.
184.108.40.206.2 Usability testing
The field of usability testing has come an incredibly long way since the early 1980s. Researchers can now reliably find usability issues and even — using new techniques such as ECG scans — figure out where products particularly challenge, stress or delight a user. In such tests, the user’s ability to achieve a set of tasks will be assessed. For example, the user may be asked to find a piece of information, or purchase a particular product on a website, or use a physical or electronic product in some way.
Such apparently simple tests can often reveal the most surprising flaws which had previously been invisible to those involved in a product’s development. Their results are as valid for new products as they are for products that have been around for years, however, don’t use usability testing to ask the wrong questions.
What usability tests provide is information about whether the user is able to make the product work in the way the inventor intended. That’s all well and good, but we can’t in this process also look for information about whether the user would choose to use the product in the first place. There is no point developing a highly usable product that no one wants. Usability testing, therefore, has a role in the late stages of product development, and not to answer the big fundamental questions of stage one.
There seem to be three reasons why focus groups and usability testing have become popular in product development:
- They are relatively inexpensive and easy to understand. Marketers in particular will be used to using these techniques.
- They are relatively easy to rig — either through influence or interpretation and either deliberately or otherwise. We have many times seen the outputs of such groups dismissed when negative for the most spurious of reasons. Particularly in focus groups, we have seen senior marketers — slightly sozzled behind the two-way mirror — mocking their customers on the other side of the glass for failing to understand the product or promotion being ‘assessed’.
- The hammer and nail issue. If the agency you hire to research a product has a user testing lab and a focus group room, does the likelihood of these techniques being recommended increase? Even more likely to bias the research methodology are the skills of the researchers and the techniques they are most comfortable deploying.
In fact, we see this bias in research regularly. As Rory Sutherland puts it with the apocryphal story of the drunk and the lamppost:
‘We have all heard the adage about people who use research as a drunk uses a lamppost — for support rather than illumination. Yet there is a better story about drunks and lampposts that David Ogilvy used to tell. A drunk had lost his keys on the street and was frantically searching for them under a streetlamp. “Where did you drop them?” asked a concerned passer-by. “Over there,” he replied, indicating a spot 30 yards away. “So why are you looking here under the lamp?” “The light is better here.” ’
220.127.116.11 Testing feasibility
Once we have a product we think customers will love, we may need to solve some problems relating to the feasibility of making it. A little like the imperfect customer prototypes and experiments we have just described, the key feature of technical prototypes should be that we focus on what we are trying to learn about, ensuring that we don’t just try and rush to build an imperfect version of the product with each element or feature ten per cent complete.
For any project, there will be technical elements which we believe can be fairly easily tackled, and — conversely — those which are entirely new to us, or may seem at first to be utterly impossible.
We need to avoid working on the former — rebuilding things we’ve done 100 times in the past — although this may be tempting because these tasks seem much easier and less daunting than the areas we’re trying to explore.
One such Fluxx project is a haptic (sensory-driven) navigation system which started life as a project called ‘Buzz Gloves’. The concept is simple. Could we prompt people to walk, run, cycle or drive correctly to a destination by giving them small haptic hints, such as gloves which subtly vibrate. There’s plenty of room to remove the actual gloves idea later (maybe putting the vibrating signals in something like bicycle handlebars) if required but this was an easy and memorable way to construct the product.
In the first prototype, all we wanted to learn is what it would feel like to have gloves on which mysteriously vibrated, and whether such an invention would be welcomed or despised by the wearer.
There are lots of bits of this that we could have built: internet connectivity, positioning, direction, decision making, user interface for setting up the route and so on. But would we learn anything by building these things, other than how clever and experienced we are?
Instead, we focused on doing the absolute bare minimum to test the idea.
Well, not quite the bare minimum. At one stage it was suggested that the test subject would walk along the street being followed by the ‘navigator’. The navigator would have a long stick and would tap the test subject on either the left or right hand, depending on which way they should turn.
Now this test would have worked but we felt the weirdness of being struck with a stick in the street would make any other feeling or sensation difficult to detect. Instead, the team built a super-simple remote-control rig. Using an abandoned remote-control helicopter, a few wires and the vibrators from a pair of dancing hamster greetings cards, we made it possible for the navigator to vibrate the gloves of the test subject remotely.
We used this simple set up a number of times to control subjects walking in and around the St Paul’s area of London, near where our office was at the time. With hindsight, the appearance of a person walking aimlessly around the Old Bailey (the UK criminal court reserved for the most serious crimes) with wires hanging out of their back pocket may not have been the wisest choice, although perhaps better than the person being apparently beaten with a stick.
The experiment was very useful, it cost virtually nothing and we were up and running in less than half a day. The tools ended up in the office bin. This is inevitable. The key is making sure they were inexpensive in the first place. Again, reducing the ‘I’ in ROI to its absolute minimum.
18.104.22.168 Dealing with conflicting results
We’ve painted a picture here of a very predictable set of tests and interactions.
Of course, the reality is far from predictable, and the three types of learning are often interdependent.
In looking into consumer desirability, we will often need to radically rethink our technical questions or business model. Technical tests will rapidly impact on viability, and so on.
So, again, it will be important to maintain imagination and flexibility, as well as logic, throughout the process.
7.1.4 Build a team to learn
Let us describe the ideal person for an innovation project. It is someone who likes to learn, someone who is not afraid to try new things and someone who does not become obsessively attached to their own ideas. It’s someone who can bear others to be successful in a project team, and who is not afraid to rapidly change what they are doing. Depending on what area you are working in, it may well also be someone who has certain specific domain or technical skills.
There shouldn’t be a shortage of people who can fill these roles. But that’s not to say that we shouldn’t focus in on the most suitable for this work. Should the opportunity to be involved in new product development be sought after in the business? Absolutely. But it should also be within the grasp of all those you value most in your workforce or, preferably, all those on your workforce.
It should certainly not be the preserve of the most ruthless ascenders of the greasy pole, the alpha males and females who rack up accomplishments inside in the organisation. Indeed, the need for openness and acceptance of the ideas of others eliminates exactly these corporate climbers. On many occasions, we’ve seen those from the factory floor or frontline of customer service be just as, or more, capable of providing useful input into the process.
22.214.171.124 Picking a project leader
Often, a team will have a project coordinator or manager whose role it is to organise things and remove issues that the project team is facing (sometimes referred to as ‘blockers’, a term from lean software engineering). This is not the leader. The leader is the person who chooses what the team will pursue, when they will change direction, how they will map out their progress and what they need to learn. The leader is also the person who must report progress to executives. It is the role that must constantly educate a variety of audiences about why the approach is being followed, effectively selling the approach to the organisation. More often than not, it will be this person who needs to safeguard the project from the parent organisation wishing to revert to form (especially in terms of forecasting, budgeting, outcomes and management).
So, it’s a role that demands experience of the parent business, faith in the approach (without sight of the outcome), an ability to convince and motivate both senior stakeholders and the team themselves, and the ability to report positively on outcomes that may be regarded as negative.
Often, the behaviours associated with alpha managers who progress quickly in large organisations are sharply at odds with such a role, in particular with the ability to associate themselves with outcomes which some would determine to be failures.
In short, it is a particularly demanding role and one which does not automatically sit well with the sorts of people who do well in more traditional large businesses.
For this reason, we often find that the role will be best taken initially by an outsider with significant industry expertise but who has likely gained that from other companies and partners. Alternatively, the leader may be a relative newcomer to the business who is yet to learn about all the things that other staff regard as impossible. Here, the challenge becomes ensuring that this non-insider can command executive credibility, and ensuring that the team themselves respects and follows this leader.
126.96.36.199 Rational versus imaginative
Rory Sutherland, the enigmatic and expansive Vice Chairman of Ogilvy Group once said:
‘Sutherland’s first law states that “All creative people must submit their thinking for appraisal by more rational people”. The second law states that “This does not apply the other way round”.’
He goes on to suggest that some of the dumbest decisions in the history of the world have been made because of the lack of creative, not rational, thought in the process. The observation is well merited.
‘… I sincerely believe that a relentless application of logic, untempered by imagination is responsible for the greatest absurdities and extravagances we see in business and government. The [UK] 3G [bandwidth] auction; NHS target-setting; the ERM [European Exchange Rate Mechanism] debacle; obsessive punctuality targets for trains — all have been perpetrated by people following the relentless dictates of logic without an imaginative grasp of the alternatives. And logic — unlike creativity — is allowed to go unpoliced.’
The correct application of creativity in innovation is the subject of much debate but it is clear at least that the creative process begins, not ends, with the creation of powerful ideas.
What we have learned is that the idea behind a revolutionary innovation is only the seed of the final product that may or may not succeed in the market. It is the detail of the proposition and execution which, more often than not, will define the product’s success. The ability to make more, to add, to rethink, to reset and restart, rather than to slash features and aim for the tried and tested, is so often the hallmark of genuinely exciting innovation.
Because there is no real coherent answer to the question ‘how did you come up with that?’ creativity can all too often disappear entirely from the accounts of how successful innovation was achieved. Again, case histories have their role in this distortion.
Sir Ken Robinson, an educator and public speaker, has it right in his analysis that education and business can try and kill creative thinking in favour of the rational and scientific. The skills we all are born with, the things we all do naturally when we are small children are gradually beaten out of us: ‘Imagination is the source of every form of human achievement. And it’s the one thing that I believe we are systematically jeopardising in the way we educate our children and ourselves.’ 30
A team that is likely to succeed will have a good mix of creative and analytical minds, as well as a healthy tension and respect between the two. Bear in mind that these two thinking modes are not mutually exclusive in individuals and you won’t often have teams you can reliably sort into two discernible groups.
Free from the traditional bias of business management towards the rational over the imaginative, all too often, the teams will re-impose the restriction on themselves when they start to think about how the rest of the business will perceive them. It is the job of the project leader to keep that tension running.
188.8.131.52 Allow them to act responsibly
Let’s get one thing clear. There is absolutely no way that a product team will be successful unless the team members involved all act responsibly. This shouldn’t be surprising. As we set sail in our new venture, we will be short-staffed and faced with enormous challenges. We will need to do a lot with a little. We simply can’t afford to waste time nor add additional unpredictability.
This shouldn’t be too difficult to achieve. In our experience, it is the average person’s default position. Most of us don’t start work with our political skills finely honed. We learn these somewhat counterproductive skills over time. The opportunity to shed these concerns is liberating. But in order to do so, we must create a work environment where the essentially defensive measures of office politics are not required.
So status, reward and job security need to be taken out of the equation. Of course, this is more easily said than done. We will need to make some promises:
- The innovation project is like a maternity leave, but for ideas not offspring. The team will return to their day jobs, or That is to say, if they so choose they can go back to the roles they previously held on the same salary, benefits and status. The foray into innovation is a temporary one, at the staff members’ discretion. A project failure must not necessarily represent a personal failure, so it should not have a negative personal outcome.
- Should the team receive a salary bump for joining the new business project? Our view is It should be an opportunity in and of itself, not one for which financial incentive is needed. By removing the potential downsides of the secondment, we make such lateral moves more attractive.
- To reduce jostling, it is typically best to remove concepts of status as far as possible from the team. Luckily, small teams don’t need a lot of structure, just one Focusing on the leader’s ability to motivate, lead and manage disparate skills is therefore key.
- We’ve already mentioned that the staff should not be rewarded based on the outcomes of their Rather they should (singularly and collectively) be rewarded for their behaviour in the project such as asking intelligent questions, effective design of experiments, clear interpretation of results and the ability to confidently answer ‘I don’t know’ when asked to make crazy predictions.
- As far as possible, reduce the process to small, self-contained projects. The leader of a project or group of projects should be able to redeploy teams depending on the skills of individuals and how well they work
- Have clear plans up front for what will happen if the current stream succeeds or fails in gaining the green light to If you don’t do this, staff will invent their own outcomes. For example, if our plan means that the team will go back to their day jobs if the project produces a negative outcome, we may have inadvertently created an incentive to carry on with the work longer than necessary or to interpret the results in too favourable a light. Perhaps it is better to have staff move on to the next stream if this one has shown it will not work. This provides a strong motivation to report findings quickly and honestly.
- Put in place a clear management structure for the teams in question. In particular, while employees are on the team, they will be subject to management by demonstration of values, rather than management by The result of a disciplinary action would be a return to the previous role, and would remain on their long-term records. This is the sort of failure we care about, and it should be very clearly separated from the perceived failure of speculative new products.
- The values and behaviours targeted can be: generating and documenting learnings; sharing of learnings in the team; avoidance of the political behaviours; being on time and the reputation of the team itself in the wider business. At some distance, this may seem obvious, but up close it rarely is. Incentivise the behaviours that are likely to result in success, and that’s it.
Success in the innovation stream (by behaviour) should be rewarded on the return to the regular business if the project closes.
184.108.40.206 Prepare them for failure
‘Success is the ability to go from failure to failure without losing your enthusiasm.’
‘Everybody has a plan until they get punched in the mouth.’
Much has been made of the ability to fail fast and of the likelihood of encountering many so-called failures before eventually hitting upon a success. That said, you don’t really want to make a habit out of failure. Rather we must redefine what it means to fail. As with the Churchill quote above, the concept that we don’t not fail but rather learn is not a new one. Edison famously said: ‘I have not failed. I’ve just discovered 10,000 ways that won’t work.’
Resilience, determination and a positive attitude are key. And that means, as an employer or leader, you must provide reinforcing structures for this sort of approach and attitude. From every negative experiment outcome, we must make sure we learn, either improving the idea we are evaluating or the evaluation process itself. The only crime should be learning the same lessons twice, or not learning them at all. As it is expressed in The Other Side of Innovation (p. 18): ‘[…], the innovator’s job cannot be to deliver a proven result; it must be to discover what is possible, that is, to learn, by converting assumptions into knowledge as quickly and inexpensively as possible.’
While detailed project planning is very important once we commence design and build, and even more important in making an idea work in production, too much of an assumption about what planning can bring you is virtually a liability. Again in The Other Side of Innovation (p. 99), this is expressed very clearly: ‘… the competitor that wins is rarely the one with the best initial plan; it is the one that learns the fastest.’
220.127.116.11 Insiders or outsiders?
At first glance, candidates from inside the business should always be a better bet for innovation teams — they know the business, what they learn will be easier to retain, and they may even look cheaper than outside candidates. On the reverse of that equation, once we’ve put in place the considerations mentioned above, are they really cheaper for the project? Will it be easier for outsiders to see the possibilities in new ventures or to worry less about maintaining existing job roles and the general status quo? And couldn’t outsiders who prove their worth be eventually converted to insiders?
The answer is that a mix of insiders and outsiders is best. Some may even be temporary inclusions in the team (such as outsiders from a company like Fluxx). But a great deal of care is needed here, around mixed incentives and conflict of interests.
Let’s say a team has been made up of insiders, outsiders and consultants. How are the consultants rewarded? More often than not they have an interest in either the project or programme being extended (more consulting is better than less), and they may too have an interest in a particular outcome (for example, other employees in their company might benefit if a technology is chosen, or if the consulting company is selected for a follow-on project). The economics of consultancies and agencies is often based on the strategy part of projects breaking even to produce profits in the production phase. If there is no production, there may be no profit incentive. Agencies then are naturally biased to encourage their clients to put their ideas into an expensive design and build process employing tens more staff than were involved in the initial stages.
It is very important to be aware of such biases and inclinations before building teams with consultants in, much as their presence can often be very valuable.
The message, internally, should be that insiders have been seconded to innovation projects on their merits — that is, they have certain skills and knowledge.
It is also important to emphasise that those chosen to work on innovation projects have been selected, in part at least, because of their behaviours — openness, honesty, integrity and the ability to operate in a team. These behaviours are vital to the team; members who do not exhibit them should not be recruited, and in modelling the behaviours we can only hope to improve the quality of candidates in the future.
7.1.5 Do something
Compelling, if blindingly obvious, advice is that the team must get on with doing something and in doing so should make some progress on the idea. As soon as possible they should get their hands dirty learning from doing, not from philosophising.
The temptation to avoid action is high. At every turn, we will find plenty of good reasons why we should do nothing. Perhaps another company has attempted the same endeavour and declared it a failure. Perhaps a competitor is sniffing around it now — so we’re too late already.
Humorist Ze Frank popularised the term ‘brain crack’ to refer to the concept of keeping ideas in your head so that they can avoid suffering from the disappointment that ideas tend to attract in the real world:
‘I run out of ideas every day! Each day I live in mortal fear that I’ve used up the last idea that’ll ever come to me. If you don’t wanna run out of ideas the best thing to do is not to execute them. You can tell yourself that you don’t have the time or resources to do ‘em right. Then they stay around in your head like brain crack.’
No matter how bad things get, at least you have those good ideas that you’ll get to later.
‘Some people get addicted to that brain crack. And the longer they wait, the more they convince themselves of how perfectly that idea should be executed. And they imagine it on a beautiful platter with glitter and rose petals. And everyone’s clapping for them. But the, but the, but the, but the bummer is most ideas kinda suck when you do ‘em. And no matter how much you plan you still have to do something for the first time. And you’re almost guaranteed the first time you do something it’ll blow. But somebody who does something bad three times still has three times the experience of that other person who’s still dreaming of all the applause. When I get an idea, even a bad one, I try to get it out into the world as fast as possible, ‘cause I certainly don’t want to be addicted to brain crack.’
How real is this motivation in business?
The idea of ‘brain crack’ might help to explain why so many of the best ideas of your business have been heard 100 times but no one has ever acted upon them. In many companies a kind of corporate brain crack is perpetuated by neither admitting any kind of failure nor being transparent about the whole process of selecting and evaluating ideas.
The main thing is to do something and at least make some progress, for good or for bad. The point of the process is that we know all of our ideas are pure speculation and the intended target audience no more than guesswork.
We could keep guessing ‘til the cows come home, or we could start eliminating some of the guesses… which is what we should do next.
So, when we were working on Buzz Gloves, we picked cyclists as the first test market. We heard quite a bit of armchair philosophising about how cyclists weren’t the right choice, they already had good alternatives etc. But ultimately, we could theorise forever, and probably scrub out each potential market that way. We were better off sticking with cyclists and learning something rather than doing nothing at all.
Now that you’re committed to doing something, do yourself a favour and do it quickly. Although lean product development isn’t explicitly about being fast, if you reduce the amount of waste and inefficiency then you will be a lot faster getting to a better end result.
Virtually every element of our prescription delivers faster progress than the alternatives. Better proposition development may take some time in the preparation, but will actually shorten the overall product lifecycle. It will clarify and focus the build stage, making it deliver better and, more likely, on time. Experimenting gets answers faster than building full launch products. Understanding consumers through real exposure is faster and more reliable than long-winded focus groups, or even building fully-fledged prototypes.
Why be fast? Well, mostly because we hate wasting time. Time wasted on drawn out and unnecessary tasks is toxic; it encourages innovation teams to feel that the whole project is a waste of time and it makes people lazy. The measure of much of our activities during our work at Fluxx, is ‘Did everyone invest their time well? And did we make good progress in good time?’
Of course, there is a business implication too. The faster a product is brought to market, the less likely it will be beaten by a competitor and the more likely the analysis of the market will remain true.
So be as quick as you can, without missing out any of the important stuff.