Embrace failure: inform research using a Pre-Mortem

A pipette fills multiple test tubes ahead of experiements

“I have not failed. I’ve just found 10,000 ways that won’t work.” — Thomas A. Edison

One of the most effective ways to kick off a new initiative is to identify how it might go wrong. Similarly, when planning research goals and methodologies, awareness of potential weaknesses can help to establish immediate learning goals.

The Pre-Mortem is a well-established facilitation tool for use early in a project life cycle. As with a number of similar tools, the Pre-Mortem makes use of future projection to address the present.

If you are unfamiliar with it, the activity works like this: an assembled project or product team is asked to imagine a version of the future, a number of months down the line. The project/product/feature has shipped… and has been an abject failure. It wasn’t bought, it wasn’t adopted — whatever failure looks like for the issue in question. The Pre-Mortem then asks, simply: What caused this? Why did it go so wrong?

Eject the elephant from the room

Using a Pre-Mortem, a safe space is created for doubts, concerns or apprehensions to emerge and be expressed as a concern for the future, not as a gripe about the present, even though that’s what it very well may be.

In any team, there are often strongly-held points of view that can seep out throughout the course of not just a meeting, but an entire project. A Pre-Mortem actively invites such thoughts and feelings to be aired. These might otherwise fester, emerging only when there is a bump in the road — “we could have seen this coming!”.

Providing a forum that permits individuals to voice and record their concerns allows these strong opinions to be aired and treated with equity.

“When people have strong feelings about the topic, they often think of the meeting as a contest where their view — which they see as the correct one — should prevail. That leads them to try to convince others that their solution is the right one.”

– Roger Schwarz, author of ‘Smart Leaders, Smarter Teams’¹

While gathering doubts and concerns is a positive step in itself, these can be actively put to work as experiments to find (in Edison’s words) the things that won’t work.

A whiteboard from a pre-mortem workshop, with inputs captured against points in a core user journey
A pre-mortem workshop, with inputs captured around points in a core user journey

Step-by-step

  1. Gather input on post-its . Ask participants to list 3 main factors that will have contributed to the project failing (online this can be done with tools such as MuralMiro, or Google’s free Jamboard).
  2. Group the inputs — take time to find commonalities in the contributions, and collate them into groups.
  3. Identify themes —it may be helpful to think about these 3 categories²:
  • feasibility: can we build, scale and support this — usually this will have a technical skew, but can apply to a broad range of resources.
  • viability: will this be profitable or advantageous for the business.
  • desirability: will customers find value in this.

4. Finally, devise small experiments³, capable of testing problems identified in each area. Be intentional about what you would need to learn, hear or see that will tell you your project will have difficulty succeeding. As a team, discuss the minimum amount of time, capital or resource required to test whether any of the issues represent credible threats to success.

Find what won’t work

If the issue is a real problem, how could you quickly test for it? Even modest pieces of secondary research, prototyping and competitive analysis can qualify, for example:

  • Discover if anyone tried and failed to build something similar before. Have they themselves or others written about their experience? Stories of failure are often well documented.
  • Put low-fidelity prototypes in front of potential customers or users. People testing low-fidelity prototypes are more likely to express negative reactions.
  • Evaluating a market opportunity is a multi-faceted pursuit. But even this can start with a simple examination of existing, successful products which offer similar features, and a realistic assessment of what is required to compete – and where you fall short.

These are rudimentary activities for sure. However, before massive resources are put into motion, such small experiments that address particular elements of risk can challenge flawed beliefs that a product will thrive.

By leveraging the combined wisdom of the team, then gathering corroborating evidence from external objective signals, UX professionals can contribute to de-risking an initiative.

Mitigating risk

The ‘flipped’ nature of the Pre-Mortem is important in establishing a critical mindset. Confirmation bias is a natural human tendency. We look for validation of an idea, and find false positives. Testing for confirmation of weaknesses introduces critical thinking and can counter inherent biases.

“Optionality works on negative information, reducing the space of what we do by gaining knowledge of what does not work.”

– Barry O’Reilly, Optimize to be Wrong not Right

Innovation cannot be manufactured on demand. Success does not follow from throwing unlimited resources and capital at the ‘big idea’. Having more successful ideas is more likely to come from more ideas — small bets — against which small experiments can be run to identify failure points.

No time? No problem

Time more than all resources is limited. If this is the case for you or your team, and follow-up experiments are not an option, try this:

  • Run through the Pre-Mortem exercise until you have a collated set of potential failure points.
  • Discuss mitigations around the issues. Look for actions that can be taken now to alter that possible future. Work towards an actionable plan in the event of anything coming to fruition.
  • Create a reference-able checklist, and review it at each retro or team meeting. If anything highlighted in the Pre-Mortem raising its head, go to your action plan.

Assuming that you have time for the Pre-Mortem itself, the very least you can get out of it is greater awareness. And that awareness can be codified.

Invest in failure

As you plan your stakeholder engagements and workshops, consider investing in a Pre-Mortem. It will typically take anything from 30–90 minutes, and it is capable of revealing that a project is partially or deeply flawed before it begins. For that reason alone it can prove invaluable to your team.


[1]: From ‘Eight Behaviors for Smarter Teams’ https://cdn.csu.edu.au/__data/assets/pdf_file/0008/917018/Eight-Behaviors-for-Smarter-Teams-2.pdf (PDF file)

[2]: From IDEO’s original ‘Three Lenses of Innovation’, introduced in the early 00s https://www.ideou.com/blogs/inspiration/how-to-prototype-a-new-business

[3]: Testing Business Ideas (Bland & Osterwalder, April 2020) is a fantastic resource for understanding how to run experiments against ideas. https://www.strategyzer.com/books/testing-business-ideas-david-j-bland

Unlock innovation with an outcomes-first approach

Drill bits

Some recent reading showcased two simple and powerful ways to boost organisational innovation. Both are proven in their own right; combined, they can offer a robust, end-to-end method for exploring the opportunity space.

I first encountered Amazon’s ‘Working Backwards’ process in this Inc.com piece last year, which I stumbled across again last week.

The Working Backwards framework articulates, through a series of exercises and artifacts, the desired outcomes for a given project or initiative. Characterised by a narrow focus on customer needs, it is an almost literal interpretation of Steve Jobs’ quote from 1997: “You’ve got to start with the customer experience and work backwards to the technology.”

Amazon CEO Jeff Bezos suggests that constructing representations of the outcomes the product will deliver is a better initial investment than any line of code. It forces those outcomes to take centre stage, as a prerequisite to any discussion of implementation.

Done correctly, working backwards is a huge amount of work. But it saves you even more work later. The Working Backwards Process is not designed to be easy, it is designed to save huge amount of work on the backend, and to make sure we are building the right thing.” – Jeff Bezos


Similar themes emerged in another forum a few days later. I was in conversation with Jim Kalbach about his book ‘The Jobs To Be Done Playbook’ for UX Belfast.

As with Working Backwards, the power of Jobs To Be Done (JTBD) lies in a deceptively simple premise. Rather than assume that everyone involved knows the fundamentals around a project or initiative, it demands that any assumed knowledge is set aside, and insights rebuilt through the lens of customer goals, or ‘jobs’.

Typified in Levitt’s famous quote “No one wants a quarter-inch drill bit they want a quarter-inch hole”, JTBD offers a refreshingly pragmatic alternative to an all-too-common approach where, armed with technology, enterprises go looking for a problem to apply it to. To use a second tool-based metaphor, “When all you have is a hammer, everything looks like a nail.”

JTBD requires that an organisation shifts from a mindset of “how can we sell our technology to customers’ to “what outcomes are customers trying to achieve and how can we adapt to become more valuable to them?”. The book presents multiple techniques for reframing challenges in this manner.

Kalbach states: “Before creating solutions, define the value you’re going after. Regardless of the starting point for innovation in your organization – either with a need or with a technology – you’ll need to align around value creation.”

Mapping customer ‘jobs’ requires understanding their intent, their ultimate aims, and then strategically positioning your own solution into the process.

The two frameworks are eminently compatible. JTBD can provide the ‘true north’ for Working Backwards to navigate to.  And these are just two of many frameworks which can be combined to unlock innovation in an organisation.

I have shelves and devices filled with books and articles on thinking frameworks, gathered over the last dozen or more years. If these tools share a common trait, it is one of simple, structured questioning. So simple that it is often a challenge to have them accepted as a credible route to progress.

Asking basic questions can be seen as facile, but more often as an unnecessary obstacle as everyone rushes to get into the weeds.  It is significant, then, that Amazon and increasing numbers of groundbreaking tech companies give store to these techniques.

There may be no silver bullet for success, but there is a sure-fire route to failure – the inability or unwillingness to ask fundamental questions.

  • “What value are we creating?”
  • “Is this solving a real problem?”
  • “What assumptions or limiting beliefs are we working with?”
  • “What outcomes do customers want us to provide? What do those look like?”

Without the capacity to stop and address these kinds of topics, companies are doomed to pursue a ‘build it and they will come’ strategy. Sometimes that works out. Many times it does not.

The irony of the ‘Working Backwards’ moniker is that it makes much more sense than trying to create something from the ether, armed with nothing but assumed knowledge, some smart technology, and marketing muscle.

As suggested in the Bezos quote above, asking hard questions is not easy. Many enterprises bypass this completely and head straight for a comfort zone of technology first. Recent history is rife with impressive innovations that led with technology but failed to solve a real problem or deliver a desirable outcome. Just ask Segway, or Google Wave, even Google Plus. 

Both Working Backwards and JTBD encourage teams to forget the technology and instead focus on what customers want to achieve, and where value can be created.

Unless such fundamentals are addressed, there is no credible foundation for product development. And yet so many are prepared to take the risk and just build.

It’s hard to imagine anything more backward than that.

Alignment above all

Gears in an engine

Technology may change rapidly, but the fundamentals of effective communication not so much.

The role played by clarity in achieving successful results should not be a surprise. What is surprising is how often it can be overlooked.

Regardless of its nature, the success or failure of any initiative will rely to a large extent on the degree of alignment amongst those involved. The question “what are we building” can yield as many answers as there are factions, and this is true of teams from enterprises to startups.

Executive decisions are rightly expected to be followed by action. But the fear that accompanies an instruction to proceed often kills the opportunity to think deeply about desired outcomes. Agile dogma tells us we need to move fast. The fear of being seen to not be moving can generate countless docs and meetings as attempts are made to assert the rightness of one set of priorities over another. Lots of noise, very little signal, and a dearth of alignment. 

The lure of consensus always looms large. After all, consensus can begin to get everyone moving. At its worst consensus is a form of groupthink. It may be capable of placating a fractious group and get them through to the next meeting. It is unlikely to impress customers however, who won’t care if you reached consensus, but will certainly sense a lack of alignment in the construction of their product or service.

Consensus is tactical; alignment is strategic.

Alignment involves orientation to a direction and commitment to a purpose. It can’t be articulated in purely marketing terms. Or design terms. Or engineering terms. And it must embody unambiguous support, across functions, to the aims of the project. Alignment should release every discipline to plan its contribution to clearly understood outcomes.

The good news is that achieving alignment is not alchemy. Any number of tools or frameworks can assist with driving a team, or multiple teams, towards an aligned state.

As both an exponent and practitioner I will always promote the design thinking process as a fast track to alignment. Your mileage may vary; anything capable of cutting across disciplines and reframing a challenge in objective terms is a positive step.

The most successful teams I have encountered used value as the ultimate framing – specifically, how is it being created or enhanced for the business, and for customers. Anything less than clarity around both of these will lead to muddy goals and unsatisfactory outcomes. We all deal with ambiguity in our working lives. But where this translates to vagueness, danger lies.

“What problem are we solving?” is an easy – even obvious – question to ask, but it can be problematic to answer definitively. It may even take an unpopular pause for breath to do it. To quote Covey, “With people, slow is fast and fast is slow.” 

An aligned team will move faster, and with greater precision, than one without it. 

The investment of time and rigour in achieving true alignment will repay itself many times over. It is worth fighting for.

Founding principles

Quite a number of Monday evenings this year were spent working alongside Big Motive as a design coach on the Catalyst Co-Founders programme.

For those with even an inkling of a product idea, Co-Founders offers invaluable early validation (or otherwise), outcomes can be entry on to fully-fledged incubation programmes, a pivot on the original idea, or abandonment. The latter outcome can still be termed a ’successful’ outcome. Some folks who could have poured months or years of their lives into something that simply had no inherent value learn the harsh truth much earlier than they otherwise would.

Based firmly around the d.School design thinking model, and led by Big Motive, Co-Founders takes small teams through a process which forces them to examine the essence of a product idea. Using tried-and-tested tools a set of tools, they deconstruct the problem they are trying to solve, then reconstruct their solution with greater clarity and objectivity. It may have its critics (and let’s be clear that all successful frameworks and processes do) but design thinking, when fully committed to by stakeholders and teams, the process simply delivers.

I’ve been amazed at the resilience and inventiveness of the individuals and teams. Participants tend to be full-time professionals, with an idea they just can’t let go of. It may or may not be related to their day jobs; but in each case they want to find out if their idea can play out in practice the way they have developed it in their heads. Watching teams go from being at a loss to what they can possibly do next, to becoming resourceful and forceful in progressing their product idea is so rewarding.

I’m not overly familiar with Yogi Bhajan, but my word is he a great source of quotes:

“If you want to learn something, read about it. If you want to understand something, write about it. If you want to master something, teach it.”

I have lectured and coached increasingly over the last five years, and I’ll vouch for this. In many cases, I’ll agree to these engagements because – selfishly – I know how much I will get out of it personally. I’ve been integrating design thinking techniques into client work, and more recently in Puppet, for the last decade. But teaching it, and seeing the enlightenment it brings, continues to inspire me.

Looking back at the year of 2018, I’m filled with a lot of gratitude for all kinds of experiences. Yogi Bhajan has my back again:

“An attitude of gratitude brings great things.”

I attempt to surface this very feeling in all that I do, again for selfish reasons. Gratitude is, I believe, the most sustaining and affirming feeling available to us. We can encourage it and cultivate it. Working with Big Motive is a pleasure and I’m thankful to them for inviting me in as one of the coaches.

Running the UX Belfast meetup has provided me with a number of transformative encounters during the year, not least with attendees who come from all manner of organisations and businesses. Book authors we have spoken with have been utterly fantastic in terms of the amount of time they have been prepared to lend to offer to a small group of designers perhaps halfway around the world. The insights and knowledge they have been prepared to share with us has been similarly impressive.

Most recently, Jorge Arango was particularly inspiring while talking about his book ‘Living in Information’. He has since become another key individual I follow and look up to as I continue to develop as a designer and a professional. I heartily commend his newsletter and blog for its sheer quality of content. And of course the book itself. In keeping with the original topic of this post, here’s a piece from Jorge in November 2018 where he puts forward a model for teaching (and learning).

I’ll cut this short of a comprehensive 2018 retrospective. Suffice to say it was another year in which I was conscious of growing and developing. Thank you to everyone who offered me inspiration in the last twelve months, whether in person, in writing or on video. The experiences of 2018 have given me ideas and new inspiration for 2019. You can’t ask for more from a professional year.

Voice interface is still overpromising and underdelivering

Cortana speakerAround 15 years ago Flash technology was in the ascendancy. One of the odd conventions to emerge at the time was the ‘Flash intro’. Very often, to build your anticipation for the website awaiting you, you would be entertained with what was essentially an opening title sequence. And if you were really unlucky, on the other side of it was a website fully rendered in Flash.

What you wanted was content; what you got was an extended journey through a designer’s ego trip (and yes I should know, I was one of those designers). The basic premise of a Flash-built website was that tricking out the interface would make for a better user experience. That assumption turned out to be wrong.

With Siri, Alexa et al entering our lives, our interfaces now have personalities. If a digital misunderstands our requests we are likely to learn about it through a witty quip. TV ads featuring virtual assistants often make a particular show of droll one-liners emanating from the device.

But as Neilsen Norman Group research shows, voice interfaces are falling far short of user expectations. It seems that priorities need to be reassessed.

A little less conversation

As part of a project last year I began designing for command line interface. With no previous experience, a terminal window or console can be a daunting place. Initially I was puzzled why user prompts and feedback in this world were so clinical and abrupt. Why would command line users not want to be addressed in a more human fashion? The answer lies in task efficiency.

Command line interface evolved from single-line dialogue between two human teleprinter operators. Over time, one end of the human-human dialogue became a computer, and the conventions remained. These interfaces provide users a more efficient method of performing tasks. In short, command line users are just like the rest of us: that is, trying to perform a lot of tasks in as short a time as possible, without surplus dialogue or clutter getting in the way.

This method of working is totally in keeping with our tendency towards ever more concise communication. Email is on the wane due to the long, unwieldy threads it encourages. The rise of chat apps such as Slack is due in large part to the tendency towards more concise messages. We’re making less mobile calls, opting instead for text messages using abbreviations, acronyms and emojis.

Many rivers to cross

As designers we are not always trying to mimic a conversation. We are creating an exchange which delivers for the user as efficiently as possible. To re-cast all human-computer interactions as conversations is to misunderstand our relationship with machines and devices.

The obstacles to success with voice UI are many. Users need to think more than once about the commands they give. They are required to speak in a manner that often isn’t natural for them. Even relatively simple queries may need to be broken down into smaller questions before reaching anything like the right answers.

When barriers are placed between a user and the outcomes they want the end result is predictable: they will simply opt out. A report from The Information suggests that only 2% of Alexa speakers have been used to make a purchase from Amazon in 2018. Additionally, 90% of the people who try to make a purchase through Alexa don’t try again.

We are still some distance away from the dream that voice UI promised. Perhaps this is voice’s Flash period, where the user needs to work hard to access the content they want. And I’m willing to bet that most frustrated users would be willing to trade every ounce of their virtual assistant’s sassy responses for just a little more efficiency.

The fact is that voice UI is still pretty hard work, no matter how hard Siri or Alexa try to entertain us.