Product Management meets Lean Startup

I’m participating in a panel at a Brainmates’ Product Talk in Sydney on 14 April. The topic is “The Intersection of Lean Start Up and Product Management"

So, I thought I’d post a few thoughts on the subject in advance. I welcome any comments, and feel free to turn up on the evening - watch the Brainmates events page for details.

The two key things that make Product Management in a corporate environment different from Lean Startup practices are:

1. You can’t fail fast (as much). You can’t fail as freely as a startup, because you can’t take undue risks with your existing customers, brand and experience. You probably can't iterate as rapidly as a startup due to many perfectly valid business and technical constraints.

2. You can’t maintain narrow single-minded focus. You’ve already discovered MVP, and need to scale and diversify. You’re dealing with a myriad of candidate features, maybe even a portfolio of related products.

But

Where the two meet perfectly is the strong emphasis on evidence-based decision-making. That’s what the lean validation board is all about. It’s exactly what happens when Product Managers specify features based on a testable hypothesis.

And

There are plenty of hacks that allow Product Managers to test hypotheses that are as risky as those of a startup, as rapidly as those of a startup. For example, not all tests need to be live in market - you can validate using survey data and tools like those provided by Survey Engine or Qualtrics

Also

Having achieved MVP, established a validated business model, having real customers and the data that comes with them, probably having competitors with some feature overlap, a Product Manager in an established businesses has a fertile field from which to generate hypotheses. You can explore up and down the value chain or supply chain for opportunities to disrupt. You can try to predict where 'the puck is going’ in terms of disruption in the broader customer journey (e.g. 65% of my referred traffic comes from Google now, but I think in 2 years it will be voice search, or wearables, or Pinterest). You can mine the behavioural data of your real, active, paying customers. You can even talk to them.

In short, Lean Startup principles don’t apply perfectly and undigested to Product Management in established business, but there are key areas in which these principles (or a close analog) are extremely worthwhile, and some of the constraints can be easily overcome.

The Voice of the Customer in Agile

The ‘Voice of the Customer’ is the weakest link in Agile web development.

New advances – which make Choice Modelling cheap, fast and relevant – solve this problem.

The ‘Voice of the Customer’ is a poorly taught and poorly understood component of Agile methodologies. There is no consensus on the tools that should be used to capture it.

Of the methods that are in use, the most rigorous are complex, costly and time-consuming, and can’t be easily and quickly repeated at each iteration. The less rigorous methods are cheap and quick, but the information they produce is actually worthless (we have proven this, as I demonstrate below).

So, measuring the ‘Voice of the Customer’ in Agile web development is generally done either:

-          not at all; or

-          poorly; or

-          fairly rigorously at the outset, then poorly or not-at-all throughout.

But there are now Choice Modelling methods available that allow us to quickly, cheaply and accurately measure the Voice of the Customer, both initially and throughout development.

Let me illustrate by example.

On a recent web project we simply wanted to know which features our customers wanted most. So we conducted a basic online survey to build a choice model. In order to prove this was a superior approach, we conducted a parallel online survey using the common ‘rating’ approach (where respondents rate each feature on a scale of 1 to 5).

Both approaches involved exactly the same cost (cheap), time (2 days) and sample (40 people). The results are summarised below.

The Rating results weren’t statistically meaningful. The features all scored about the same average, and the highest scoring feature was actually the number 6 ranked feature in the Choice Model. These results were either useless, or - if you ignored statistical significance and took notice of the highest average rating – wrong.  

Because the choice survey was so quick and cheap, we’ll be able to repeat it between each sprint, in order to see if (and how) customers’ preferences change as they became exposed to more fully developed iterations of some of the features.

Having seen the benefits of this approach, I think you’d have rocks in your head to use anything else.  The adoption of these new Choice Modelling tools can only be a Good Thing for product innovation online.

I am happy to point anyone who wants to know more in the right direction.