Startups Get User Behavior Wrong and What To Do About It
Dan Ariely and NFX wrote an in-depth post about the pitfall of assuming user behavior that simply does not match reality. Read on for 3 Ways your startup can avoid failure.
Most startups look back in hindsight and realize they held onto poor assumptions about user behavior for too long.
Nobody would ever run a biology startup without having a biologist on board, but people have no problem with a startup that is ultimately about consumer behavioral change but doesn’t include anyone on the team who knows about behavioral change. It is surprising. — Dan Ariely
1. Test Your Assumptions before you marry them
A startup is based on belief and faith. As a result, it is naturally very hard to divorce assumptions. People are wedded to an idea and, as a result, it can take months to get away from a bad assumption.
Knowing this failure mode, design intentionally to test the assumption before marrying it.
Example:
A 2-sided labor market startup was helping workers pick up potential shifts at nearby employers. As a transactional marketplace the team asked how might it help workers pick up more shifts? One idea was to build a matching algorithm that would ask users when they wanted to pick up shifts and then send them shifts that matched their requirements. With the engineering horsepower to build sophisticated matching algorithms on hand, the idea had great momentum.
But rather than marrying the idea right away, the product lead asked engineering to test the assumption with a toy (throw-away) implementation which engineering could build in a couple of days. To the user it looked like the product feature, but it was non-scaleable implementation. The team enrolled a subset of users in the trial, gathered their shift preferences, started sending out shifts that matched those requirements, and watched what users actually did.
The results were astounding! Users simply did not accept shifts that matched their preferences! Not even if they exactly matched! In some cases users picked up shifts for times that they said they did not prefer! The acceptance rate of matched-shifts was no higher than for random shifts.
The team realized that their users were not planning out the week at all. The real behavior for users was spur-of-the moment decisions (what they named the “whimsical” segment). It was easy to drop the idea at this point, because no-one had married it yet. The team learned a lot about users. And they weren’t stuck with a high-cost feature that, once launched, you keep investing in to try to make it work.
2. Talk to failures
We’re always looking for successes to borrow ideas from. Doing the opposite — looking for failures — is incredibly helpful. Ideas that your startup is considering were almost certainly tried by someone else. Look for those companies and talk to them.
It’s not hard. In my own start ups I’ve found good advice from product managers everywhere — from startups and from very large companies. They will share what they learned the hard and painful way about user behavior. “We were sure of this, but it was a failure.”, “What we learned was ….”. Those lessons can save you going down a path that would have wasted your startup’s scarce resources.
3. Hold the Crazy meeting, regularly
At NfX we’d meet frequently with founding teams — the main 2–4 people in the company. We’d have incredible white boarding sessions, producing clarity of thinking and prioritizing the most important paths to go down. We did 20–30 of these every week, week after week. It was consistently very successful. I wondered why this worked so well?
Part of it was the knowledge and frameworks that NfX was providing. A big part was, I realized, the element of freedom. The teams did their best thinking when they were meeting with us, in these short white boarding sessions. When meeting with us everything was on the table. We could ask any question. We could think broadly. We could admit what was not working. We could be creative. We could talk about what worked in other companies and consider applying it here.
In my last start up I wanted to recreate that experience from the outset. We didn’t have the benefit of an external advisor like NfX, but we created the same dynamic amongst the core team. We intentionally started “The Crazy Meeting”. This was completely outside and orthogonal to all the usual types of interactions and led us to break throughs. Some tips on doing this:
- Say “This is the crazy meeting”
- Meet somewhere completely new
- Whiteboards and drawing space
- Avoid day to day topics (people will want to talk about something urgent— but kill that thread)
- Keep it crazy. Depending on the personality of your team, figure out how to keep them in the crazy zone. When Uber was in the news for cutting edge tactics, I asked engineers, “What would Uber do?”. It was a way of making it clear that nothing was off the table.
- Don’t have an agenda. This was a big one — I tend to be the person who wants an agenda, along with pre-reading and structured conversation. But for the Crazy Meeting there was only one sentence.
- 90 minutes provides enough time to explore, think and whiteboard.
Conclusion
Most startups look back in hindsight and realize they held onto poor assumptions about user behavior for too long.
If you build these habits in your start up you can avoid a lot of problems. You will start to build a culture that is truth seeking (my favorite words in a start up) and able to zone in on true user behavior.
