John Egan

Managing a Growth team in many ways can be like managing an investment portfolio. Each individual experiment or project is an investment and the goal is to maximize the long-term return (i.e. impact) of your portfolio. Having worked in Growth for almost 10 years I’ve seen how the portfolio of a Growth team can evolve over the course of several years. In this post I’ll cover the three major classes of projects that a Growth team works on and how Growth teams should be adjusting their portfolio allocation over time based on changing needs.

Three Asset Classes in Your Growth Portfolio

Growth portfolios are primarily made up of the following three “asset classes”:

Iterative Experiments – This is what Growth teams are best known for. Iterative experiments are A/B experiments in an established area. Examples could be changing the color of a button, building a new email type, etc. Typically, the average success rate for iterative experiments can range from 30% to 70% and often depends on the domain the team is working in.

Investments – These are either tech investments or user experience investments. Tech investments should increase the capacity of the team to drive more impact in the future. Examples include building a new tool to automatically localize copy experiments or refactoring a major piece of code to run experiments more easily in that area in the future. User experience investments are investments to make the product experience more delightful without the expectation of driving growth (or possibly even slightly hurting growth). The goal of user experience investments is to strike a balance between trying to drive metrics and protecting the brand reputation. Examples could include improving an email unsubscribe flow to make unsubscribing easier and clearer.

Big Bets – These are what they sound like – big bets to open up new opportunities. They take a lot of effort and have a lower probability of success, 20%-40%, but can be very impactful if they work. Examples are building out new growth channels or a complete overhaul of your new user onboarding flow.

Increasing Your Portfolio Returns

One of the most important ways to increase the return on your portfolio is picking the right investments to make (i.e. project selection). You have limited capital (i.e. engineering time) so it is critical to make wise investment decisions. When it comes to improving project selection there are two big levers:

Lever 1: Increase the rigor of your project selection process

In his seminal book on management, High Output Management, Andy Grove, former CEO of Intel states that a common rule we should heed is to detect and fix any problem in a production process at the earliest stage possible. In Growth, every experiment you work on has an opportunity cost. Every failed or low impact experiment you run was time that could have been spent working on a different experiment that maybe could have had a better shot at being successful. On the very first Growth team I was a part of, the way we selected projects was very democratic. We brainstormed a bunch of ideas, wrote them up on a whiteboard, and then people voted on which ones we thought we should do. What we later realized was that this is one of the worst ways to select projects because people vote without any data and just go on gut feeling. In hindsight, we realized that people were voting to work on flows that weren’t perfect and that they felt confident could be improved. However, the reason why those flows weren’t perfect was because they were often not that important or impactful. In Growth it is usually more impactful to make a small incremental improvement at the top of the funnel of a core flow than to fix a broken less used flow.

To improve how we selected projects, we introduced a much more rigorous process and started to calculate the expected value of the project per week of engineering work and picked the ones with the highest estimated ROI.

Expected Value = probability of success * # new active users / amount of engineering work

You are making an estimate for each part of this equation, but it is still helpful in comparing different projects. For estimating the # of new active users, it is helpful to not just pluck a number out of the air. You should start with a hypothesis on what metric you will move and by how much and then model out how that impacts your number of active users. For instance, if you think a new email subject line will increase the open rate by 5%, a rough way to model the number of incremental new users = # of clicks from the email * percent of clicks coming from dormant users * 0.05.

Lever 2: Increase the number of high quality project ideas

Once you have a good process to select projects, the second lever is to increase the pool of good projects you have to select from. To do this you need to make sure it is not just the Product Manager’s job to come up with ideas for the roadmap, but instead build a bottom-up culture where it is the job of every single person on the team to come up with ideas. That means every engineer, every designer, every analyst, should be required to contribute experiment ideas. When it is everyone’s job to contribute to experiment ideas, the number and pipeline of experiment ideas will dramatically increase. With proper training and feedback you can ensure that those ideas are high quality.

Lever 3: Maximize impact of every experiment you ship

The final lever is to make sure you are really maximizing the impact of every experiment you ship. The way to do this is related to lever 2 of building a bottoms-up culture. Every engineer should have ownership over experiments they are assigned to work on, even if it wasn’t originally their idea. What ownership means is that they should be striving to figure out how to make the experiment as impactful as possible by improving on the initial idea or by coming up with additional variants to test. Doing this will help make sure you are squeezing as much juice as possible out of every experiment. If a team of 10 engineers works on 120 experiments/quarter there is no way a single person can think deeply about each and every one of the 120 experiment ideas. Taking a bottoms-up model where the people spending the most time working on an experiment idea are empowered to come up with suggestion and figure out how to maximize that idea will help make sure you’re not leaving any money on the table.

Evolving Your Portfolio Over Time

While it is important to try and maximize your portfolio’s returns, it is also important to understand how the portfolio makeup needs to adjust over time. Growth teams typically go through three stages and how they allocate between different project classes should evolve with what stage they’re in.

Startup Stage:

For a Growth team at an early stage startup or a Growth team working in a brand new area at a mature company, there are a lot of unknowns. Not much has been tried, there is a lot of low hanging fruit, and a lot of foundational stuff might be missing. For instance, you might not have the dashboards you need or you might not even have an experimentation framework in place at an early stage startup. However, the team can’t just focus on investments the entire quarter. The team also needs prove to leadership that they can drive results and show they can move business metrics up and to the right.

At this stage you might want the following allocation:

Iterative Experiments (33%) – You will want to get some early quick wins to prove to leadership that the team can drive results. This will build confidence in the team and give you the space to work on projects that will pay off in the long term.

Investments (33%) – There are typically several foundational investments that need to be made early on to set up the team for long term success. It could be setting up logging pipelines, reports, etc.

Big Bets (33%) – Because there are so many untapped areas, the team needs to take a stab at a couple big rocks to start exploring the problem space, see what pays off, and start to form a longer-term strategy.

Growth Stage:

Typically, after 2-3 quarters a team has gotten some traction in the area and has started to figure out what works. This is where they enter the growth phase, where they typically really focus in on iterative experiments and driving impact. The growth stage lasts many quarters or years and this stage embodies what a stereotypical Growth team looks like.

Iterative Experiments (70%) – This is where the bulk of the team’s impact will come from.

Investments (15%) – Continue making investments to increase the team’s ability to drive long term impact.

Big Bets (15%) – Continue making some big bets to open up new opportunities for the future.

Mature Stage:

Finally, after a team has worked on an area for a long time and has spent many quarters, or many years picking off the low hanging fruit you can start to reach a local maxima where the team has really milked a strategy or an area for all its worth. You’re starting to reach a point where you are faced with making a call on if you should pause working on this area entirely or if it is time to try and break out of that local maxima.

Iterative Experiments (30%) – You should significantly ramp down how much time you’re spending on iterative experiments, but you probably still have some backlog of experiment ideas you still want to work through.

Investments (10%) – Pause most work on investments unless they are tied to the big bets.

Big Bets (60%) – Big bets ramp up significantly to try and find new opportunities and break out of the local maxima. When in a local maximum, it is important to note that you probably have spent quarters or even years optimizing what is now the current control experience. Your first stabs at a radically different approach likely won’t beat control off the bat and you need to be prepared to spend time doing iterative optimization on the big bet to see if it has the potential to eventually surpass control. If one of these bets open up a whole new area, the team might transition back to the Growth Stage.

Conclusion

Take all the percentage allocations in this post as just a rough guideline. However, every leader in Growth should be actively thinking about and managing their portfolio and their team processes that produce that portfolio. Make sure you are maximizing your return through really assessing and improving your project selection process. Build a culture that not only empowers, but requires, everyone to think about how to help their team hit its goals. Finally, understand what stage you are in and look ahead, out in front of the team, to anticipate how your portfolio allocation needs to change to ensure the team’s long-term success.