John Egan

There are tons of posts on how to set a Growth team up for success from a hiring perspective, process perspective, tooling perspective, etc. However, one of the most important things to get right is setting up what each team is goaled on. This is especially true as a Growth Org starts to scale with multiple teams working in different areas, developing their own strategies, and each team figuring out how to drive their metrics up and to the right. It becomes critical that the goals and metrics are architected in a way to ensure each team is building towards a long term sustainable business.

Casey Winters wrote a great post on 5 mistakes to avoid when setting goals for a Growth team and why you should set goals on absolute numbers instead of percentages. In this post, I want to build on that and share 5 guiding principles I’ve found to be effective when architecting goals for a team, ensuring the team is aligned with the rest of the company and setting them up for long term success.

 

Principle #1: Isolation

The first key ingredient for a team level goal is making sure that it is a metric that the team can directly influence and has some degree of isolation from other teams, seasonality, etc. Team level goals should be a yardstick against which the team can measure progress. However, if that metric can be heavily influenced by other teams, by seasonality, or other factors outside of the team’s control, it starts to lose value as a way for the team to measure its impact and progress.

At Pinterest, the way each team in Growth measures its progress is by calculating the absolute number of incremental Weekly Active Users added by every A/B experiment run and summing up that impact over the quarter. By using A/B experiments to measure each team’s impact, the A/B experimentation framework provides isolation between teams, even when teams work in overlapping areas. The one downside of summing up experiment impact is that it has a tendency to overestimate the impact. Another approach is having team level holdouts for every experiment they run in the quarter, which can result in a more accurate measure of a team’s impact, but at the cost of significant additional engineering maintenance and overhead.

 

Principle #2: Avoid Misalignments

The second key ingredient of a team goal is making sure it does not put you at odds with other teams in the company. I recall on one occasion speaking to a Growth PM and they were complaining that their team goal was to grow the total number of registered users with the app installed. On the surface, this might seem like a fine metric. The problem was that this metric put them directly at odds with the Notifications team. This is because any time the Notifications team would send a push notification, some small portion of users will delete the app. Misalignments like this suggest that optimizing that metric might not correlate 100% to business success.

One possible way to overcome this is by having multiple teams in Growth all work to drive the same metric while defining swim lanes and areas of ownership to ensure each team has a clear charter. For instance, at Pinterest, most teams in Growth goal on Weekly Active Users, and each team has clear areas they own such as Emails & Notifications, or Signups & Logins.

 

 

Principle #3: Tied to Long Term Success

            The third key ingredient of a team metric is tying it to long-term business success. Growth teams are hyper metric focused and even with the best intentions, they can sometimes over optimize for a metric. To counteract this, you want to ensure that metrics cannot easily be “gamed” and that they are tied to the long-term success of the company. You can do this by going deeper than the surface level metrics and setting goals based on down funnel engagement.

A simple example is signups. It is natural for a team focused on user acquisition to perhaps goal on signups as the metric they drive. However, signups are a metric that can relatively easily be juiced in ways that might not lead to the best long-term outcomes. For instance, you can remove steps from the signup flow to get more users through the signup funnel, but those steps may be important in significantly improving the user experience once the users are in the product. Or perhaps, the acquisition team starts driving a ton of signups from low-quality traffic sources and signing up a ton of users that end up not sticking around. A better metric for a team focused on acquisition might be a metric like activated signups, where they only count signups that result in a long-term retained user. In this way, the team is ensuring the metric they are trying to drive is always aligned with long term business success.

 

Principle #4: Guardrails

Even with metrics that are tied to long term success, it can sometimes be necessary to setup guardrails to protect the user experience. A great example is emails and notifications; sending more emails always lifts engagement. Even if those additional emails result in more unsubscribes, those unsubscribes never overcome the lift in active users from just sending more. Intuitively though, everyone knows that there can be a long-term cost to sending too many emails which can cause user fatigue or brand perception. The problem is the long-term cost doesn’t appear in days or weeks, but over years.  To help protect the user experience, the team needs to establish guardrail metrics around unsubscribe rates and spam complaint rates to help protect the user experience while they drive towards their goal. The guardrail metrics would ensure that optimizing their northstar metric of active users isn’t at the expense of user experience.

Guardrails can also be helpful as a cross-check for the main metric the team focuses on to help catch potential issues. For instance, if the Emails & Notifications team decides to measure its impact through A/B experiments, it is important to setup a cross-check metric for the team to track the health of the channel outside of experiments. If a particular ISP starts marking the email as spam, it could significantly impact the business and the team needs to be able to catch that. The team would need to setup a guardrail metric like daily email clicks, or even better, daily engaged sessions from email, to ensure the team can catch and identify issues that fall outside of experiments.

 

Principle #5: KISS

            Finally, there is a well-known principle in Computer Science called KISS, which is an acronym that stands for “Keep it simple, stupid.” This principle was originally created by the Navy and states that most systems work best if they are kept simple rather than made complicated. With metrics, it can be easy to get a bit carried away and create extremely complex metrics or have several different metrics that a team is responsible for driving. When designing team metrics or a team goal, try not to overcomplicate things and just keep it simple, stupid.