Aligning growth strategy with core product value: surveying our Biggest Fans

Context

I was a growth product manager at an EdTech startup, the second product hire as the team grew from one product manager to four. The product was an education platform for healthcare students, providing in-depth illustrated videos and images to explain medical concepts, study tools, and study schedules for major exams.

Why

The problem

There were two motivations behind this research project. The first was to directly inform the work of the new growth team I was leading. Our charter was to optimize top of funnel flows, including free trial signups, new user activation, and purchase conversions. The product was already getting a lot of traction with students, and stakeholders had many different theories about why. These mixed theories were reflected in the messaging of our landing pages, positioning statements, and early product flows, and the lack of coherence made it hard to design A/B tests that built on each other. I wanted to better understand who our core users were and the value they derived from the product, and then develop a holistic growth strategy around that information.

The second motivation was to drive alignment as we grew from one product manager to four and added a new VP between the product team and the founders. We needed a shared language about who our product served and its core value as we grappled with tough decisions about the overall product direction.

Research goals

I outlined these research goals for the project:

• How do the users in our core market describe themselves?
• Why do our core users love the product?
• What do our core users want us to improve about the product?
• How do the users in our next adjacent market describe themselves?
• What holds our next adjacent market back from loving the product?

Anticipated outcomes

I anticipated creating these artifacts as a result of the research:

• Create a description of users in our core market
• Identify and list core benefits for our core market
• Identify and list opportunities to double down on our product’s core strengths
• Create a description of users in our next adjacent market
• Identify and list opportunities to improve product market fit with our core market
• Identify and list opportunities to improve product market fit with our next adjacent market

I expected those artifacts to influence additional work:

• Develop growth team strategy for top of funnel growth aligned with core market and core product value
• Kick off conversations with product team and leadership to discuss overall product direction

Approach

Method

I thought that a product-market fit framework popularized by Rahul Vohra could answer our questions. The survey and analysis segment users into three categories (segment nicknames are mine):

• Biggest Fans, who would be “very disappointed” if they could no longer use the product and represent the core market
• Lukewarm Users, who would be “somewhat disappointed” if they could no longer use the product and represent the next adjacent market, and
• Misfits, who would be “not disappointed” if they could no longer use the product and represent a market that doesn’t have a strong fit with the product.

For my team’s initial growth strategy, I would be focusing on the responses of the Biggest Fans: how they described the ideal user of our product (typically describing themselves) and the biggest benefit they got from the product. We’d want to align our landing pages, sign up flow, onboarding experience, and purchase flows with their language.

For aligning the growing product team around our overall product direction, I would share about both Biggest Fans and our Lukewarm Users, opening up conversations about ways to double down on what’s already working well in the product and the areas that these two segments most want to see improvements.

Recruitment

I set up in-app messaging to recruit active users, knowing that would give us quick results even if it meant our respondents skewed toward users that were engaged; because I wanted to learn about engaged users (Biggest Fans) and somewhat engaged users (Lukewarm Users), this selection bias wouldn’t be a problem.

In total, we received nearly 400 survey responses, with a roughly even mix of users with free, paid B2C, and paid B2B accounts.

Survey design

A sample of survey questions were:

• How would you feel if you could no longer use Osmosis?
  • Select from: Not disappointed, Somewhat disappointed, Very disappointed
• What is the main benefit you receive from Osmosis? [Short answer]
• What type of people do you think would most benefit from Osmosis? [Short answer]
• How can we improve Osmosis for you? [Short answer]

I also included demographic questions such as:

• What degree are you pursuing?
  • Select from: MD, DO, RN, NP, PA, Dentistry, Pharmacy, Health Professional, Other
• What stage of your education are you in? [Select]
  • Select from: Pre-clinical, Clinical, Residency, Fellowship, Continuing Education, Other

For the full survey design, see Appendix A.

Findings & impact

Growth strategy

Insight #1: How do the users in our core market describe themselves? Our Biggest fans describe themselves as students and learner type, not by profession or stage of their education

Internally, the company’s large content team focused on profession and educational stage to craft curriculum, and that language had seeped into our landing pages and product flows. However, the survey revealed that our Biggest Fans did not often name profession or educational stage. They identified as medical/healthcare students more broadly, often describing a specific learning need, learning style, or learning goal. Instead of seeing themselves as “a second-year RN student,” they used language like “a visual learner with a short attention span” or “a student who doesn’t have enough time to read from books.”

Impact: We targeted landing pages and onboarding flows centering profession and educational stage for experimentation.

An example landing page that centers profession.

We conducted an audit of landing pages and onboarding flows that focused on profession and educational stage first and foremost. This insight helped us target areas that weren’t resonating with users, and the next insights gave us some valuable ideas about what to try instead.

Insight #2: Why do our Biggest Fans love the product? They value videos, topics covered, improved understanding, and saving time.

Our Biggest Fans valued videos, topics covered, improved understanding, and saving time. Seeing videos and topics covered wasn’t a surprise, since our deep collection of custom-illustrated videos was how the product was originally born as a YouTube channel before becoming a more full-featured study platform. What was valuable here was why learners loved the videos so much: it made it easier for them to understand complex topics, and it saved them time in their jam-packed medical school schedules.

Impact: we designed experiments to test value-based messaging

The previous messaging compared the product to traditional lectures by focusing on clinical skills.

We tested messaging focused on comprehension and time savings.

We also tested simple messaging focused on visual learning.

While our previous messaging focused on profession, educational stage, or clinical skills instead of traditional classroom learning, we started testing messaging that highlighted the core market (i.e. visual learners) and the product value (i.e. improved understanding and saving time). We started testing on less risky, lower traffic landing pages, and some initial successes led us to adopt similar messaging on the highest-traffic landing pages and home page as well as in onboarding flows and purchase pages.

Impact: we designed our growth strategy around connecting learners to relevant video content quickly (instead of pushing for fast sign ups and purchases).

When we started the growth team, our initial focus had been to reduce usability friction on free trial signup flows and purchase flows. These findings completely reoriented our strategy around connecting learners to relevant video content quickly, and trusting that the videos would get them to an “Aha!” moment. Even if this strategy delayed their path to the signup and purchase flows, we had more confidence that prioritizing this value would still drive signups and purchases in the long run.

Our most successful A/B test ever was to add a content search component to one of our biggest landing pages. While direct click through rates to the sign up flow decreased, as expected, engagement with video content sky-rocketed, and sign up conversions after viewing a video increased by 20%! Purchase conversions downstream of these actions also showed positive signals. We later invested in onboarding flows that highlighted content and guided learners to relevant content and saw great engagement and growth.

Product direction

Impact: we considered new opportunities in content-product feedback loops.

In order to apply our insight about the core value (videos, topics covered, better understanding, and time saved) beyond our top of funnel growth work, we broadened our thinking to longer-term product investments. How might we double down on producing and showcasing high-quality video content? We did some cross-functional brainstorms and came up with new ideas, such as:

• Invest resources in improving internal tools for video production to make our production process more efficient.
• Create more visibility about upcoming content releases. Because the video production process was rigorous and time-consuming, we knew what videos were coming down the pipeline long before learners got access to them - but we could drum up more excitement in anticipation of their release!
• Create better feedback loops between user-requested topics and the content roadmap.

Insight #3: What do our Biggest Fans want us to improve about the product? Both Biggest Fans and Lukewarm Users want a lower price, more video content, more questions and flashcards, and fixes for bugs and UX issues.

There was more overlap between the Biggest Fans’ and Lukewarm Users’ requests than we expected. They both asked for a lower price, more video content, more questions and flashcards, and fixes for bugs and UX issues.

Over the course of the next year, many of the opportunities uncovered in this study were addressed.

Impact: We invested in redesigning pricing and packaging.

As part of a cross-functional pricing committee to redesign pricing and packaging, I ran a Van Westendoorp pricing study. The committee’s work was still in progress when I left the company, but they were close to rolling out new options, including a more limited package at a lower price point.

Impact: We paid down technical and design debt.

We also worked with the engineering and design leadership to audit areas of debt, scope projects to pay down debt in key areas, and prioritize those projects against other product work. In addition to some focused projects, we set up systems to address these issues in a more ongoing way.

Impact: We made it easier to find video content in our huge library.

Finally, we did a big redesign of the information architecture of the platform’s content organization. This was no easy process and involved many expert stakeholders contributing to design sessions! Ultimately, it drove strong activation and engagement with content and helped users quickly find content they were seeking.

Insight #4: What holds our next adjacent market back from loving the product? The main request from Lukewarm Users that differed from Biggest Fans was for clinical content.

Compared to our Biggest Fans, our Lukewarm Users requested clinical content more often, suggesting that this segment had more later-stage medical students. Our core market was likely pre-clinical students; after all, the founders had dropped out of medical school after their preclinical years, intent on revamping how pre-clinical material was taught. It made sense that our next adjacent market would be clinical students, whose training had changed from classes and learning huge amounts of new information to on-their-feet clinical rotations that emphasized clinical skills over book smarts.

Impact: discussions about market expansion started including clinical learning among many other options.

While the company started by focusing on pre-clinical MD students in the US, it had rapidly found eager learners in other markets: different healthcare professions and different geographies. The company had also grown from only B2C (students purchasing individual subscriptions) to B2C and B2B wings, making deals directly with institutions to give cohort-wide subscriptions. The company had focused a lot on these expansions, but the lack of clinical content was holding back retention after students finished their pre-clinical studies.

The insights from this study added another voice in the company pushing to keep clinical content in mind. Clinical skills didn’t become the focus of expansion, but we did get buy-in to pilot a clinical skills series with live-action demonstrations instead of the classic illustrated style. It was a hit with third- and fourth-year students!

Summary

This study gave us a description of our core market (visual learners, learners that need help understanding, learners with limited time and attention) and the core value they found in the product (videos, topics covered, better understanding, and saving time). We were able to update our product positioning and use the findings to shift our top of funnel growth strategy to prioritize connecting new users with the core value quickly - not just sending them as fast as possible to the sign up and purchase flows.

We also learned what our core market wanted us to improve about the product (lower price, more video content, more questions and flashcards, and fixes for bugs and UX issues). While the impact of those findings was less immediate, over the next year we were able to build support to address most of them with strategic product investments.

Finally, we better understood our next adjacent market as quite similar to the core market, but with one key difference: a lack of clinical content was holding them back from loving the product. This finding helped raise the visibility of the need in the ongoing conversations at the leadership level about where to focus resources for expansion. Even though it didn’t result in a big, company-level pivot, it still garnered support for piloting more clinical skills content that showed promise.

This study also had a big impact on the research function at the organization. I conducted it at a time where a small product and design crew was trying to show stakeholders the value of UX research and get buy-in to do more. The insights report I published in our fledgling research insights repository, and the presentations and facilitated discussions that followed, received record engagement and gave UX research much bigger visibility in the company. It was a key turning point in moving research from “just” a small footnote within the product and design departments to establishing research as valuable for cross-functional teams to learn together and coordinate their work according to those learnings.

---

Appendix A: Survey design