Surveys are an excellent way to better understand your audience and the value they get from your product. With that knowledge, you can explore ways to improve retention by better solving your customer's problem.
This is so interesting. Until now I was very survey shy as I felt that they can often not really make a difference, your Substack has started a rethink, thank you
I think it's rare that a survey will totally surprise you. You probably already know why folks value what you do. But it can help understand some of the nuances or the degree to which certain things are true.
When done well, other parts of the survey *can* produce some surprises. Like the "how to improve" and "who else like me do you enjoy" questions — those can spark some new ideas.
Thanks again for dropping a comment. If you do end up running a survey, would love to hear how it goes!
This is great Reid. Thanks. Does Substack have some 'boiler plate' suggestions for survey questions and formats we could simply pick up and use? A bit like the pre-cooked invite emails for Chat and Notes. Also, does Substack do cancel surveys? Is it planning to give us the option? With a pre-cooked suggestion? Cheers. Bernard
For cancel surveys, each paid subscriber that chooses to cancel can provide a cancel reason and open-ended feedback. You can see the results to your survey here: https://your.substack.com/publish/stats/unsubscribes
For the audience survey, it's certainly something we've thought about and hope to launch soon. In the meantime, feel free to make a copy of this template:
1) How would you feel if you no longer received Growth Croissant? (Multiple choice: "Not disappointed" or "Somewhat disappointed" or "Very disappointed")
2) What is the main benefit you receive from Growth Croissant?
3) If you are a free subscriber, what would encourage you to pay for Growth Croissant? If you are a paying subscriber, what encouraged you to pay for Growth Croissant?
4) How could we improve Growth Croissant for you?
5) How did you first find out about Growth Croissant?
6) What other writers, podcasters, creators, or media brands do you enjoy that are similar to Growth Croissant?
We would usually ask for demographic information, which was usually secondary to the questions above.
May 14, 2023·edited May 14, 2023Liked by Reid DeRamus
Thank you Reid for this great article! I have a quick question if you don't mind:
I have recently co-founded a new startup and have been sending out a 1-2 minute anonymous survey to a waitlist. As we are not actively promoting the tool yet, the majority of those who have been signing up for the waitlist come from organic traffic to our landing page – around 75% through google search. I think the survey is good (I have lots of research experience) as well as the emails that get send with it (I send follow up emails as reminders). I think they follow the advice in this blog post quite well. The survey also offers the possibility to win a $100 Amazon voucher for those who fill it out.
However, only around 5% have been filling it out. My interpretation of this outcome is that it reflects the quality of the people signing up to the waitlist more than anything – meaning the majority of these leads are weak and possibly not our target users (considering the organic nature of most), or fake email addresses.
Question: Do you think my interpretation is correct or could there be another likely explanation/assumption I could investigate? And is there anything more do you think I could try to increase the percentage of people filling out the survey?
Honestly, 5% seems pretty great! It’s worth keeping in mind these folks are on a waitlist and haven’t actually used your product yet. Without any kind of relationship yet, I would have modest expectations on what share would complete a survey. For reference, the 5% - 20% email sent to survey completed conversion in the post above is for engaged subscribers that have been using the product (e.g., reading the newsletter) for more than 30 days.
I wouldn’t read too deeply into the quality of the leads, and it looks like you’re doing all the right stuff to incentivize folks to take the survey. You could try saying that survey respondents will climb the waitlist and get access quicker. If possible, you could try having the survey prompt right after they signup for the waitlist (vs a follow up email). But ultimately, 5% seems pretty good, and you’re time may be better mostly spent on getting more people to signup for the waitlist.
Thank you for your thoughtful insights, Reid! Your advice is greatly appreciated. I'll definitely consider your suggestions and keep you updated on my progress. Thanks again for your help!
May 13, 2023·edited May 13, 2023Liked by Reid DeRamus
I'm curious at how we can turn our audience into a more hungry audience. If a cancel reason says the main reason for leaving is time, how would you approach a solution to that churn problem? I noticed Azeem is doing videos. But for me, as a reader scanning articles, especially bullet point lists is much more time efficient than listening to a podcast necessarily.
But I just don't know, it would be useful to have standardized bassline of typical survey questions per category that were aggregated anonymously within the substack network. As a writer, this would provide me with a lot of insight. As a new newsletter, I'm not sure I would trust the sample size of my audience to conduct a survey quite yet.
As a high frequency publication, I'm expecting to have issues with open rates and churn, and I consider this strategy as an outlier that's possibly killing my conversion rate to the point of potential failure. However, on the upside, if my growth of free readers continues to accelerate, I'm creating more opportunities for conversion overall and literally narrowing down my paid audience.
I'm not sure I'm clearly understanding the pros and cons of my approach that boils down to intrinsic obsession with my topic and a category that's relatively curation friendly. As my competitors multiply in the AI space on substack I'm very curious to see where I end up.
To me, “time” is kind of like “price”, in that they’re both kind of cop out reasons. A lot of the time, these folks may not be part of your “core” audience and, for the purpose of *retention*, not worth considering. (They can still be a part of your overall growth trajectory and business, but maybe as free subscribers, podcast listeners, etc.)
Some “time” responses may be your core audience, but what they’re really saying is that they’re choosing to spend their time & attention elsewhere. The tricky part: the solution may not be to better engage & retain them by saving them *time* with curation or summaries; they may actually want deep-dive exploration of complex topics they can’t find elsewhere, which kind of runs counter to their “cancel reason”.
I think you’re totally right regarding small sample size earlier on. I still think reading through feedback is massively helpful, but definitely wouldn’t suggest following it unilaterally, or viewing it as statistically significant insights.
Thanks as always for dropping by with some excellent questions & thoughts, Michael!
Also probably helpful to keep in mind you don’t need to do any of this stuff, especially earlier on. What’s most important is just to keep publishing and trying to make what you’re bringing into the world gradually better. On the latter, your intuition is usually the best tool.
Well my intuition tells me to keep going like a bat out of hell against all the odds underdog style. That's my hero's journey so far. I am playing around with the idea of writing longer reads though that I feel better represents my ability of curating but also adding my takeaways with a bit more meat in the soup.
Hi Reid, I know it's been months, but I just got around to figuring out my "next step" and I think it involves finding someone to help me promote my substack. Since you are an expert at all of this stuff, I figured you might know someone or a company I can look into? I'm looking for general publicity help, but also someone who might have good advice for which posts and podcasts to paywall, and at what frequency, to get more premium subscriptions...Thanks in advance if you have any help, and no worries if you don't! :)
thanks for your insights, Reid 🙏🏼
Thank *you* for sharing, Darren! 🙏
🤙🏼
This is so interesting. Until now I was very survey shy as I felt that they can often not really make a difference, your Substack has started a rethink, thank you
Thanks for the note, Jon!
I think it's rare that a survey will totally surprise you. You probably already know why folks value what you do. But it can help understand some of the nuances or the degree to which certain things are true.
When done well, other parts of the survey *can* produce some surprises. Like the "how to improve" and "who else like me do you enjoy" questions — those can spark some new ideas.
Thanks again for dropping a comment. If you do end up running a survey, would love to hear how it goes!
This is great Reid. Thanks. Does Substack have some 'boiler plate' suggestions for survey questions and formats we could simply pick up and use? A bit like the pre-cooked invite emails for Chat and Notes. Also, does Substack do cancel surveys? Is it planning to give us the option? With a pre-cooked suggestion? Cheers. Bernard
Thanks for dropping by with the questions!
For cancel surveys, each paid subscriber that chooses to cancel can provide a cancel reason and open-ended feedback. You can see the results to your survey here: https://your.substack.com/publish/stats/unsubscribes
For the audience survey, it's certainly something we've thought about and hope to launch soon. In the meantime, feel free to make a copy of this template:
https://xvksy7tzgqs.typeform.com/to/yXMuucXX
Or, here's a list of questions we typically used:
1) How would you feel if you no longer received Growth Croissant? (Multiple choice: "Not disappointed" or "Somewhat disappointed" or "Very disappointed")
2) What is the main benefit you receive from Growth Croissant?
3) If you are a free subscriber, what would encourage you to pay for Growth Croissant? If you are a paying subscriber, what encouraged you to pay for Growth Croissant?
4) How could we improve Growth Croissant for you?
5) How did you first find out about Growth Croissant?
6) What other writers, podcasters, creators, or media brands do you enjoy that are similar to Growth Croissant?
We would usually ask for demographic information, which was usually secondary to the questions above.
Let me know what you think or any questions!
Thank you Reid for this great article! I have a quick question if you don't mind:
I have recently co-founded a new startup and have been sending out a 1-2 minute anonymous survey to a waitlist. As we are not actively promoting the tool yet, the majority of those who have been signing up for the waitlist come from organic traffic to our landing page – around 75% through google search. I think the survey is good (I have lots of research experience) as well as the emails that get send with it (I send follow up emails as reminders). I think they follow the advice in this blog post quite well. The survey also offers the possibility to win a $100 Amazon voucher for those who fill it out.
However, only around 5% have been filling it out. My interpretation of this outcome is that it reflects the quality of the people signing up to the waitlist more than anything – meaning the majority of these leads are weak and possibly not our target users (considering the organic nature of most), or fake email addresses.
Question: Do you think my interpretation is correct or could there be another likely explanation/assumption I could investigate? And is there anything more do you think I could try to increase the percentage of people filling out the survey?
Thanks again!
Thanks for the question, Chris!
Honestly, 5% seems pretty great! It’s worth keeping in mind these folks are on a waitlist and haven’t actually used your product yet. Without any kind of relationship yet, I would have modest expectations on what share would complete a survey. For reference, the 5% - 20% email sent to survey completed conversion in the post above is for engaged subscribers that have been using the product (e.g., reading the newsletter) for more than 30 days.
I wouldn’t read too deeply into the quality of the leads, and it looks like you’re doing all the right stuff to incentivize folks to take the survey. You could try saying that survey respondents will climb the waitlist and get access quicker. If possible, you could try having the survey prompt right after they signup for the waitlist (vs a follow up email). But ultimately, 5% seems pretty good, and you’re time may be better mostly spent on getting more people to signup for the waitlist.
Keep us updated on how things go!
Thanks again for dropping by with a question,
Reid
Thank you for your thoughtful insights, Reid! Your advice is greatly appreciated. I'll definitely consider your suggestions and keep you updated on my progress. Thanks again for your help!
I'm curious at how we can turn our audience into a more hungry audience. If a cancel reason says the main reason for leaving is time, how would you approach a solution to that churn problem? I noticed Azeem is doing videos. But for me, as a reader scanning articles, especially bullet point lists is much more time efficient than listening to a podcast necessarily.
But I just don't know, it would be useful to have standardized bassline of typical survey questions per category that were aggregated anonymously within the substack network. As a writer, this would provide me with a lot of insight. As a new newsletter, I'm not sure I would trust the sample size of my audience to conduct a survey quite yet.
As a high frequency publication, I'm expecting to have issues with open rates and churn, and I consider this strategy as an outlier that's possibly killing my conversion rate to the point of potential failure. However, on the upside, if my growth of free readers continues to accelerate, I'm creating more opportunities for conversion overall and literally narrowing down my paid audience.
I'm not sure I'm clearly understanding the pros and cons of my approach that boils down to intrinsic obsession with my topic and a category that's relatively curation friendly. As my competitors multiply in the AI space on substack I'm very curious to see where I end up.
To me, “time” is kind of like “price”, in that they’re both kind of cop out reasons. A lot of the time, these folks may not be part of your “core” audience and, for the purpose of *retention*, not worth considering. (They can still be a part of your overall growth trajectory and business, but maybe as free subscribers, podcast listeners, etc.)
Some “time” responses may be your core audience, but what they’re really saying is that they’re choosing to spend their time & attention elsewhere. The tricky part: the solution may not be to better engage & retain them by saving them *time* with curation or summaries; they may actually want deep-dive exploration of complex topics they can’t find elsewhere, which kind of runs counter to their “cancel reason”.
I think you’re totally right regarding small sample size earlier on. I still think reading through feedback is massively helpful, but definitely wouldn’t suggest following it unilaterally, or viewing it as statistically significant insights.
Thanks as always for dropping by with some excellent questions & thoughts, Michael!
Thank you Reid. Your feedback is always appreciated. I will reread this article when I have a clearer head.
Also probably helpful to keep in mind you don’t need to do any of this stuff, especially earlier on. What’s most important is just to keep publishing and trying to make what you’re bringing into the world gradually better. On the latter, your intuition is usually the best tool.
Well my intuition tells me to keep going like a bat out of hell against all the odds underdog style. That's my hero's journey so far. I am playing around with the idea of writing longer reads though that I feel better represents my ability of curating but also adding my takeaways with a bit more meat in the soup.
Another great one; I feel far less timid about this tactic now. Thank you!
Excellent! Send me a note if you’re thinking of doing a survey & would like a sounding board / extra pair of eyes.
Thanks Mike!
Hi Reid, I know it's been months, but I just got around to figuring out my "next step" and I think it involves finding someone to help me promote my substack. Since you are an expert at all of this stuff, I figured you might know someone or a company I can look into? I'm looking for general publicity help, but also someone who might have good advice for which posts and podcasts to paywall, and at what frequency, to get more premium subscriptions...Thanks in advance if you have any help, and no worries if you don't! :)