24 - Product Analytics

Episode 24 August 11, 2023 00:20:00
24 - Product Analytics
XO Capital's Fund Stuff
24 - Product Analytics

Aug 11 2023 | 00:20:00

/

Show Notes

Topic: Rethinking Product Analytics for SAAS and Understanding Customer Needs

View Full Transcript

Episode Transcript

 Hello? Hello. It's Andrew from XO Capitol. And this week I wanted to give a quick update on our flip fund. We have 220,000 hard committed. It's been less than a week. And we're pretty stoked. So if you want in, take a look at our flip fund deck and all the details are in there, including a link to directly invest, or let me know if you have any questions. A quick update on the MNA market on the very low end where we play, I'm starting to see multiples kind of come down, buyers being more realistic about asking prices, slight uptick in distress ventures. So ventures that have previously raised money. Lower quantity of quality deals. I just not seeing quite yet that the level of pain that I expect to come later this year, maybe even into next year, where a lot of these companies that. Ray's a modest seed round 500,000 a million under 2 million, for sure. That's kind of the realm that we plan. I think a lot of those guys are going to run out of money and realize that they were built on kind of an idea and traction in 2023. It's nothing like getting traction in 2020 or 2021. And they will be on micro choir and I just absolutely love the idea of a team. That has raised half a million to a million, put that into a bad-ass product. And I can pick that up for one or two X, a R R, and scale it and scale it on our terms, which generally means. It's roughly 3% month over month growth target, which is modest, but over two years, that's doubling the business and that's great for us. I'm also seeing a slight decrease in BS, AI startups. There was a while there where it was like, you know, tens of of, of these little AI company. Does that we're very, very thin wrappers around chat GPT. No real product. Insight. Many of them are probably illegal. And anyways, I'm seeing much less of those, which is great. Today. I wanted to talk mostly about product analytics. I think to a large extent SAS product analytics are broken. I know PLG or product led growth as a thing is relatively new. And the closest tool that answers all these questions, I think is June dot. So. It's a pretty cool tool, but I still think it kind of misses the mark. And I just logged in today just to double check. I had all of my facts straight. It's definitely gotten better over the past, maybe six months or so, but it's still very difficult to get real deep insight into what the heck is going on in your business. I always like to start these experiments with what would like a level 11 version. So not 10, but even just turning it past, bringing on a magical experience for an end customer. And I love doing this. It's, it's kind of a fun exercise. So for a product analytics tool, I think what that looks like is. The zero configuration. So today state-of-the-art, you have to send events to these platforms. You have to send the right events and you have to sprinkle these, these little. Event center handlers. Into your code and it will put those events to it. We'll send those events to the product, and then you can use those as the basis for your analytics. Totally understand why it needs to be there. But in this level 11, I think it would be zero configuration and it would just kind of figured out. Can I push the right info to myself at the right time. So this isn't. This isn't me going into a dashboard and sorting through a bunch of tables and forms. And. Graphs that are supposed to show me insights, but actually I have to go find the insights. I have to fish for them through the data. And. Under the constraint of the UI, that's already been built by this product. And that takes a lot of time. So for instance, I use mixed panel and mixed panel is just pretty much like useless. I mean, we could even log in and I could show you for a product called 2%. That it just doesn't quite hit the markets. It's very difficult to understand. Okay, great. I have a create campaign. Event, I have an ad contact event. And here's some charts and stuff and a create center event. Like, what does this, what does this mean? Like, are people happy? Like, what is my retention rate? Why did somebody turn? Why didn't somebody sign up? All of these things are very difficult to do in an app. By looking at aggregate charts like this. I think their reports suck the flows and funnels like funnels. For instance, I don't want to have to care about a order of a particular funnel. So in SAS, which I'll get into in a second there's many paths to a goal and they don't always look similar relative to like an e-commerce site, which I'll get into again in a second. But. All of these funnels are based on sequential steps that a user needs to take to reach some kind of end goal. And in my experience, that's just not how the real world works. I want to be able to describe a bag of step, several steps, all of which need to be complete, but I don't necessarily care about the order and get some interesting stats that way. That's very difficult to do in Mixpanel. I don't think it's possible. June is a little better, but I still can't quite do it. But again, I want to be able to push, have the system push info to me at the right time. So, for example, someone from Google just signed up, I want to know that. And if, unless I'm looking through my database or have some kind of, you know, strict logic around these big companies that sign up, I'm not going to get notified of this. And if I wait an hour a day a week and I miss that opportunity to just reach out and say, you know, welcome and roll out the red carpet for them, what a missed opportunity. The level 11 version of this, I think would also predict intent and intention and accelerate someone through a funnel. If they're ready, right? You don't want to prevent somebody from going at the speed that they want to go at. But on the low end for people that aren't sure or new to, let's say a field. So in growth bar, right? Writing SEO. Optimized articles might be a new thing for them. And doing SEO, keyword research might be a new thing to them. I want strict bumper rails for that person to be able to easily get some kind of useful output from the tool, some kind of utility from the tool. But if I have a power user on my hands, I want to almost immediately figure that out, remove all the guardrails and just show them. How they can. Quickly get the thing done in the system that they want to do and kind of get out of their way. I also want to know when a customer's about to churn and maybe remediate the problem. So tools like turnkey, but I think that should be kind of automated. And ideally it's not just a cancel button where you are. Forcing the user to work through this turnkey flow at a very, I guess, emotionally unstable point in, in their, in the life cycle, in their life cycle. So at their peak frustration, or once they've already been charged for a month on a thing they meant to cancel that's when they go and find the cancel button, they don't want to fill out any forms. They don't want to tell you why they don't want to talk to you ever again. You've already lost this person. It's very difficult to. Recover them. Turnkey is, is pretty good in the sense that it can help you. Set up flows for when a customer is exiting. So an exit flow and offer discounts or things like that. It's very cool, but I would love to be able to do this before they're at that moment, knowing that maybe they're getting less value. Then they used to get out of the product for whatever reason. And maybe it's asking them if they just want to pause instead of, instead of cancel so that they can come back when, when they need to. And on the flip side, I want to reward engaged customers. I don't totally know what that looks like. I think it would be unique to the application. But it's these, these are kind of my top five for what a really insane product analytics tool at level 11 would do, but basically it's, it's giving me. The insights I need. Right. And another example of this might be take a look at the actions of all the people that didn't convert. Take a look at the actions of all the people that did convert. And just show me what, like my ideal user looks like. And then show me how far off these churned users are. These people that didn't convert and automatically figure out what we can put in front of them to see if we can get them to take those baby steps so that they can get value from the product. Now, of course, if the product doesn't deliver the value you need, you're not going to. Capture all of your funnel ever. That's just not, I don't think a realistic goal, but I think you could capture much more if you could present that frustrated user or that new person. Around your UI so that they have to learn as little as possible to get value from your product. And as a complete aside, I'm playing around with this kind of fun concept of, of agents and for growth bar in particular, in SEO agent, you could kind of do this same exercise with, with a type of SEO agent. So. With Google search console, it's actually pretty straightforward to grab the keywords that you are ranking for and all your blog posts and what the state of the world is for your kind of SEO. And look at the competition and there's a ton of research tools, growth bar included that can give you great deep insights into how your competitors are performing. And this agent could go out, figure all this stuff out for you, help you create some kind of content strategy and then actually go and generate that. That content strategy. And then I think the cool part is by using Google search console, you could actually have that feedback on how well the tool is doing. So the tool writes the blog post, the blog posts after. A couple of months gets ranked and has some kind of search volume and it could, it could feed back into itself to, to learn what is working and what's not working. So again, all of these things are a lot to ask of a tool and but I find it to be a useful exercise on, on any product to say, like, what's, what's the level 11 version of this. And. In my younger years, I was pretty hesitant of this, but sometimes that means inserting a user or. I am a human into the process. So for growth, by for instance, a lot of people are new to SEO and SEO research as a whole thing. And I think growth bar, one area of improvement. Is, we need to simplify the product and make it. Very simple to get some kind of insight around what you're trying to accomplish with SEO for your brand. And then generate some kind of content calendar. I think we could do that automatically and then start generating blog posts that you could edit and, and do do that stuff with, but, but at least you have something to play with a sort of straw man to start with. I think that that would be a really amazing addition to a growth bar. But the point is, is that even still, the software may not deliver the entire experience to the user, the entire value for the user. Some people, a certain percentage of users, and this is why productized services exist. Don't want to do any of this crap. They like the tool. The tool is fine, but it's the five hours a week. They need to put into it to. I get the output they need. And that might be a place where you could actually insert a human in there and some kind of service, a function to do the parts that the user doesn't want to do. Even if the tool could in theory, let them do all of those things. There's going to be people that don't want to do it themselves. And so buying the SAS and then adding a service component. As a very unconventional thing, but we're exploring it with growth bar and I think it could have, you know, any kind of service revenue has with, with reasonable margins. Like, let's say 50% margins. At least it's going to have really positive impact on your P and L. And so, yes, it's a SAS company and people are obsessed with SAS, but as I get older, it's sort of like, you know, the money's all green. And if we can add or layer into a SAS product, some kind of service offering that increases the. Revenue of the company at reasonable margins. I think that to not do that, just because you want to stay pure, SAS is relatively short sided and that's why we're exploring it with growth bar. What's frustrating about this whole thing is that e-commerce really has this lockdown. And I mean, just by that, I mean, out of the box, you get a Shopify store and there's analytics that already plug into Google analytics and you could set up a Google ads account and start spending money and get a real. Dollar amount for return on ad spend relatively simply. It's very frustrating, but there's very few paths in e-commerce for a user to go down. I mean, of course it can get more complicated than this, but at a high level users can view products, then they can add them to cart and then they can check out right. There might be many other steps in between. Again, it could get very complicated, but. Your E-commerce store is going to have those three functions are typically well. And those just work so well in these analytics tool out of the box. And there's really no proxy for that for SAS. So of course there are things like login sign up and check out slash subscribe. But other than that, you kinda need to know a little bit more about the product. So, and sheet best. For example, it's a developer tool to turn a Google sheet into an API endpoint. If you don't know what that means, it doesn't matter, but there's, there's basically only a few paths that a user can go down in this app. It's a single purpose application. They log in or sign up, they add a Google sheet. And then they start calling the API. And then at a certain point, they need to check out or subscribe to the service because they've hit some kind of threshold, whether in this case and she best it's either a time-based one or a number of API calls during their free trial period of two weeks. So. It's really difficult to say. When does the user start to get value? Even in a very, very simple product, like sheet best. Is it the first API call? Is it the 10th? Is it the hundredth? And depending on what they're using it for healthy usage for that account might look like somebody logging in once a month. Healthy usage could also look like once a day. Those are, it's very difficult to make broad generalizations even about our current user. User base and a product that we own and have owned for three years. Exactly what healthy looks like and exactly what number of API calls results in value for that user. It's a very soft heuristic, right? This isn't always going to be easily quantifiable, but I think it's a important exercise to try. But that starts to hint at the complexity of why some of these. Product analytic tools, even for even PLG specific tools, just kind of miss the mark because we're looking at a portfolio of six companies currently, and there's just such a wide variance of needs across those companies. And it's still to this day, again, we have unanswered questions about. Exactly what a healthy account looks like. You know, it's like, you'll, you'll know it when you see it, right, because it'll show up in your LTV, it'll show up. And some of these other mastered metrics, but it is hard to say on the margin. When did the user get at how many API calls did it take for that user to start to get value? So one kind of thought experiment around this is if you have a button that says, do my job for me, whatever your job is, let's say you make 5,000 a month doing your job. Would you pay $5,000 to click that button? Probably not right. You don't get an arbitrage between the tool and how much you're getting paid for the work, but what about 4,500 maybe, right? You net $500 for clicking a button that's not bad. Maybe you can get two jobs and you click the button twice and now you make a thousand dollars and maybe that scales for you. But a certain number of people are not going to want to pay a $4,500. Right? There's going to be a cohort of users that are only willing to pay $50, even though the value to them, is there their end to end job being done? Let's say at least as well as them and arguably the $50 button versus the $4,500 button. There are probably two totally different businesses or at least different lines of businesses. Those customer bases are going to look different. Expectations around the product are going to look a lot different for those two For those two price points. But back to SAS analytics for growth bar, for example, we're looking to invest heavily in these base metrics right now, just to figure them out, right? It's a new product for us. We're still trying to wrap our heads around. What's working and what's not working. We recently removed when you sign up, you used to have to enter in a credit card. We've removed that. And now we have a two week free trial And it has been extremely painful. To wait for those two weeks to see what conversion numbers are. And literally today we're starting to get results in and I want to leave it for another two weeks. And again, this is our biggest acquisition ever. So we've got some real stakes on the line here. But I want to leave it for another two weeks just so we can get hard numbers around what conversion actually looks like with this versus what it was beforehand. But if you look at our Stripe dashboard, The data is totally screwed, right? So the, the MRR numbers, because of the way that there were automatic credit cards being charged, those were on annual payments. There was a lot of refunds. There were some disputes around those things because people were forgetting to. Hit the cancel button. And then they were seeing this you know, five figure charge for a piece of software that they were just trialing. And, you know, obviously that's, that's not great. So we we've changed that, but it's still difficult to see. To answer the very simple question of like, is this better? Dollars wise or is this worse? So what are some of the questions that are hard to answer and analytics products? So they churned, what did they do in the app? Like what, what exactly did they do? And June. So you can find this in there. But it's a little bit buried. And I still want to be able to compare those users that churned, those users that converted and what exactly was the difference between those two. And I think a large language model would have the ability to. Glean some insight between those two different types of personas and you do this across a hundred users for the non-converted and a hundred users for the converted. And maybe you can start to see something interesting and maybe that results in. No product changes, maybe it results in guiding users down a particular path or some kind of a modal that pops up that steps them through certain things, et cetera. But again, these are, these are numbers that we're going to have to go hunt in the database and across several different tools just to get. And it's a, it's pretty annoying and pretty frustrating that e-commerce companies get like a reasonable baseline analytic toolkit that just works out of the box. So in the department of random this week, I discovered Ben Chestnut's talk. He is the founder of MailChimp. And I just want you to, before you watch it, think of all the misconceptions or even all the TV shows that you've probably not watched on these. Crazy founders like Travis calender with Uber, et cetera. And think of all those, all the sticky notes you have attached to like what a unicorn founder looks like. Sounds like. Talks about, and I want you to watch this Ben Chestnut interview. Of course his story is slightly infuriating because he was focused on something else, built MailChimp as like this kind of side thing, and it's just started to grow. And he was doing like as little as possible for, for years and looked up one day in his own words and was like, oh yeah, wow. This thing is, is making way more money than this other business. We should just like do that. And It, of course wasn't that easy, but it, it wasn't variating to hear him. Talk about it in that way, versus like an Uber where they literally had an execution plan down to the nth degree for any type of town they were going into and, and totally steamroll. The entire town and both were both were equally effective, but it was refreshing to hear of another founder's perspective who was operating at that scale at, at that level with a wildly different operating framework. And I would go work for Ben Chestnut and I don't think the same is true for Travis Kalanick. I revisited Ray Dalio's principles and I'm stuck on one of his components in there. Most of their management decisions. So like their soft decisions, they have this whole framework around like when somebody is in an argument and they have like baseball cards of somebody's skill set and they have like believability indexes on everybody. So how believable is somebody who's opinion and how much should we weigh that? They have these cool algorithms that allegedly I've never seen them or worked there. So I don't know exactly, but they're making management decisions and people decisions and soft decisions. In addition to all of their market decisions, algorithmically. And that's both totally insane and totally amazing. And one of these weekends, I would very much like to take snapshots of every acquisition we've made at the time. That we made it will then. And what it might look like machine learning. Guide invest

Other Episodes

Episode 11

May 12, 2023 00:17:27
Episode Cover

Maybe A Thesis - v3

Picking an investment thesis is hard. We started out with one, ditched it, and are now re-thinking how we evaluate companies to buy or...

Listen

Episode 31

October 13, 2023 00:11:13
Episode Cover

33 - Post-Acquisition Growth Strategies

Summary for Podcast 33 - Post-Acquisition Growth Strategies Hey, it's Andrew from XO Capital. Dive in with me as we discuss how to approach...

Listen

Episode 13

May 26, 2023 00:10:09
Episode Cover

13 - AI Generated Content

In this episode, we dive into the exciting world of AI-generated content. Starting with the acquisition of Supportguy.co, we embark on a journey to...

Listen