A CAC Evaluation

One of the biggest mistakes I see junior investors make, particularly when they come from a purely financial background, is they look at numbers in early stage companies as points, or as simple trend lines. I don’t think this works.

In the early stages, so much is changing that the numbers don’t tell you that much. You have to dig into the business and understand where it is, how it’s moving, what the market and competitors are doing (and likely to do) and how all that will change the metrics as they are today. Using CAC as an example, here are some ways I think about early stage company numbers.

  1. When a company brags about how they have almost no sales and marketing spend, it’s usually a meaningless statement because to grow a large company they will almost certainly have to spend that money eventually. So all this statement means to me is that they haven’t really started marketing yet. The one exception may be if a product has some viral or consumerish characteristics, then maybe they will be right, but that’s the rare exception.
  2. When CAC payback is really high, it doesn’t really mean it’s a bad business. There could be a few reasons for it. One is that the ASP may be artificially low because the product is new and the sales are really paid betas priced for adoption. Or sometimes the first channel the company is using to drive sales is a more expensive channel than they should be using initially for their product category, but they started there because they were most comfortable with that channel. As they move to better channels CAC may come down.
  3. When CAC payback is really low, it doesn’t mean it’s a good business. It might be, but CAC changes a lot over the life of the business. If the company has started with a very efficient but not-that-scalable channel, then I expect CAC to go up over time and have to estimate what I think that will look like in a couple of years.
  4. Competition can both positively and negatively affect CAC. If you are in a market where you are a clear market leader, but other competitors are big enough to spend decent marketing dollars, then you may get a boost from those marketing dollars as well. Your competitors make people aware of the category but then the buyer does research and disproportionately chooses you. It can negatively affect CAC if your market becomes more competitive, or big players come in to compete with you, or one competitor raises a boatload of money and spends like crazy.
  5. You also have to think about how saturated is the market for the target buyer’s attention, and where do you fit in the priority of their attention stack? You aren’t just competing for your product category, but for attention against the other things that buyer is buying, as those companies are trying to get their attention too.

My point is, I don’t really pay much attention to the metrics of early stage companies, or the trend lines they are on. I mostly use them to connect those metrics to the operations of the business and make some educated guesses on how those metrics will changes as the business grows, based on the items above. If I invest, it’s because I think CAC is going to be good in a few years, and whatever CAC is at this current moment is less relevant.

If I share a deal with an investor, I hate when they say something like “CAC is too high.” I’d rather hear them explain why they think the number will stay that way.

Words Now Speak Louder Than Actions

When I was growing up, there were two common things people said about actions and words. The first was “talk is cheap.” The second was “actions speak louder than words.” For a long time, those statements were true. But today I think we live in a different world, a world where words speak louder than actions. How did we get here?

Some of you who have known me for a long time know that I was an early blogger. I started blogging in March 2003 and had what used to be a very popular blog, at a time when not many blogs were around. I sold that blog in 2008. What struck me at the time was that most bloggers were talker/writers, not doers. There were lots of people who wrote for a living, or wrote a lot as part of their jobs, and not a lot of say, middle managers who were actually in the trenches working on things.

It makes sense. Those who write a lot found it easier to blog. Those who had more flexible jobs like consultants or academics found more time to blog. A VP at Fortune 500 company was likely working 50+ hours per week just to make things hum at work and probably couldn’t write about most of their situations anyway.

The problem is, that impacted what people consumed and affected how people thought. Everyone reading the blogosphere got a certain perspective on work that was limited by the fact that the content produced was skewed a certain way. (I’d love to see more people writing the kind of practical explanations of work that Steven Sinofsky writes.). This was the beginning of online ecosystems favoring talkers over doers, ever so slightly.

Fast forward to the rise of social media, and algorithms that promote popularity over everything else. This just made the problems worse. Now it seems, your ability to talk, to sling zingers and memes, is what leads to popularity, because gaining popularity around actions is more difficult. Words are easier than actions. Cranking out words is faster than doing an action.

As cycles in business, politics, news, and media get faster and faster, words can compete for attention better than actions because more attempts for attention can be achieved with words than with actions. The cycles are too fast for actions to matter.

In a world where promoting your words becomes increasingly easy, I’d expect that to cause words to lose value. I’d expect it to cheapen them. But somehow the echo effect, the cascades around catchy words, have actually made them more effective. Hopefully it reverses someday, and we become a people of action, where what you do matters again more than what you say.

I’m Back

Wow. I can’t believe it’s been 3.5 years since I posted here. I like to write, and have been writing at Investing In AI for a couple years, but my professional and personal lives have been so crazy busy I haven’t gotten around to more general writing. That is about to change again.

This blog is called Coconut Headsets because it’s about clear thinking, and is a reference to cargo cults and their behavior after WWII (google it if you want to learn more). I want to get back to writing about how to think more clearly because it’s sorely lacking in today’s meme-powered online discourse. Wisdom and thoughtfulness don’t carry as much weight as social media showmanship and the ability to sling one liners at your opponents. It isn’t really very useful for getting at the truth.

There are some things here you should check out if you are new. In particular, lots of VCs are writing about this new model of AI of turning services businesses into software. I’m glad they finally caught up as I first wrote about that in 2020. A recurring theme of this blog will be areas I think people are wrong or misled about AI, because there are many.

Anyway, expect 2-3 posts per month. Maybe more some months. I hope you find it useful.

Are There Diseconomies of Network Scale?

Albert Wegner wrote an awesome post recently on how Innovation Upends Extrapolation. The gist of the post is that it is dangerous to extrapolate a trend into the future when you are dealing with complex systems. One of the reasons for this that isn’t directly mentioned in the post is the idea of “diseconomies of scale.” Most students don’t learn this in business school – they just learn the idea of economies of scale. But diseconomies of scale are real. They arise because, at some level of scale, a system gets so large that it becomes tough to manage, and the costs of management make additional growth less economically attractive, or more difficult.

Diseconomies of scales are one reason that, despite all the concern in the mid 2000s that Walmart was going to own the world, they didn’t. Similarly, the concern in the 1980s that Sears would own the world, on in the 1940s that A&P would – were all wrong. Why? Diseconomies of scale eventually catch up to most physical systems.

Facebook, Google, and Amazon though, have gotten so big because diseconomies of scale are much rarer in digital systems with lower marginal unit costs and network effects. I don’t see the natural drag of diseconomies of scale slowing these businesses down. But Wegner’s post made me wonder if we will see a related concept for network effect businesses, something like “diseconomies of network density.” As networks scale, are there things that operate as natural attenuators on their growth and power?

Wegner’s post looked at the urbanization trend, but denser populations in certain areas can make the world ripe for a pandemic like covid. There are downsides to network density. Is the same true for digital networks?

Well, we see in all the talk about the election and social media that disinformation could be one attenuator on scaling network effects for certain types of networks. When barriers to publishing information were higher, and distribution was more local and contained, the value of pushing disinformation was low. Networks like Facebook and Twitter changed that incentive and value structure. And now the results are starting to affect the usage and growth of those tools.

This ties back to where things are going with intelligent systems because, on the one hand, intelligent systems might fix some of these problems for digital networks by working as auto-arbiters that make decisions about these things as good or better than humans. But as intelligent systems learn more, they could have their own new knowledge based networks, which could have their own problems too. And I wonder if there will be attenuation forces that will slow down their power, or if they will grow until they too are manipulated. Will there someday be “diseconomies of knowledge scale”?

These are issues we should think about as we build intelligent systems.

Can Venture Capitalists Be Helpful? Should They?

There is a new “VC Burn Book” going around where founders blast VCs who they feel treated them poorly. This is the kind of thing that crops up from time to time, and I understand why. My single worst experience as an entrepreneur was when a west coast VC, less than 10 minutes into my presentation, stood up and said “Rob, not only do I not like your idea, I don’t like your personality” and walked out of the room. It was pretty painful but, I’m also a pretty difficult person to offend so it didn’t particularly bother me. People have bad days. People have opinions that I disagree with. But I don’t think I’d recommend he behave that way to everyone. VC behavior can be pretty bad sometimes so I understand why these sites exist.

On the flip side though, having pitched over 60+ VCs across 2 companies and 11 years as a startup CEO, and now being on the other side, I find that founders sometimes have unreasonable expectations about how VCs should treat them. Respect and directness are deserved, I think, and I don’t like VCs who don’t offer that, but, some founders believe that every VC should worship the ground they walk on, even when they haven’t yet proven themselves, or they expect VCs should help in all kinds of ways that we really shouldn’t. This is a small minority of founders, but, I think some of them are the ones driving these lists.

It brings me to a bigger issue though, which is – can a VC really help, should a VC really help, and if so, how much?

When I got into the investor side, I was repeatedly told that former operators like me often invest in companies that have problems we know how to fix, but that isn’t how you make money. And I believe that. I can’t invest and then run the company, or hand hold the CEO at every step. CEO is a job where you constantly feel like you are in over your head. You have to be able to figure things out, learn from various sources, etc. VCs need to be careful about carrying a company too much because the company has to earn it’s own battle scars. It’s like being an over-protective parent. It seems smart when your kid is young but then when they get older and don’t know how to operate in the world as an adult, you’ve failed.

On the flip side, some VCs do basically invest and go away. They can’t, or won’t, be that helpful to their companies. And some founders believe VCs can’t be helpful. What I want to share today is some ways my own investors were helpful at times (and one example of when they weren’t) to show what I think are appropriate areas VCs can help.

Focus and Markets

The first story comes from the early days of Backupify. We were doing $300K in ARR, and we had 3 products: a Google Apps backup product, a business social media archiving product, and a consumer social media / email backup product. After we closed our Series A, it became clear we couldn’t support all three products from the development side, and it was also fragmenting our marketing efforts. We had to choose one. After two days of meetings with the leadership team, as a first time CEO I was agonizing over what to do.

One of my investors asked me a question. “Rob, why are you so worried about this? You have paying customers in all three markets, just pick one.” I told him “I want to make sure I pick the biggest market.” His response was “how much revenue are you doing?” I told him $300K ARR. “Rob,” he said “every market is big compared to where you are.” It was a great point. We knew we were struggling in all three markets. Better to be successful in any of them, even if we chose wrong, than lose all three.

Channel Programs and Focus

The second story is also about focus (sensing a theme?). We hired a MBA summer student to launch a channel program when we were about $1.5M ARR. After one year, in which we grew to almost $4M ARR, we had signed 65 channel partners, and they were 3% of our revenue. So I told the Board it wasn’t worth it and we were shutting down the program. One of my VCs says “wait, your 3%, how many partners was it?” It was 4 partners, so he says “Ok, fire the other 61 and put one full time employee on those 4 partners.” I pushed back. It seemed like a waste of money. If 65 partners couldn’t sell more than 3%, how could 4 sell enough to justify a FTE? But he talked me into it, and one year later, those 4 partners did 9% of our new revenue. Now this was interesting. We had focused first on how to make a partner successful, and now that we understood that, we slowly added new partners. We never got back to 65, but when we sold the company, channel partners were just over 30% of our total ARR.

When Investors Are Wrong

The core product we sold at Backupify was hard to keep up with. It was engineering heavy, and expensive to develop. At one point I gave the Board a presentation about how we were going to develop a second product – a Salesforce backup. The unanimous Board feedback was “um, your first product seems hard enough, and you guys are doing good but not great at it, we don’t think you are ready to launch a second product.” I took their feedback and chewed on it a few days, then had a Board call to thank them for their feedback and explain that I was going to do the second product anyway, and I realized they were unhappy and that I was putting my job on the line as a result. It worked out in our favor. As the best funded company in the space, we put pressure on competitors to channel their development resources into a new product too, because they were losing business to us since we could offer more SaaS data protection.

In general, this is how I think investors should behave towards CEOs:

  • Provide frameworks for how to think through things, and advice on how to handle complicated situations or situations that are new to the CEO. But focus on advice, not answers or dictates.
  • Support CEOs when they make mistakes or go against Board wishes, but, hold them accountable for their performance.
  • Do not get in the weeds of the company and try to fix detailed problems unless you have specific expertise, and that expertise is actually wanted by the CEO.
  • I don’t believe VCs, or VC firms more broadly, should be overly supportive with tons of operational help. It risks making a company weaker than it should be. As an example, helping to source or hire a few key people from network is helpful, and allows you to work with the company on hiring. Providing a free recruiting resource the company can use for years to source all roles means the company never builds that very important capability on their own.

The other thing I would add is, there are times when VCs can be really really helpful. For example, as an entrepreneur you probably have not sold a company before, or maybe if you have it was one or two max. VCs are part of this process regularly, and can help navigate it with much more expertise and experience than you have as a founder.

Overall, I think VCs can add a lot of value at key points in the business, but it isn’t always in the ways some founders want. Occasionally the bad reviews on these VC review sites come from these unreasonable founder expectations of what VCs should be doing, which is what prompted this post. And or course sometimes VCs try to force things that they think are valuable on a company that really detract from the company’s goals and focus. It’s always a balance. That’s why I believe founders should prioritize alignment and culture fit with a VC over anything else. You want to work with people who want to build a company the way you do.

Read The Paper. Don’t Be A Victim of Algorithms

There are a few lessons I’ve learned in life. The first was about reading hard things. I started my career as an ASIC/FPGA designer, and I often designed chips that had to connect to other integrated circuits. These ICs often had specification documents that described how they worked that were over 100 pages long. I found it difficult to read through them, and I could rarely find what I was looking for, so I frequently asked colleagues questions about those ICs. They would get frustrated and point to page 70 where I could find the answer. And so I realized that I needed to suck it up and learn to read these dense technical specs.

The surprising thing that came from that is after a year of doing it, it became much easier. The specs started to feel “normal” and easy to read. They made sense and didn’t seem as dense as they did when I started. I even became more willing than most people to read the dense specification documents, and thus became a valuable resource to my team as everyone knew if they had a question I had read the spec.

I’ve maintained this view since then that occasionally struggling through dense writings is valuable, and does something to your brain that shallow writing doesn’t do.

That is part of the reason I never gave up on the paper. I’ve been a regular paper Wall Street Journal subscriber since 1996, when I was in college. And today I also get the paper New York Times, and a handful of tech and business paper magazines. The reason I stuck with paper is because I realized in the early days of the internet that the dense stuff rarely gets popular online, and online algorithms, even the early ones that were mostly just crowd sourced, focused more on linkbaity kinds of content. If I wanted meatier information, I had to stick with paper.

But over time I realized something – that by sticking with the paper version of these things, I get a more balanced view. Why? Because when an editor has to choose what goes into a paper that is going to be distributed to thousands of different people, and the format is unchangeable, it can’t be highly personalized, and they can’t put in much linkbait. The very fact that the paper has to go to a whole neighborhood insures it will have more even-keeled content than online algorithms.

I know. I know. It’s terrible that editors are gatekeepers and are in control of what you see if you read the paper. They suppress things sometimes. I understand that. But, it’s naive to think any other systems are better. Reddit or Hacker News? The crowd is the gatekeeper. Google or Flipboard? The algorithm is the gatekeeper. There is too much content in the world, so in every scenario there is a gatekeeper, and those gatekeepers are changing what we read. Once you accept that fact, you ask yourself a different question – what are the incentives of that gatekeeper? And I believe a human editor is the gatekeeper that has the strongest incentives to stay the most balanced.

One way to be less of a victim of the online algorithms or crowd based information cascades is to read the physical paper. It is more difficult for a medium to control what you think when it can’t hyper-personalize the information to your specific needs and wants. And that’s a good thing.

The Counterintuitive Effects of Cancel Culture. How Banning Things Makes Them Stronger

They say sunlight is the best disinfectant. I think about that every time I hear about another protest against a speaker or an idea. If you really want to kill stupid ideas, you should actually let people discuss them publicly. Otherwise, they get pushed to the far corners of the internet filled with other nutjobs* There they get sympathy, not just for their idea, but for the fact that they were not allowed to speak publicly about it.

If someone wants to give a controversial talk, like to say that women are less intelligent than men, or that the Holocaust never happened, or that one race is genetically inferior to the other races, the best way to marginalize those ideas and highlight how stupid they are is to let them speak. Don’t even waste time on protesting the speaker. Don’t cancel them. Shrug and say it’s not even worth your time to fight, or, if you do want to fight, go listen to the talk and write a summary online about how idiotic the talk was.

The way to defeat ideas you don’t like is to make them compete in the public domain of ideas, not to censor them. When you censor them, you think you are squashing them out, but you are just giving them protected time and space to grow and find more supporters.

*if you are reading this at some point in the future, it is quite possible that the term “nutjob” is politically incorrect for some reason, but as of the date I am writing this it is within the bounds of things you can say, so please don’t crucify me in 2035 for writing this in 2020.

The CUP Theory of AI Defensibility For Services As Software Business Models

I’ve written a bit about “services as software,” one of the models I like best in a world of AI. The gist of the model is that you take a services business where humans used to provide the service and you either use robotics ( if it’s a physical service) or algorithms (if it’s a digital service) to do it instead. But the rest of the value chain remains the same.

Most people, when they hear about this, think it’s just about cost savings, but it isn’t. It is so much more. And it’s that “so much more” that can provide defensibility to the business model.

My theory here, which is admittedly untested by the market, but that I am investing against, is that defensibility in services-as-software businesses will arise from 3 things: Consistency, Ubiquity, and Personalization, which is why I call it the CUP theory.

Consistency is the value people most often overlook when thinking about services-as-software models. The reason to have a robot or an algorithm provide a service is because it can do the exact same thing over and over and over. If you use the robotic barista CafeX in San Francisco and then again in Atlanta, the experience can be 100% the same in both places. You don’t realize how important this is until you realize how finicky humans are about many of the services we consume.

That brings me to Personalization next. Services-as-Software companies don’t personalize the way a Netflix recommendation engine does. It isn’t about “what people like you may like” next based on what other people like you did. It’s about learning what your actual preferences are by interacting with you. This is part of the value of machine intelligence – the machines get smarter about you, meaning they can provide you more personalized service. And, to the point above, that service can be consistent across time and geography.

But is it defensible? Can’t I go build and deploy many more machines that are like CafeX and then you will take whichever one is nearby?

That’s where Ubiquity comes in. The defensibility comes from the fact that Consistency and Personalization are valuable enough that, if there is a CafeX nearby, I don’t see the need to use a competitor. My theory then, is that defensibility in some of these services-as-software businesses will arise from the combination of Consistency, Ubiquity, and Personalization.

This raises an interesting question for companies building services-as-software models, which is – how fast must effective ubiquity be achieved? If these companies want defensibility, are they in a race for market share? And if so, how does that impact financing milestones?

My personal belief is that early companies in this space should figure out what “ubiquity” means from the user perspective for their service. If it means worldwide geography, then you probably have to raise more money faster than if it is purely online, delivered at home, or only applies to local geos. I also believe that there will be some market adoption time needed by consumers to shift to accept these models, and while it will be short, it does exist. I don’t think companies can scale until that transition happens. So balance your fundraising against that backdrop.

I used CafeX here as an example because I am not an investor, so I don’t know if CafeX thinks this way about their business, and if it is indeed defensible long term, but I am invested in a handful of early stage AI services-as-software companies that are thinking about it this way, and once they launch I will write more publicly about whether this thesis pans out. If you are building a services-as-software company, I’d love to hear from you. At PJC we think a lot about this emerging business model and it is one of our favorites.

When AI Fails: Welcome To The Real World

In the early days of Talla, one of our data scientists, Daniel Shank, attempted to replicate a popular paper out of Google, on “Neural Turing Machines.” He spent a ton of time trying to get it implemented in the real world, and never could. He gave a public talk about it here, which is interesting to watch.

I was reminded of this today when I read about the real-world failures of Google’s diabetic blindness AI. While showing 90 percent accuracy in the lab, it did poorly in the real world. Why? Because the real world is messy.

If you aren’t technical, some things seem easy that actually aren’t. For example, a user needs to add their name to a form so you can store their name in your database. Easy enough right? All you have to do is add a field and a button. But, what if the user puts in numbers, or weird symbols? What if they say their name is 6$#rt!? Should you accept that? What if it isn’t capitalized? Do you need capitalized names? This is called input validation. You have to make sure people don’t put bad input into the system, because every product manager who builds tech products knows that users will definitely put things into your system you did not expect. Always. Now extrapolate that across all areas of data input and output.

What you learn is that the real world is messy. Data is incomplete, inaccurate, on media you didn’t expect, and then other things happen that aren’t data related. Power goes out. Wifi goes down. There are a million things that could go wrong, and so inevitably some of them do.

But AI hasn’t been heavily impacted by these things yet because AI has been in the “research” phase these past few years. The AI celebrities we all listened to were researchers who could make new models or solve new problems, or deal with new types of data. But they are losing steam because so much of this doesn’t actually work in the real world.

It works in the lab where the world is tightly controlled, data is clean, and often generated just for the target application. In the real world, AI systems have to deal with the messiness, and they generally don’t do that well. It is a big problem.

The reason this is important is that, the power dynamic in building AI is currently shifting from researchers to application engineers and product managers who can make stuff work. As tools and platforms abstract away more of the matrix algebra, hyperparameter tuning, and other things that required a deep understanding of how neural networks work in order to do, the power is shifting instead to optimization and application. It’s less about can you build a model to do X, and more about can you get it to run with a certain latency, or power budget, or on a specific footprint. The mindset is different.

A lot of my investing focus has changed over the past 9 months to figuring out what really works in applied AI. I’m more concerned with teams getting AI into the world for real applications than I am teams that publish lots of papers, because building real world AI is still hard, and requires a combination of AI understanding and real world engineering that is still relatively rare.

While some say we are going into an AI winter, the truth is, the Covid crisis is accelerating a secular trend towards AI deployment in the real world, making the companies that can actually build and implement things very valuable.

The Lily Pad Problem – Why We Mishandled Coronavirus

There is a problem you sometimes see on IQ tests that reads something like this:

If the number of lily pads on a pond double every day, and it takes 30 days for them to cover the entire pond, on what day is the pond half covered?

Many people answer 15 days, as that is the basic human intuition answer, but the correct answer is 29 days.

Now, imagine that the pond being fully covered in lily pads is a really bad thing, and your job is to watch the pond and warn someone if this comes close to happening. So you watch the pond, and for the first 26 days, the lily pads are doubling, but, at this point, only 1/16th of the pond is covered with lily pads. At this point, it doesn’t seem like much of a threat, but over the next 4 days, the lily pads take over.

Day 27, 1/8th of the pond is covered, and for the first time it seems like a noticeable jump, and makes you think you should watch more closely. Day 28 1/4th of the pond is covered, and you realize something must be done, but you only have 2 days. And marshaling the resources to do anything may take longer than that. Day 29 the pond is half covered. Day 30 you are done.

The best way to fix this is to let people know on day 15 that the lily pads have been doubling, and something should be done. But it is hard for others to listen and take that seriously. They would say things like “so what if they are doubling, it’s such a small number” or they will look at the total number of lily pads and the nominal growth since day 1 and thing you are just crazy. Or sometimes they will assume there are other natural circuit breakers in the system, something that will slow the lily pad growth.

The lily pads don’t seem to be a problem, so it is hard to move resources against them. This is the ultimate problem with many complex systems that have such explosion points – once you realize they are a problem, it is too difficult to stop.

I joined the Board of the New England Complex Systems Institute last year because I have believed for a long time that we are going to see more issues like coronavirus, and that existing mental models for dealing with them are not adequate. We need better policy at all levels of government and industry.

On day 15, when the lily pads are small, so many people will argue that draconian measures to stop the lily pads are not worth the economic damage or resource drain to fight. But, in some cases, this is about survival, or about the fact that once the lily pads cover the entire pond, fighting them off is a 100x harder and more expensive.

If you are interested in understanding more about this way of thinking, you should consider taking a NECSI course, or I recommend the book Logic of Failure.