In September of this year a compilation video of failed SpaceX rocket landings was uploaded to YouTube. The video featured rocket after rocket going up in smoke and flames all set to the Monty Python theme song. To me, the most interesting thing about the video wasn’t the explosions, it was the fact that the compilation was uploaded to YouTube by SpaceX itself with CEO Elon Musk promoting it. Musk celebrated the failures because he recognized that each fiery explosion represented a valuable lesson for SpaceX as they sought to improve. The value of those lessons became crystal clear when on December 21, 2015 , SpaceX’s Falcon 9 sailed back to Earth and successfully completed a vertical landing at Cape Canaveral, Florida.
I’ve long admired Musk’s ability to look at failures as learning opportunities. It’s a mindset I try to promote to everyone here at CircleUp. Silicon Valley celebrates tech entrepreneurs who will always tell you they are “crushing it.” I’ve sat down with many such entrepreneurs over the past year alone. Two in particular told me they were doing incredibly well and raised over $80 million…and both closed their companies down within months of our conversation. Being an entrepreneur is exceptionally hard and stressful, so I fully understand why some feel compelled to adopt this attitude. However, I think it’s important that we, as an industry, acknowledge limitations much more often. This helps you stay honest and promotes a culture of constant improvement. You either succeed or you learn.
I often tell people that CircleUp’s technology lets us help entrepreneurs in a way that no one else can. That being said, Helio is far from perfect. There are holes and inaccuracies, and it will never be perfect. It makes mistakes every day and there are many things I wish it could do that it just can’t yet. Helio’s mistakes may not result in fiery explosions (knock on wood), but they still cause us to pause and ask ourselves how we can improve. We also have setbacks that are less about mistakes and more about how our technology is perceived or used, and I think these setbacks are equally important to callout. We learn from mistakes. In this post, I’d like to give an honest assessment of Helio’s strengths and limitations today and how I see its future.
I’ve previously written at length about how Helio works and why we built it, so I’ll give just a short refresher here before discussing our recent progress. Helio collects data on 1.3 million companies (as of December 2017) and evaluates them across an array of different factors to determine which will make great investments. Helio tries to evaluate things like a company’s distribution, packaging, brand, team, content, financial performance, product reviews, growth, and much more. Doing this for one company on a regular basis is challenging enough, but Helio is able to gather this information on nearly every single consumer company in the United States and updates constantly.
Here’s a high level overview of how Helio works.
Types of data
We think of the billions of data points that go into Helio as belonging to three distinct buckets.
1) Public data – Comes from collecting massive amounts of publicly available information on companies.
2) Partnership data – Comes from our relationships with a variety of partners that provide value-add data to Helio.
3) Practitioner data – Comes from our relationships with entrepreneurs and the information on their companies we receive when they work with us. Nearly 17,000 companies have applied to CircleUp, providing powerful ground truth data that is instrumental for training our models.
Combining all these sources at once is what allows Helio to produce such useful and compelling insights.
Helio is constantly getting bigger and better. The number of companies Helio tracks is now at 1.3 million and we are close to having data on every single consumer and retail business in the country. As mentioned above, we evaluate a variety of metrics on companies including distribution. The number of retailers Helio has data on has increased from 80 to over 270 and the number of products we track has increased by nearly 500%.
Here is a quick sense of scale. If we were just a $100m venture capital fund, we would be spending 1/4th of our annual management fee just on storing the data – this doesn’t include the resources spent on collecting the data, processing it or turning it into helpful algorithms. But it is, of course, not perfect. There are many retailers we don’t yet track, and a lot of data about companies that we still want to capture. While we are proud of the growth, we know there is a tall mountain ahead.
The accuracy of the models that make up Helio has also greatly improved. For example, we measure the accuracy of our revenue model by calculating the percent variance in ground truth revenue data that can be explained using our model. Updates to that model, including new features and new data sources, have improved the model by up to 11 percentage points.
All of our business units at CircleUp use Helio to find and evaluate great consumer brands early on, and Helio lets us provide valuable data and insights to these brands after they’ve received debt or equity financing to help their brands grow.
Despite the progress, Helio is not without its challenges. I see two main types of limitations to Helio today: technical limitations in the platform itself, and business limitations in the way the platform is utilized at CircleUp or perceived by others.
Technical limitations
We deal with datasets from hundreds of different sources. We’ve spent a lot of time and energy making sure that all this data is seamlessly linked for the companies we analyze, but we still have a ways to go. For example, if multiple companies share the same name, Helio sometimes mixes up the data points for those companies, thinking the distribution data for company A belongs to company B. We are using much more than name to solve this problem, but even so, in an omnichannel world with thousands of sources of information, many of which exist in an online or offline world in isolation, this problem is very complex and challenging. We’ve created what is essentially a normalized id for each company to help overcome this issue, but we still have work to do here. The below graphic demonstrates at a high level the complexity of the problem.
In this example, we are interested in collecting data on Duke’s – a meat snack company. There are a multitude of different data points that exist on the company, but there are also other companies with the name of Duke’s- in this case, a mayonnaise company and a dog food company. Figuring out which address, social media handle, product packaging, and many other artifacts belong to which Duke’s is very complicated, and we are simplifying things for this chart! Doing this for 1.3 million companies at once is what makes this task so complex.
When models are built on hundreds of data sources, it can be difficult to decide what to prioritize. For example, would it be better to have our data engineers and data scientists acquire data and build models that improve the revenue estimates for companies or their distribution growth? We work closely with our teams internally (marketplace, credit and fund) and selected partners externally, to understand what information would be most valuable. Given our limited resources, we don’t have the capacity to work on everything at once. We have to make hard choices when it comes to prioritizing data and we don’t always get it right. For example, early on we had a model that evaluated a company’s management team. We eventually moved away from certain aspects of this model because we discovered that its outputs, based on the data we have today, were not highly predictive of future growth. We still think a company’s team is very important, but we now have a different way of evaluating that importance.
Helio currently analyzes 1.3 million companies and classifies whether these companies are consumer or not, and if they are, what category they belong to. The majority of the classifications are correct, but every now and then we will see a company that isn’t properly classified (e.g. Helio tags a company as skincare when it is actually a vitamin brand). This adds unneeded noise to our category analyses, so we’re working hard to improve this system.
Business limitations
Having worked in investing for almost my whole career prior to CircleUp, I can tell you, investors almost always think they are the smartest people in the room. Every investor believes they have the magic touch that no machine could ever replicate.
That’s why many question whether it will ever be possible to use technology to effectively find and evaluate private companies. What these investors don’t realize is that following their gut can often get them into trouble. Recently, I was talking to a well-known institutional investor who had earlier passed on two companies on our marketplace. These companies went on to be huge successes and were acquired by top tier private equity firms. When I asked the investor why he hadn’t invested in these companies when they were on the CircleUp marketplace, he couldn’t name a specific reason. He told me that they just didn’t “feel right.”
I believe the more data we apply to investment decisions, the better those decisions will be. At CircleUp, we have a saying that Helio > Heuristics. We’re working to move past human bias, but investor reliance on gut is still something that slows down our conversations with investors on our marketplace or funds that we might want to co-invest with.
Now that we’ve covered the past and present, I’d like to discuss a few of the things we’re building towards.
Gartner’s framework is a helpful reference point for thinking through the various stages of analytics. The more valuable a type of analysis is, the more difficult it usually is to compute.
As an example, Helio can now tell us how a product performed at a certain retailer being sold for a certain price with a certain label. Soon, Helio will be able to tell us which of those factors (among many others) will correlate to sales growth the most in future settings. As one use case, we think we will then be able to advise the brands we work with on whether to make their labels a certain color or to target certain stores to sell in before others. This will be an exciting transition that will allow us to bring our entire analytical capability to bear in improving the way the consumer sector operates.
In that same vein, once Helio is able to predict and prescribe actions, it will be able to make investment decisions for us, without a human having to be involved at all. We already see successful systematic funds in public markets. Renaissance Technologies is one example of a fund that has had breakaway success in trading using quantitative models. To date, quant funds have been much more difficult to implement in private markets for many reasons, but primarily because structured data in private markets is much less readily available and it is much harder to back test algorithms and data sets because of the lack of liquidity.
But with Helio leading a new future for private investing , we plan to eventually build the first truly systematic private investment funds. Today, consumer investors spend upwards of 75-85% of their time sourcing deals manually. We think those resources can and should be shifted dramatically towards helping companies post close. Instead of spending months doing things like going over financials and analyzing the market size of a brand, we will be able to use that time to help the brand launch a new product or get into a new retailer. Instead of hiring lifelong private equity professionals, we can hire ex Sephora executives, supply chain or branding experts, and other functional experts who help our portfolio.
The billions of data points that Helio collects are hugely valuable to CircleUp as a company, but we think they have also have enormous value to others- not just investors, but brands and retailers, too. We’ve already begun to share some of our data with co-investors, retailers and of course entrepreneurs who have raised on our marketplace, but in the future, we want to help other industry participants in ways consistent with our mission.
We’ve come a long way with Helio but still have a long ways to go. I’ve always valued transparency and I commit that I’ll come back to all of you again in the future with an update on our progress. I hope by then that we’ll have tackled many of Helio’s limitations and have made strides towards the goals I set above, ultimately working towards our most important goal of all—helping entrepreneurs thrive by getting them the capital and resources they need.
At the same time, I can’t guarantee that everything we want to do will get done. We will have many more failures. One of the exhilarating things about being the CEO of a company like CircleUp is that new opportunities constantly present themselves but they often go hand in hand with new problems. Thinking back to the SpaceX video, we’ll probably have more of our version of things going up in smoke and flames over the next year. After all, this is complicated stuff. Building Helio may not be rocket science, but some days it feels pretty darn close.