Open Source vs. Cloud Castles

🎧 Listen to this article

In my previous essay, I explored strategies for how cloud startups can compete with “Big 3” cloud providers – breaking up their dominance by avoiding them, or going up the stack, or building deep IP.

There’s another strategy that startups can consider when going up against Big Cloud: Open Source.

Former Microsoft CEO Steve Ballmer once called Linux “a cancer.” But Microsoft’s $7.5B acquisition of Github shows how far Microsoft and the entire industry of vendors and customers have come from cautiously using open source to embracing it.

While the Big 3 may arguably be the greatest beneficiaries of open source, a growing group of open source companies are also proving to be among the incumbents’ most successful challengers.

These include a mix of public and private companies such as CockroachDB, Chronosphere, Confluent, Databricks, Elastic, Hashicorp, MongoDB, MuleSoft, Postman, Redis Labs, and Temporal, which have all achieved multibillion dollar valuations in recent years. The number of open source projects and companies have exploded the past few years, and I have found a couple of online resources like the Cloud Native Computing Foundation (CNCF) market map and OSS Capital’s list of commercial open source companies useful to keep track of all the projects.

In previous computing eras, we watched startups employing open source to directly compete with the incumbents. Examples: Red Hat used open source Linux to go against the Microsoft Windows monopoly. Xen and KVM projects competed against VMware in virtualization. And Postgres and MySQL against Oracle in databases.

Now, we are seeing open source startups built in (and for) a cloud-native ecosystem. Previously, older open-source companies were mostly open source versions of existing on-premise systems. As a result, these new startups are making a market impact far quicker than those from earlier generations, because they are combining the low friction distribution of a cloud service with the open nature of open source communities to reach developers.

Case in point: Chronosphere, which launched in 2019 and has been landing enough customers to see its ARR increase nine-fold in the past year. The company was built on the open source observability platform M3 – which founders Martin Mao and Rob Skillington developed at Uber – and Chronosphere has steadily expanded its products. To keep up with the demand, the company just raised $200 million at a $1B valuation and recently released a distributed tracing product.

Chronosphere is part of a notable group of recent open source startups that have achieved success with a variety of approaches; namely, leveraging an open source project that solved a hard problem for a cloud scale company (in this case metrics at scale at Uber), and then creating a cloud-hosted offering for other companies to use.

The lines between open source companies and cloud companies leveraging open source are beginning to blur. While Chronosphere is an example of a company commercializing a popular open source project, many startups have built proprietary tech atop the open source offerings of the Big 3 without having to become open source businesses themselves. For example, companies like Instabase, Rockset, and Snowflake are not open source business models, but have built interesting (and technically hard) products in data warehousing, real time analytics, and document understanding, respectively. Each of those companies leverage deep open source tech under the hoods (like RocksD) but have focused on reaching developers directly with a cloud service.

Why is OSS on the Rise?

Open source has become the best way for new projects to start and generate collaborations by companies, universities, and individuals across the globe. You can find open source projects in every software category from javascript libraries, machine learning frameworks, and design tools.

The most popular open source projects are attractive for their distribution advantage: they reduce friction to awareness, trial, and adoption by customers. (Plus, they often have cute animal logos!)

A popular project can solve the top-of-funnel concerns for startups and allow them to reach their customers directly to compete against the cloud vendors. If a startup’s open source project can achieve the status of the de facto solution in a particular market and own the developer mindshare, then the startup has a marketing advantage against the big cloud players.

Finally, making the code free and easy to download (or easy to launch on a cloud) reaches the developer directly, bypassing the corporate procurement and other gates that can increase friction to adoption.

The strategy is spilling over into other technology industries. For example, RISC-V is taking the open source business model to the semiconductor market to disrupt the ARM franchise. In fact, almost every application – from a mobile app game to a database to the browser you are reading this blog on – uses several open source libraries.

Google, Amazon, Azure, and most cloud vendors are all built on open source technology. Amazon’s original web service, Elastic Compute Cloud (EC2), was built running open source Xen virtual machines. AWS has continued to leverage the mainstream adoption of open source by offering dozens of services either built on open source or simply hosting open source projects. The latter has prompted a recent response from startups by changing open source licensing to prevent Amazon and other players from benefiting from the code created by the open source community. (We will dig into this licensing debate later).

I believe most infrastructure and areas of computing can benefit from an open source project to help push research, development, and community – but that doesn’t necessarily mean every area should have or need a commercial option from a startup or company. Some areas of technology, like javascript frameworks, are notoriously hard to monetize. Github is littered with thousands of open source projects that have no activity

What Makes OSS Work?

For an open source project to potentially yield an interesting company, I believe the following elements are needed. (And even then, it doesn’t guarantee an outcome. See “Field of Dreams” above.)

Solving a Hard Problem

This is probably the most obvious factor. You need to be taking on something that requires solving. There are many ways to define what makes a problem hard – from issues like virtualization that may require deep operating systems knowledge, to challenges in categories like data integration and messaging, which require interoperability – the list goes on.

The commonality among these various problems is that they must be hard enough to not only guarantee there are few substitutes, but also worthy enough that developers and customers will value it. If there are several substitute open source projects, then creating another company that is just one of “N Projects” means that the project has to be the clear leader. In an ideal world, the company is one of one.

In most cases, I believe that if you solve a hard problem to which there is a free open source alternative, customers will pay for the security and usage of a superior project. This is precisely why Hashicorp has been such a successful company: it has several open source projects including Vault, Terraform, Consul, and others, and their business model is a combination of governance and security for these projects for on-premise use as well as closed,hosted services for cloud-native customers.

Making the Product the Business

Dev/Test is nice, but production is nicer. The technology should be used in the actual operations and run time of your business – and hopefully powering your business-critical applications. You can make money in dev/test, licensing, or other areas of software, but customers will pay more for production bits.

Like I said, success is not guaranteed, and there have been unfortunate examples to the contrary even with critical open source projects. For example, OpenSSL had no company and no supporters, and as a result was underfunded and under-supported. Additionally, some projects fall into a tragedy of the commons: they become so popular and widely used that everyone is happy to use the technology but few are willing to pay for a productized (or even supported) version.

Having Community-led Growth

The vibrancy of the community around an open source project is how you know you are solving a hard problem, and also typically an indicator that there are few substitute projects. It also allows you to address several critical factors at once, including:

• Problem awareness: Does the custom know this problem exists and is a priority?
• Solution awareness: does the user know that a solution or open source project exists?
• Company awareness: does the community know and recognize that a company is providing a product or offering support?

Finally, community-led growth reduces friction to trial, adoption, and usage of the project either as an enterprise product, or more frequently today as a hosted cloud service.

The more people using a project, the easier it is to attract contributors, making the project better, which in turn attracts more customers. The first to do that is most likely to create a flywheel from these network effects.

The owner and audience winner isn’t necessarily always the one offering the best technology at first, but often the most vibrant and dedicated developer community, which improves the product over time into a winner. MongoDB is another good example of being unique as a document DB. Elastic wasn’t the first or only search indexing technology, but stood on the shoulders of previous projects like Lucene to become the community winner.

Getting the Timing Right

Open source projects have network effects. Being first to market as the first open source project isn’t always guaranteed to be a winning strategy, but the first project to gain critical mass of developers and community interest is typically the winner.

Likewise, being first to a cloud service isn’t a guarantee to beat the big cloud vendors – but it helps. Mongo Atlas and Elastic’s cloud services were released ahead of Amazon’s DocumentDB and hosted Elastic, which probably made all the difference in the world.

MongoDB has been aggressively moving its business to the cloud over the past few years, and in Q1 of 2021 more than half of its revenue came from the cloud Atlas business. Other data companies are trying to make this transition to cloud. For example, Confluent, the company behind popular open source project Kafka, published in its S1 that only 18% of its revenue is in the cloud.

Evolving with Tech Transitions (Cloud-Only)

In the past, open source companies started with a commercial core on-premise product and then created a cloud hosted solution to complement the private cloud offering. As the cloud has evolved and cloud native is the default starting point for developers, startups are skipping the on-premise business model altogether to build for the cloud.

This strategy not only has the advantage of timing discussed above, but also allows the startup to architect the technology for a cloud-native environment, free from the compromises needed for hybrid offerings.

There are plenty of examples of companies that bypassed the on-premise to cloud transition and went straight to cloud-only. Databricks, which commercialized the open source spark project, started entirely on the cloud and shunned on-premise customers originally after seeing how Amazon’s Elastic Map Reduce (EMR) was an effective defense against Cloudera, which was slow to move to the cloud.

Remembering That Licensing is Not a Moat

While licensing does help with timing strategy, I don’t think it is enough to build a moat against Big Cloud. Recently, more companies have been changing and using restrictive licensing to prevent cloud vendors from taking the open source project and running on their clouds. Mongo was one of the first when it started using the AGPL license, but companies like Confluent and Elastic have all released a version of an open source license that makes it difficult for Google, Amazon, or Azure to run their software in the cloud without cost or penalty.

If a company can slow a cloud vendor or competitor from embracing your project, that gives the startup a timing advantage. However, these licenses will not stop an Amazon, for example, in building a compatible service like DocumentDB is to Mongo, or even potentially forking a project like they did to Elastic.

Conclusion

At the core, the tenets of successful open source practices hold true for any strategy you undertake: solving a hard problem that people are aware of; providing a solution that people agree is helpful; establishing your company as the problem-solver; making usage and adoption of your product easy; and, of course, crafting a successful go-to-market strategy that is scalable.

However, bear in mind that while open source is a popular, logical, and often successful strategy for many reasons, adapting it to the cloud-only era comes with an extra set of challenges.

Startups using open source strategies during the cloud transition era provided critical products and services (which allowed them to compete with the Big 3), but the same tactics may not necessarily have the same impact on in the cloud-only era. Thus, challenger startups must carefully consider their entry point and develop a product and business model adapted to the current, evolving cloud ecosystem.