After Spencer Kimball left Google, he found himself missing some of the custom-built software the company uses internally. So he and a bunch of fellow ex-Googlers started building their own. And now they want to make it available to everyone to power the next Google or Facebook.
Specifically, Kimball wanted something like Google's database system Spanner. Spanner is designed to juggle data between potentially millions of database servers, a tool that keeps Google's services online even if several servers, or an entire datacenter, go offline. While few companies need to operate at quite the scale Google does, the ability to stay online even if many systems fail, and to automatically balance resources between servers, would be useful to many other companies.
Kimball and his colleagues didn't work on building Spanner at Google, but they used it for their own projects. Seeing that there was nothing else like it on the market, they created CockroachDB, an open source database built using a white paper Google itself published about Spanner.
CockroachDB quickly attracted dozens of contributors after it launched in February of last year, but the project wasn't advancing as quickly as the team would have liked. Even now, the software is not yet ready to be used in the real world. So Kimball and eight other developers quit their jobs and founded Cockroach Labs. Today the company announced that it has raised a $6.25 million investment from venture capitalists including Benchmark, Google Ventures and Sequoia.
The CockroachDB team isn't the first to bring internal Google technology to the masses. Over the years, Google has published several white papers detailing many of its key innovations, leading to the creation of some of the most important pieces of software of the past decade.
Two of its papers inspired the open source data crunching system Hadoop, which became the foundation of the big data revolution and is now used by everyone from tech companies like Facebook to huge financial services companies to the National Security Agency.
Meanwhile, a paper about Google's BigTable data storage system helped kicked off a whole wave of innovation in database design, popularly referred to as NoSQL. Today, Apple, Facebook, Netflix, and countless other companies depend on databases inspired by BigTable's design for at least some tasks. But Google itself has largely moved on to Spanner, which the company claims can reach extraordinary scale while making fewer tradeoffs, especially concerning the consistency of data between servers.
Cockroach Labs is betting that Spanner-like databases will soon be just as important to the rest of the technology as Hadoop and NoSQL are today. And he thinks it can get there in part because it lets users start small.
"The biggest innovation in CockroachDB over Spanner is the simplicity of deployment," Kimball argues. You can install a single instance of CockroachDB on your laptop when your app is still small, he says, and then scale to hundreds or thousands of servers as your business grows.
Although Kimball thinks that Spanner represents the cutting edge in database technology today, he doesn't think it will remain so for long. He points out that traditional relational databases, as sold by Oracle and the like, have dominated the market for decades. And the first wave of NoSQL databases are less than a decade old, but Google is already moving on.
"The innovation has just begun," he says. "And there's nothing to indicate to me that it's going to do anything but accelerate."
