When Alex Ratner joined the Stanford Artificial Intelligence Lab (SAIL) as a PhD student in 2014, he planned on spending most of his time researching better machine learning modeling techniques and algorithms. This, after all, was the main thrust of AI development at the time.
But not long after starting in the lab, a variety of people from across the Stanford campus began coming to him with unusual requests. Genomics collaborators at the Stanford Medical Center said they were buckling under the time-consuming effort of prepping and labeling thousands of patient records that they needed to train AI models for automating tasks like pulling facts from the scientific literature. Radiologists wanted the Lab’s help labeling medical imaging and monitoring datasets for triaging cases in under-resourced hospital systems. In these cases, and many more, there was no ask for a new algorithm or model. The pain point was all about labeling and curating training data.
“At first, we ignored these requests, thinking – like most did in AI then – that this wasn’t our problem,” says Ratner. “Labeling and developing the data used for training and evaluating AI models was viewed as an upstream janitorial task. All people cared about in AI was the fancy new model.”
But then he and several SAIL colleagues considered whether it was time to adjust their focus. “All these users were telling us they were getting stuck on the data before they could even get to the algorithms,” Ratner remembers. “We decided to start thinking about this messier system-building problem that seemed to be blocking real-world AI progress 90% of the time. It turned out there was something really meaty that no one else was paying attention to.”
Data, Data, Data
In fact, the process of training models with the right data became far more important than Ratner and his team initially surmised. He now regards data — whether patient records, financial reports, maintenance manuals, or customer databases — to be the deciding factor in whether a machine learning project succeeds or fails.
The customers of Snorkel AI, the company Ratner and four colleagues spun out of SAIL in 2019, seem to agree. Memorial Sloan Kettering works with Snorkel to automate the process of understanding pathology reports and identifying potential patients for clinical trials. Wayfair uses the company’s Snorkel Flow product to tag product information for better, more accurate search results. Seven of the top US banks use Snorkel to extract information from complex financial documents. In each case, Snorkel has helped companies shrink the time to prepare and label their raw data from months down to weeks.
Ratner attributes these successes to those lessons learned back at Stanford in 2014. “Everyone at Snorkel spends time with customers, and we build based on that feedback,” he says. “It’s this philosophy of listening to our customers and building in the field with them that drives us.”
A Pivotal Mentor
A New Jersey native and Harvard physics major, Ratner landed at Stanford’s computer science PhD program after 18 months of trying to get an AI startup called SiftPage off the ground. It was a largely self-taught, solo endeavor. After doing related work at a small consulting firm and taking some Coursera courses on AI, he started SiftPage which involved extracting data from patents and other legal documents, on his own. Applying to Stanford reflected both his academic side (“I’ve always loved abstract problem solving.”) and a turn toward a formal education in computer science and AI.
At Stanford, Ratner studied and researched under the tutelage of Christopher Ré, a MacArthur genius grant winner, Stanford AI lab faculty member, and serial entrepreneur. In the summer of 2015, Ré, who later became one of Snorkel’s co-founders, urged Ratner to investigate the data management requests SAIL was getting from various points around the campus, framing it as a low-pressure “afternoon project.” When it ballooned into something much bigger, Ratner and his team (Snorkel co-founders Paroma Varma, Braden Hancock, and Henry Ehrenberg) began developing experimental open-source software for programmatically labeling training data. They hosted weekly “office hours” to test their ideas and find out what other developers wanted to see. Tech teams at Google and Intel began using the open-source software.
As word spread, the weekly office hours often swelled to 50 or 60 people. A team of more than a dozen Accenture consultants and partners showed up, as did journalists from the International Consortium of Investigative Journalists. Investors reached out with offers for seed money to start a company, applying increasing pressure for Ratner to drop out of the PhD program and take the funding. His advisor counseled patience.
“I got really good advice and support from Chris [Ré] to be patient with listening to the messy, painful problems of users and not to rush this work,” says Ratner, who stayed at Stanford for two more years before the team left the lab, raised funding, and set up shop at Greylock’s Menlo Park office.
The surging interest, however, became a clear indicator for how broadly applicable the problem of organizing and labeling AI training data was. “The fact that there was such excitement even for a lightweight system meant this was a much bigger shift in how people thought about developing AI,” Ratner says.
Betting Big on a Huge Customer
While Snorkel’s open-source project had no shortage of big-name tech company users from their days at Stanford, what the company now needed was paying customers, who could act as “design partners” for the new startup. The first big lead came during a banking industry innovation summit in Menlo Park. A leading US-based bank was looking to develop AI to help it comply with the industry-wide transition from one credit benchmarking rate to another. This so-called LIBOR transition represented a huge undertaking, requiring an understanding of each instance where the old rate was used across all of the bank’s global contracts. The bank wanted to come by Snorkel’s offices and talk.
Still in its early stages, Snorkel still had fewer than 15 employees and was settling into new offices in Palo Alto. In an attempt to make their office look more populated, Ratner remembers frantically unpacking computers that had been ordered for future employees. Impressed with what they saw, the bank eventually asked Snorkel to help them develop a machine learning application that would save the company hundreds of hours of tedious labor.
Snorkel had a decision to make. The five co-founders wondered about the implications of working with a major financial institution with more than 200,000 employees. “It was this very gnarly, very high-stakes problem with a very large organization,” Ratner says. “There were debates among the co-founding team and our advisors that this was going to take away resources from our open-source build.”
In the end, Snorkel decided that this opportunity, while risky, was an unparalleled chance to learn. It also represented an ideal use case for Snorkel’s software. “We always knew the biggest need for our product would be with sophisticated teams who were working with lots of private data that they needed for training specialized AI models,” Ratner says. In addition, the automation of the bank’s LIBOR transition wasn’t just another AI pilot project; it would deliver an actual eight-to-nine-figure ROI for the business. For the next several months, the Snorkel team went all in on understanding the documents that would train the AI application, then worked with the bank’s technical teams to build software that could automatically adjust contracts to the new financial standard.
The project became a model for Snorkel’s R&D. For the first few customers in each sector, the company dove deeply into each organization’s unique data sets, business objectives, and specific AI-enabled applications they wanted to build. In 2022, the company folded these learnings into Snorkel Flow, its standardized, generally available platform.
A Ferocious Hype Cycle
By this time, Snorkel customers spanned not just the banking industry, but also telecom, biotech, oil and gas, and the federal government. In 2022, the 3-year-old company posted triple-digit customer growth. It seemed like nothing could stop the momentum.
Then ChatGPT appeared. By March 2023, when version 3 of the large language model chatbot launched, nearly all of Snorkel’s early sales conversations came to a screeching halt. As companies scrambled to redraft their AI strategies, re-org their teams, and get low-hanging-fruit chatbot demos built as quickly as possible, conversations with external platform vendors like Snorkel were deprioritized. Ratner recalls the mood: “People thought, ‘Why do I need to do data development? I’m just going to use ChatGPT for everything, to create my data set, to train my model.’ People thought it was magic. The whole market hit a red pause button.”
The Snorkel team knew that ChatGPT, while a remarkable advancement, couldn’t do everything. If anything, out-of-the-box models like ChatGPT elevated the need for solutions that could help evaluate and specialize them on specific sets of data. There would always be simple AI use cases, such as copywriting assistants or chatbots for common customer questions, that could leverage ChatGPT without much modification. But for many uses, companies would need to input their own data. “The hype ignored the common sense that no matter how smart a model gets, it can’t know the private data and private objectives and standards of an enterprise,” Ratner says.
As the fever broke, companies began to realize that bigger wasn’t always better. Deriving real value from AI meant developing intelligence from highly specific data. In the fourth quarter of 2023, Snorkel booked more revenue than it had in its nearly four years as a company.
Greylock partner and Snorkel board member Saam Motamedi attributes the Snorkel team’s success to a rare combination of scientific expertise and industry pragmatism. “They stayed laser-focused on their original insight of the importance of the data, which has become the bedrock of their success.”
Repeatedly Firing Himself
As Ratner migrated from researcher and PhD student to CEO of a now 190-employee company, he learned to dramatically adjust the way he spends his days. “One of our advisors wisely told me that my job as CEO is to fire myself as rapidly and repeatedly as possible,” he says.
The first role he fired himself from was software developer. “Letting go of day-to-day edits to the codebase I spent years on was a tough step,” recalls Ratner, “but at a certain point of scale it was clear I was blocking more than helping.” More recently, he sacked himself as the company’s internal lead for messaging and positioning, hiring an experienced marketing executive.
Ratner says he tries to stay connected to important areas of the company in “spikes.” For a set period of time, he goes deep into some initiative or problem where he thinks he can have impact. One of these current spikes, he says, is creating the next iteration of Snorkel’s product roadmap. On another occasion, he stepped into product development, helping to define the plan around Snorkel’s ability to support generative AI use cases.
With no shortage of interest from the marketplace, Ratner says his goal as CEO is to keep executing effectively, listening to customers, and tuning out the hype. On a day-to-day level, this entails taking a bifurcated approach: either going deep in founder mode or operating at a higher level in manager mode and fully trusting the people he’s hired. “Leaders sometimes get lost in the middle,” he says. “They have their hands in a lot of things, but they’re not deep enough to really contribute and not removed enough to be scaling efficiently. I’m always trying to do this better – to really go deep and lean in wherever I can help, no matter how tactical. And to quickly fire myself where I can’t.”
