Welcome, Mustafa Suleyman
A Pioneer of Applied Artificial Intelligence
We are thrilled to share the news that Mustafa Suleyman is joining Greylock as a Venture Partner. There are few people who are as visionary, knowledgeable, and connected across the vast artificial intelligence landscape as Mustafa. At Greylock, Mustafa will spend his time advising early-stage companies and investing in promising startups in the AI space.
Mustafa joins us from Google, where he was Vice President of AI Product Management and AI Policy. Before that he co-founded DeepMind, the world’s leading artificial intelligence company, which was acquired by Google in 2014 for $650M.
I’ve known Mustafa personally for more than a decade. We first met at a pub in London. While I have some fun memories of that night (including trying my first “toad in the hole”), what struck me most was our inspiring and provocative discussion on the power and potential of AI to help solve some of humanity’s most urgent challenges. It marked the beginning of a deep and lasting friendship between Mustafa and I.
There’s no doubt that AI is one of the most transformative technologies of our time. Mustafa has been at the forefront of some of the most exciting advances in this space. During his time at DeepMind, Mustafa led teams inventing and deploying cutting edge AI systems to more accurately detect breast cancer in mammograms, to diagnose 50 different eye diseases in OCT scans, and to control Google’s multi-billion dollar data centers to optimize energy consumption. He also worked with many teams across Google to apply the latest AI techniques in Android, Hardware, Play, and Cloud.
Over the years DeepMind achieved many groundbreaking contributions to the field of AI research and applications. Most notably DeepMind developed AlphaGo, an AI system that beat the world’s strongest Go player in a now legendary multi-day competition broadcast live and watched by millions of people across the world.
Moreover, Mustafa and I share a common love of philosophy, which drives how both of us approach our work in technology and entrepreneurship. At its core, philosophy focuses on improving our understanding of humanity, and how we evolve as individuals in a society. Mustafa has spent years thinking about how technological advances impact society, and he cares deeply about the ethics and governance supporting new AI systems.
What’s more, I know Mustafa is a builder at heart and he is excited to spend time with founders and to explore new ways that AI technology can make an enduring difference in the world.
Recently, Mustafa joined me on Greylock’s Greymatter podcast to talk about the current state and future of AI, his reflections on building DeepMind and his time at Google, and what’s next for him. You can listen to the podcast here.
Episode Transcript
Reid Hoffman:
Hi, and welcome to Greymatter, the podcast from Greylock, where we share stories from company builders and business leaders.
I’m Reid Hoffman, partner at Greylock, and I am thrilled to welcome the newest member of the Greylock family, Mustafa Suleyman, to the pod.
Mustafa Suleyman:
Hey Reid. Thanks for welcoming me to the podcast. It’s great to be here and I’m also very excited to start at Greylock.
RH:
Mustafa and I have known each other for over 10 years. For those of you who need a quick summary on Mustafa, he is a world renowned expert in artificial intelligence. He’s one of the co-founders of DeepMind, an artificial intelligence lab in London that was acquired by Google in 2014, for $650 million. And in the last couple years, he’s been a VP, working on AI Google.
Today we’re going to spend some time across a whole wide variety of topics: philosophy, current state and future state of artificial intelligence, reflections on DeepMind, being an entrepreneur, your time at Google, and what’s next to you as an entrepreneur.
But part of the reason I’ve been looking forward to this for years is that I know you personally, and as a friend, and what I have learned so much in so many different ways is the kind of questions around society and humanity and technology and governance and putting all these things together.
I’ve been looking forward to this direct journey of working on artificial intelligence together because we’ve been working on it in so many other contexts.
And while normally one might start with the bio and DeepMind – we’ll get there because it’s really interesting – I think one of the things that we should start with, because we have a shared interest in it, is philosophy and how philosophy leads to kind of shaping technology and shaping technology for humanity. And it’s one of the things that I think has perhaps been kind of under-discussed with you.
So let’s start. How did philosophy launch you on your path towards technology?
MS:
Yeah, that’s a great place to start. So back when I was doing my undergraduate degree in philosophy at Oxford, the big thing that I loved about that training was that it helped me to become a systems thinker – to think about, structurally, how the big chunks of our world at every layer of abstraction, from our inner experience that then leads to creating our social relations, which then leads to the way that we create ideas and culture, how this flow of information and ideas then goes back and gives us top-down causality and actually shapes who we are as people.
And that cycle of feedback and interconnect actually has some very interesting parallels to the way that we design software and technology platforms at scale today.
In many ways these platforms represent a set of values that product designers and engineers have. And when they set out to create these things, they go and deliver a product or a service that is hopefully useful, it’s fun and entertaining or informative. But in doing so, it shapes behavior. And I think that that’s a helpful way to think about how we can try to create technology, which really serves us well and collectively helps to move humanity forward in a positive way.
RH:
So what was the aha moment that said, “Actually in fact, technology is a path to greater humanism and possibly ways of really helping society”?
MS:
Yeah, it’s a great question. I mean, as much as I was interested in the structural side of philosophy, understanding the nature of the human and our social relations, I was also very much interested in moral philosophy. And as a committed, effective altruist, even at the time, I was always thinking, “How do I use the time that I have on earth to have the maximum positive beneficial impact?”
And that came with driving my motivation to drop out and start a charity, which I ran for a bunch of years. And I then went from there to work in local government, hoping that I could sort of scale up the influence and effectiveness that I was having with nonprofits.
And over time, I then quickly realized that, actually, the real thing that I wanted to do was around conflict resolution and figuring out how we can run these large-scale, multi-stakeholder change labs, which I was doing in 2005, 2006.
That led me to the climate negotiations in Copenhagen in 2009. We were convening a huge group of academics and researchers and nonprofits who were involved in one of the nine negotiating tracks, [in this case], reducing emissions from deforestation. And we were trying to align all these different people to get them to have a consistent negotiating position with the states.
And as I’m sure many people will remember, it was actually the first year that Obama was going to make a very big speech and hopefully make a big commitment. And unfortunately it was all very disappointing and no agreement was reached.
And I think in that moment, I basically realized how difficult it is for us to achieve consensus and deliver these large-scale agreements in the world in order to make progress on our tough social problems.
And the funny thing is that in parallel, I was sort of watching the rise of Facebook at the time. I think it was only a few years old and maybe in 2009, I think it had like a hundred million monthly active users.
And I was just totally blown away that a new technology company, a new platform that was maybe only three, four years old at that time, could have brought together a hundred million monthly active user and was shaping the way that we think and sort of influencing the way that we connect with one another and so on. And that was just profoundly inspiring to me.
And that was when I sort of realized that technology was really the most important thing that was going to happen in my lifetime. And I wanted to be right at the center of that. And so that’s how I sort of set off looking for some co-collaborators and co-founders for a new technology endeavor.
RH:
So say a little bit about that. Because one of the things that I find really amazing and fun about your journey is that you turned to an area – artificial intelligence – that was deep and prescient. And how did you go about your process of going, “Okay, technology is a way of saying the future can be shaped importantly for humanity?”
MS:
Yeah, I mean, like most people, I guess, I set about on a quest to find like-minded [people] who could teach me things and who I wanted to collaborate with.
And that led me to Demis Hassabis, my co-founder at DeepMind, and he introduced me to our third co-founder Shane Legg. They were both working on their PhDs and post-doctoral work at the Gatsby Computational Neuroscience Unit at UCL in London at the time. And they kindly invited me to come to some of their lunchtime seminars. And I ended up just spending a bunch of time with researchers who were working on what was called machine learning.
At the time, it was kind of taboo to say that you were working on AI, which seemed super far out and wacky. And we had just come through the sort of AI funding winter, where it was really difficult to get research funding for AI, but nevertheless, Shane to his credit, had spent his entire PhD working on a definition for intelligence. And he had looked at 65 different definitions from a wide range of different cultures and sectors for what it is that actually makes up intelligence. And he had aggregated these into a single formulation and turned it into an engineering problem.
And this was the kind of key thing that probably [led to] my first moment of optimism that this might be a tractable problem to work on. Shane, for his PhD, had articulated a way that we could actually (in a very, very, sort of engineering-focused approach) measure the progress that we were making towards systems that were more intelligent. And that felt very, very promising. Even though it was extremely nascent, it felt like a great place to start.
RH:
And, so this is 2010, if I recall.
MS:
Right, exactly. Yeah. 2010 in the summer. And very much at a time when most people were trying to work on very narrow applied problems for machine learning. And Shane was very much focused on, “How do we take the big theoretical question of defining intelligence and then operationalizing it?”
RH:
There had been a bunch of very good academic work, but the effort to actually make an effort to go very deep and compute and to be broad was not yet kind of the common technologist wisdom, which it is now.
It’s one of the most amazing technology efforts in, I think, the world, and definitely in Europe and London. When did you get your aha moment that, “We are going to build something new that the world hasn’t seen” ?
MS:
Well, I remember one of the first moments that really got me excited was when I saw us make progress with learning to identify numbers – handwritten digits from an image. And that sounds like a really simplistic problem, but back in 2010 and 2011, most of machine learning was characterized by what’s called handcrafted feature engineering. And so engineers would literally sit down and define the optimal shape and angle of lines and edges in order to be able to identify objects within images. And that handcrafting process is very brittle and it doesn’t scale well, and doesn’t generalize to new environments that your AI hasn’t seen before.
So this new wave of approaches was trying to train an AI system to learn its own representation of good edges and lines to better detect objects scenes. And in a very, very simplistic way, the team was trying to do this for digits.
And what I saw was a short video showing the learning process for how it was doing this classification. So it would go from a very blurred mushy, black and white representation to resolve quite a distinct, say a number seven, for example. And it looked pretty sharp. And I was like, “Wow, that’s really encouraging. That’s the first time an algorithm has learned its own representation of digits.”
And over time, a few years later, the team combined this with reinforcement learning to play the classic Atari games to superhuman level performance. And that in itself was like another incredible moment for me. It was pretty remarkable. I mean, I remember standing in the office watching the learning process for our DQN algorithm, our deep reinforcement learning algorithm play the game of Breakout, and many listeners will know this game, but you basically get a control, a paddle at the bottom of the screen, and there’s a ball that bounces up to knock the bricks out. And the more bricks that you knock down, the more score you get. And in this case, the DQN was given just the score and the raw pixels to try to learn a relationship between pixels and the control actions of moving the paddle left and right. And the amazing thing was that it had discovered this incredible strategy of really efficiently tunneling a route up to the back so that it could get behind the bricks and get the maximum score with kind of minimum effort.
And this was the first time I saw an example of a system that could learn its own representation of what was valuable and rewarding, and in many ways learn knowledge that wasn’t available to many other humans. Like many regular players, they didn’t discover the strategy. I certainly didn’t discover the strategy. And that was the holy grail to me. I was like, “Okay, we’re really onto something. This is an example of something that can learn new knowledge.”
And that’s obviously the real attraction of building these AI systems; that they could potentially learn new insights that could help us do great things in the world.
RH:
Yep. And obviously this kind of thing is counter to the classical stereotypes that machines can’t learn creativity and can’t learn new things; that they can teach us. And I think that naturally leads to the kind of epic moment by which I think DeepMind blazed [out] on the stage, which is the AlphaGo Lee Se-Dol moment. So please share that with us.
MS:
So, Go is played on a 19 by 19 board with black and white stones. And the objective is to try to surround your opponents stones with yours. And then you take them off the board and the rules are really as simple as that. There’s nothing else to it. But the complexity of the game is phenomenal. I mean, there are 10 to the power of 170 possible configurations of the board, possible state spaces. So the traditional methods of search through all the different options just don’t work because you don’t have the compute [power] for that.
So the algorithm really has to learn clever strategies to navigate that search space. And the way that we trained AlphaGo was that we first gave it 150,000 or so games from human experts. And we said, “Okay, learn from the corpus of the best possible experts we have.” And it was great. And then played reasonably well after that point. But the key insight was that we then basically spawned a whole series of instances of AlphaGo and we got it to play against itself.
In doing so, it was able to simulate millions and millions of new games, which obviously had never been played before, and therefore efficiently explore the space of all possible games. And then of course, we’ve set it loose by playing Lee Se-Dol the world champion of Go at the time in Korea, in this incredible live match over the course of five days and ultimately AlphaGo won.
The amazing thing there was that it learned some incredible new strategies that had never been discovered before. And that was the holy grail for me. I was like, “Okay, we really can train an algorithm to discover new knowledge, new insights. How could we apply that? How can we use that training method for real world problems?”
RH:
Yep. I completely agree. And this is, I think, one of the places where DeepMind has been one of the strong contributors to now what I think is a broad renaissance in the next generation of evolution.
One of the things that I really appreciated about your focus was How can it be applied? How can these applications of artificial intelligence be something new and important in the world? And that was one of the functions that you did as a co-founder of DeepMind. Could you go into the applied areas sense?
MS: