The more human-like artificial intelligence becomes, the more we understand how our brains actually work. Through that discovery process, researchers are identifying ways to design artificial intelligence in ways that factor in the safety and morality of their potential impact.
Greylock general partner Reid Hoffman interviews Dr. Fei-Fei Li, the co-director of Stanford’s Institute for Human-Centered Artificial Intelligence (HAI) and a professor of computer science, and Mira Murati, the CTO of OpenAI. In this interview, they discuss the recent advances in the field; the process by which technologists train sophisticated AI tools like GPT-3 and DALL-E with ethical considerations; and the need for comprehensive guardrails developed in collaboration between researchers, industry leaders, and policymakers.
“We’re trying to build these general systems that can think of the world in a similar way that humans do, so that they have a robust concept of the world,” says Murati, whose organization’s mission is to ensure AI is developed and deployed in ways that benefit all.
As artificial intelligence advances, that task has gotten more challenging. With AI’s enhanced capabilities come enhanced complexities, and researchers and entrepreneurs are constantly discovering and defining new safety problems to solve.
“Safety is one of those words like health: everybody wants it, but it’s really hard to define it,” says Dr. Li, who also spoke with Hoffman last year, shortly after HAI launched the Ethics and Society Review Board. “And AI is not one thing. Designing AI systems are really stages of work decisions, and we believe that at every stage of this AI development we need to infuse the ethics and human-centered values into this.”
This interview took place during Greylock’s Intelligent Future event, a daylong summit featuring experts and entrepreneurs working in artificial intelligence. You can watch the video of this interview on our YouTube channel here, and you can listen to the discussion at the link below or wherever you get your podcasts.
EPISODE TRANSCRIPT
Reid Hoffman:
While we’ll be covering a bunch of things on safety – which is a highly relevant thing, especially in this new foundation models universe, because both Fei-Fei and Mira are accomplished technologists massively beyond the scope of safety; both building amazing things and have been part of the historic contributions – I thought we’d start with a more broad question, which is: what’s currently most exciting you in AI? Fei-Fei, I’ll start with you, Mira I’ll go to you, and then we will dive into safety.
Fei-Fei Li:
Well, so many things are exciting me, but I guess I’ll just say something that’s on my mind right now. We are finishing the review process of a paper that is coming out, which is redefining a north star for robotics.
I feel like this moment working in robotic learning feels a bit like back in the days of image databases where we’re really imagining what can be done that would make robotics’ dream come true. And this particular paper that we (hopefully, fingers crossed, we’re in the rebuttal review stage), will lay out a benchmark of 1000 robotic tasks that are inspired by actual human activities, so it’s a scale we’ve never seen in robotic research anywhere.
RH:
Is it too much of a presage to say what one or two of the most interesting of those tasks are?
FFL:
Yeah. Sure. One or two. Actually, all of the 1,000 tasks come from the American Labor Bureau’s survey of American time usage, and their equivalent of European government agencies, where we look at what humans do in their daily activity, and we actually did quite a bit of study on what people want for robots to help. For example, most people do not want robots to open your Christmas gift, apparently we still want to do that ourselves, but almost everybody wants the robot to clean the toilet, so that’s ranked really high. Packing kids’ lunch is actually ranked fairly high.
RH:
Oh, interesting.
FFL:
We’re talking about real human activities instead of… You know how in robotics we tend to see toy examples?
RH:
Yes, exactly.
FFL:
This is going to be 1,000 real tasks.
RH:
Awesome. Mira?
Mira Murati:
What’s most exciting to me these days is seeing how far we can push the paradigm that’s driving all AI development in the field, which is this combination of large neural networks with a ton of data, and a vast amount of compute, and in the past few years we’ve seen this formula drive a ton of progress in AI research. We saw that with GPT-3, and then again with Codex, and the first generation of DALL-E.
So from OpenAI’s perspective, we’re trying to build these general systems that can think of the world in a similar way that humans do; so systems that have a robust concept of the world. If an AI system sees an avocado or reads the word “avocado,” the concept that gets triggered is exactly the same. And it’s exciting to see that we’ve got systems that have achieved some linguistic competence and understanding of visual concepts, and we’ll continue to push this paradigm ahead.
“We’re trying to build these general systems that can think of the world in a similar way that humans do; so systems that have a robust concept of the world.”
RH:
Let’s shift to safety because both of your organizations in different ways are some of the most important industry leaders on what we encounter with thinking about AI safety.
And so Mira, why don’t we start with you, just so we mix it up a little bit? And I’ll always go in this order, but will vary. Say a little bit about OpenAI’s approach, being this novel organization organized around a 501(c)(3), and organizations like this, what you’re trying to do to identify what safety is, and how to create those norms both in your own action, but also catalyzing other industries.
MM:
As you know, our mission is to build a general system and figure out how to deploy that beneficially to the world. And it’s just one word – beneficially – but figuring out how to robustly do that is actually an immense challenge. It’s hard to predict the future. It is hard to predict all the ways in which these systems might create harmful biases, or other risks that we can’t even imagine, but at the very least we can try to get as much understanding as possible, gather as much knowledge as possible, and leave the options open, and that’s OpenAI’s strategy. We are trying to deploy these systems continuously, but in a controlled way, that means an API. GPT-3 was first deployed through an API to a small group of users, and then eventually we broadened access as we understood how to get a good handle on the risks.
But I think it’s really hard, and that’s actually part of the reason why we decided to deploy GPT-3. Because if you are in the research lab, you may think this is going to be the most prominent risk, based on what we see; but when the model comes into conduct with the real world, that gets tested. And we found out several times that we’ve deployed that we were wrong, and the most prominent risk was something else.
For example, with GPT-3 we were convinced that misinformation was going to be the most important risk, and it is very important; but in practice, we saw that spam was actually a much bigger risk that we had to focus on, and the same thing happened with DALL-E as well. So I think it’s really important for these models to come in contact with real users, with the real world, and understand where the friction is, where the limitations are, and iteratively build-in mitigations. And the mitigations we’re building are probably not future-proof, but it’s a place to start, and it gives us enough knowledge of where to go.
But at the same time, we need to think about how the complexity increases as the models become more and more capable. For example, with language models now we oversee the output of the model by a human for sensitive use cases, and that’s not something that would scale with more powerful and advanced models, so then we have to come up with techniques to help humans evaluate the output of these models. And OpenAI has been working with other language model developers to coordinate on some of these standard practices to figure out how to deploy language models safely.
RH:
Fei-Fei, same question, but as opposed to OpenAI, Human-Centered Artificial Intelligence Institute, role of universities with the industry, governments, plus the work you guys are doing?
FFL:
Great question, Reid. Safety is one of those words like health: everybody wants it, but it’s really hard to define it. Wearing the hat of the Co-Director of Stanford Human-Centered AI Institute, which we call Stanford HAI, we think a lot about what we really want for AI, for future AI, and we come to the word “human centeredness.” We focus on infusing human-centeredness into every stage of AI research, development, education, policy work. Obviously, we’re a university so we don’t deploy products, but we hope that what we would inspire as applications will have an impact in productization downstream.
So AI is not one thing. Designing AI systems are really stages of work decisions, and we believe that at every stage of this AI development we need to infuse the ethics and human-centered values into this.
Simplest way to put it, how do we define a problem? For example, is your goal to replace humans without consideration of all the social implications, or augment human capability? Before you write a single line of code, you already are thinking about human values. The data, where does it come from? How do you collect it? How do you ensure data integrity? How do you annotate it? There are a whole bunch of… you know, from fairness, to privacy, to just a whole bunch of issues and considerations. Then the algorithm itself, is it safe? Is it secure? Is it biased? And then the decision-making using the algorithm, the inference, the human. Does it assist the humans or inform the humans?
Every stage of AI development needs human consideration, and Stanford HAI is really trying to embody that process, and it’s not even a side product. A central product of that process is the people we educate, the students, the undergrads. The graduate students, when they come out of this year of working or learning from Stanford HAI’s courses and lab work, they become technologists or business leaders or policy thinkers who understand the human centeredness of AI.
RH:
Let’s shift to safety because both of your organizations in different ways are some of the most important industry leaders on what we encounter with thinking about AI safety.
And so Mira, why don’t we start with you, just so we mix it up a little bit? And I’ll always go in this order, but will vary. Say a little bit about OpenAI’s approach, being this novel organization organized around a 501(c)(3), and organizations like this, what you’re trying to do to identify what safety is, and how to create those norms both in your own action, but also catalyzing other industries.
MM:
As you know, our mission is to build a general system and figure out how to deploy that beneficially to the world. And it’s just one word – beneficially – but figuring out how to robustly do that is actually an immense challenge. It’s hard to predict the future. It is hard to predict all the ways in which these systems might create harmful biases, or other risks that we can’t even imagine, but at the very least we can try to get as much understanding as possible, gather as much knowledge as possible, and leave the options open, and that’s OpenAI’s strategy. We are trying to deploy these systems continuously, but in a controlled way, that means an API. GPT-3 was first deployed through an API to a small group of users, and then eventually we broadened access as we understood how to get a good handle on the risks.
But I think it’s really hard, and that’s actually part of the reason why we decided to deploy GPT-3. Because if you are in the research lab, you may think this is going to be the most prominent risk, based on what we see; but when the model comes into conduct with the real world, that gets tested. And we found out several times that we’ve deployed that we were wrong, and the most prominent risk was something else.
For example, with GPT-3 we were convinced that misinformation was going to be the most important risk, and it is very important; but in practice, we saw that spam was actually a much bigger risk that we had to focus on, and the same thing happened with DALL-E as well. So I think it’s really important for these models to come in contact with real users, with the real world, and understand where the friction is, where the limitations are, and iteratively build-in mitigations. And the mitigations we’re building are probably not future-proof, but it’s a place to start, and it gives us enough knowledge of where to go.
But at the same time, we need to think about how the complexity increases as the models become more and more capable. For example, with language models now we oversee the output of the model by a human for sensitive use cases, and that’s not something that would scale with more powerful and advanced models, so then we have to come up with techniques to help humans evaluate the output of these models. And OpenAI has been working with other language model developers to coordinate on some of these standard practices to figure out how to deploy language models safely.
RH:
Fei-Fei, same question, but as opposed to OpenAI, Human-Centered Artificial Intelligence Institute, role of universities with the industry, governments, plus the work you guys are doing?
FFL:
Great question, Reid. Safety is one of those words like health: everybody wants it, but it’s really hard to define it. Wearing the hat of the Co-Director of Stanford Human-Centered AI Institute, which we call Stanford HAI, we think a lot about what we really want for AI, for future AI, and we come to the word “human centeredness.” We focus on infusing human-centeredness into every stage of AI research, development, education, policy work. Obviously, we’re a university so we don’t deploy products, but we hope that what we would inspire as applications will have an impact in productization downstream.
So AI is not one thing. Designing AI systems are really stages of work decisions, and we believe that at every stage of this AI development we need to infuse the ethics and human-centered values into this.
Simplest way to put it, how do we define a problem? For example, is your goal to replace humans without consideration of all the social implications, or augment human capability? Before you write a single line of code, you already are thinking about human values. The data, where does it come from? How do you collect it? How do you ensure data integrity? How do you annotate it? There are a whole bunch of… you know, from fairness, to privacy, to just a whole bunch of issues and considerations. Then the algorithm itself, is it safe? Is it secure? Is it biased? And then the decision-making using the algorithm, the inference, the human. Does it assist the humans or inform the humans?
“Safety is one of those words like health: everybody wants it, but it’s really hard to define it.”
RH:
Let’s now dive a little deeper in each specific organization. For DALL-E and GPT-3, the front-end of the API is in part to understand what the possible risk flows look like, so that you can begin to train those. How do you take that information and iterate to a beyond human safety model? How do you take that and also help lead the way in thinking about this safety within the industry?
MM:
For GPT-3, for example, initially we opened up access to use cases that we felt we had the right mitigations in place, so that means that almost like on the first few days, the allowed use cases were around search, as well as classification, but we were not quite comfortable with open-ended generation. And so we worked with industry experts from different domains, as well as other researchers, to red-team the model a bit further. We worked with other trusted users as well to understand possible ways in which the model would fail the expectations of the user, and there are many. From there, we try to build-in mitigations both from the model perspective, but also the tools that come after deployment.
From the model perspective, one of the central things with GPT-3, but also other generative language models, is the fact that the model will make-up stuff, and it will not admit when it doesn’t have expertise in a specific topic or when it doesn’t know the answer, so obviously that’s a problem. And in other ways, it would also mislead you with the answer.
So we wanted to figure out, “How do we make the model more robust, more reliable?” and we used the feedback that we got from our users on the API, and we used reinforcement learning with human feedback to make the InstructGPT models, which is a series of models that are far more reliable, and do the thing that the operator actually wants them to do. So not only are they more helpful and safe, but they’re actually useful.
And so there are default models in the API today, and our users prefer them rather than the base model. This is one way where we use deployment to actually make the models safer, more reliable, and more effective as well. And it is very interesting because it’s the first time that safety moves from the theoretical domain into the practical domain, and it merges with capabilities; and by doing that, it forces a standard in the industry because not only it’s safe, but it’s actually more effective and more useful.
RH:
And then similarly, one of the things where Stanford HAI has pioneered some things, and specifically I’m gesturing at the Ethics Review Board, but go anywhere you like with the question.
FFL:
Just to follow up, it’s actually been a very encouraging trend to see that companies are really thinking about ethics and the safe ways of developing and releasing products. In the meantime, in academia we are also thinking, for example, a typical way of putting some guardrails in our research activity is traditionally if the research involves human subjects we have IRB review boards. This has been critical across academia in U.S. universities, and also international universities, that a lot of medical research and human subject related research gets reviewed by a board of expert faculty, and so on. But when AI is starting to become a really big chunk of university research, we don’t have an equivalent of an IRB. At the beginning, it was okay, it’s theoretical, but very quickly we realized, “Wait a minute. There’s so many examples of AI research.”
For example, face recognition. We see globally the biases and the harms of bias. So how do we put guardrails in the design of AI research, and to ensure even the most technical researchers and students have in mind the ethics and the social impacts of their work?
At HAI, one of our functions is to facilitate multidisciplinary research. And Reid, you have been a board member of our institute so we talk a lot about that. It’s really important. We encourage researchers to do interdisciplinary research, but we need a new way, a new IRB to guide our researchers to think about ethical and social implications. So HAI was, as far as we know, the first organization, university organization in the country, that formed what we call an Ethics and Society Review board, ESR, that reviews all grant applications; and as we assign grants to our researchers, we ensure that the grants work with the ESR board to articulate the understanding and potentially mitigation solutions of the social and ethical issues of the research. That’s one example of how much we put that emphasis in.
Another example is our work with the policy makers. We work directly with regulators, policy makers, and frankly just civil society, different people, multi stakeholders, to hear and to communicate the implications of AI, to hear their thoughts, to facilitate dialogues. We particularly focus on bringing industry and civil rights organizations, federal and state and local governments and policy makers to the same table, create a forum, a safe neutral forum to have these conversations because we all care about innovation and guardrails, and we need this dialogue.
RH:
And add a little bit about the National Research Cloud too because it’s part of enabling universities, and other entities like this, in order to be able to fully participate.
FFL:
One of the things that we recognize is magic. One of the magic sauces of America in the past century is our ability as a country, as a people, to innovate. We have an incredible ecosystem, thanks to our entrepreneurial world, our higher education, the labs, the researchers; and also the role the federal government has played in incentivizing research, whether it’s NASA or DARPA or NSF, and all that.
But as AI took off in the past decade, what we observe is that the resources are quickly concentrated in a few companies with compute and data and talent, the three critical resources for doing incredible AI innovation, which was great. We see incredible things coming out of OpenAI Microsoft, coming out of Google DeepMind, coming out of Facebook, but that’s not enough to lift the entire nation, to educate more AI talent to stand up in the face of global competition. We need to ensure this ecosystem remains healthy so that it’s not just concentrated in industry per se.
And right now under the Biden administration, the White House and the Congress have mandated a task force study group, a 12-person taskforce to figure out what this National Research Cloud would look like. I’m actually one of the members of this taskforce, along with other people from industry, academia and government, and hopefully very soon we’ll come up with our report and push this bill forward to establish this national resource.
“We need to ensure this ecosystem remains healthy so that it’s not just concentrated in industry per se.”
RH:
Mira, I’m going to ask you… I have obviously a thousand questions for both of you; and depending on what the questions are from the audience, I may ask more. But I’m going to ask you the last question before I turn to the audience.
One of the things that GPT-3 and DALL-E have both done is shown a path for really amplifying human creativity. It isn’t just the downsides that is part of the question around safety, and so forth, but also in a sense helping us be more human, be more creative. Say a little bit about the lessons from DALL-E and GPT-3 and what the potential is for this human amplification.
MM:
Yeah, exactly. From GPT-3, early on we were surprised to see the ability of the model to generate creative and even touching poetry. So one of the prompts that we gave GPT-3 was to generate a poem in the style of Pablo Neruda that talked about Maxwell’s equations, and we were quite surprised that it had this ability to pick up on the elements of Maxwell’s foundational electromagnetic equations, but also do that in the style of the love poems of Pablo Neruda, so that was really beautiful.
And a lot of people were playing with poetry, and the creative side of GPT-3, and I think we saw that even more with DALL-E. DALL-E maybe more so because it’s images, and also the form factor in which we made it available through DALL-E labs, everyone was just having so much fun with it. Even at OpenAI we would spend hours just generating DALL-E images. This really just shows how technologies like DALL-E can democratize high-quality creation of images and ideas, and can push them so much further.
And often we get asked, “Well, does this in some way dilute the human creation, the original human creation? And what happens in the future?” There is this almost instinctive human reaction to protect our own original creations. And I think that if we look back in history, it’s actually not so different from what happened in 16th and 17th century where there weren’t that many people that could afford paintings, and so things were quite binary, you were either Rembrandt or nothing, and there wasn’t so much of a nuance appreciation. It was either great painting or not.
And so I think as we get tools like DALL-E or GPT-3, maybe there’s going to be a more nuanced appreciation for this co-creation, and a different appreciation for the original human creations. For example, today you can have a skilled artist that went to art school that can try to create a replica of The Night Watch; and to my undiscerning eye, it will look great, but we still value a real Rembrandt differently.
But you can say, “Okay, this is an elitist point of view. What about the broader impact?” and I think that’s actually not so different from the effects of globalization because it’s really an exchange of ideas, a cultural exchange. And of course, there are unwanted and undesired effects of globalization, but overall it does create more diversity and it does create more prosperity. And we’ve seen this in history with Western Europe, the 19th century being one of the most fertile and diverse from liberalization of cultural exchange; and by contrast, after the collapse of the Roman Empire it was quite the opposite.
And so there is actually an interesting book about this called Creative Distraction that talks about this perceived fear that human talent will be diluted in a way. But actually, if you look at it long term and the global effect, the global effect is one of diversification, one where we end up with more ideas in total, more prosperity, and we will continue to develop more information and create artistically, scientifically, and also in a social context.
“I think as we get tools like DALL-E or GPT-3, maybe there’s going to be a more nuanced appreciation for this co-creation, and a different appreciation for the original human creations.”
RH:
I think we have time for one question from the audience. I did also ask that last one because if you haven’t had a chance to play with DALL-E and do your own printout, it’s one of the things that OpenAII helped provision us with, so it’s a fun experiment.
Audience Member:
Hello, just to the last point that you alluded to: first, it’s very heartening the thoughtfulness and thoroughness of preempting and mitigating potential impact of launching some of these technologies.
And I’m wondering, some of the effects cannot be simulated until you put that out there in the real world and probably at scale, so how do you think about the risks and issues at large-scale societal effects, and potential second-order effects as well? And what do you think is the role that we as technology companies can play in that? And probably related to that is… Fei-Fei, you mentioned governing boards as an oversight, and I’m wondering how you address the tension between the process introduction and the pace of innovation? We were colleagues in CloudAI, if you remember. We’ve had oversight boards internally, and things get slightly slowed down as a result of… So would love to hear your thoughts, how you think about this.
RH:
Mira, why don’t you start?
MM:
You’re right. With deployment, we’re just scratching the surface, and we’re putting these models out there, seeing what people are doing with them, seeing the risks, and building tools to mitigate them; but more importantly, bringing the feedback back to the model development stage, to build models that are more robust, and I think the alignment technique is really core to that.
But more broadly, it’s very important that we collectively are thinking about the governance systems of AI because obviously the developers of these models need to be a diverse group of people, but even that is not enough, and we have to think about AI like electricity. And so we have to think about the deployment and the governance of the systems more broadly in a global setting, and bring in input from all different fields.
FFL:
Yeah. And also, great questions about innovation versus regulation or guardrails. I think we all get that question and it’s always wonderful to think about it.
First of all, I don’t believe they’re mutually exclusive. They’re at odds with each other. I do believe, as everything else in life, we have to strike the right balance. If we go to the extreme route that there should be zero guardrails, innovation might come out to fail us all, so it’s not necessarily a world we want to live in, and of course the other side of the extreme is also not good.
In the case of AI, I think it’s time for every organization, from individual researchers all the way to the government, to have that dialogue, to think about how this can be done. If it’s done right, actually good guardrails can encourage good innovation.
I’ll give you one example of smart camera machine learning work in the healthcare environment, which is part of my lab’s work where we actually talk about safety. We actually use cameras to help doctors to monitor the safety of patients. Now the intention is all great, but immediately we get into privacy issues, we get into “bossware” issues. Who are you watching? What are you watching? This comes back to us as an important social human feedback to push us to innovate more.
My team of students, who are used to just doing deep-learning algorithms, are now using differential privacy algorithms to push for privacy protected machine learning. And then suddenly they realize differential privacy algorithms are too slow; it’s never used in videos where we’re using cameras. Suddenly, that guardrail push, the regulatory push, is actually incentivized innovation.
We’re publishing papers that push differential privacy machine ML algorithms to deal with the large-scale data. This is a perfect example of, when we care about these guardrails, they come back to motivate us to innovate better technology. And I think there’s plenty of examples in AI that we can do that, and that everybody’s better off if we do that.
RH:
And with that, you can see why both Mira and Fei-Fei are people that I also constantly learn from, and so let’s give them a hand.
MM:
Thank you.
FFL:
Thank you. Thank you guys.
