Podcast: SciCafe—Teaming Up with Robots with Julie Shah

NARRATOR: This is [email protected], the podcast of the American Museum of Natural History.

Imagine a robot that knows without being told which tool to hand to an autoworker, or how to match hospital patients with the most appropriate medical staff. This next generation of robotics is right around the corner. Scientists are already using artificial intelligence to create robots capable of learning tasks on their own rather than simply follow their programming. Julie Shah of MIT explores how new technology can help humans and robots work side by side naturally, efficiently, and safely.  

JULIE SHAH (Assistant Professor in the Department of Aeronautics and Astronautics, MIT): I’m Julie Shah. I lead the Interactive Robotic Lab at MIT and our research is at the intersection of artificial intelligence and cognitive science. And we’re really working to reverse engineer the human mind to make robots that are better teammates with people.

You know, to a large extent, robots work separately from people today. Our goal is to do more than just make robots that are safe enough to work with us, it’s to make robots that are smart enough to work with us as easily and naturally as we work with other people so that we can ultimately harness the relative strengths of people and machines working together. So, my aim in this talk is to be able to convince you what an incredible world we have ahead of us as our machines become more intelligent to be able to augment what we do.

So, I’ve spent much of my year working in robotics and manufacturing, in factories for building planes and building cars, and in new sectors, as well as many other sectors, we’re actually still quite limited in how we use robots today. So, I’m showing you one picture here from Amazon Robotics. This is a picture from one of their warehouses. When you order products online from Amazon, there’s actually a fleet of robots that are delivering those products to you. But the thing to note is that the warehouses are full of robots, but there’s people that line the edges of this warehouse. The robots bring your product to a person who then boxes it and ships it to you. So people and robots are working together in teams, but they’re working in space that’s physical separate.

Similarly, on automotive assembly lines, we have robots that are working near people. So this is a universal robot. It’s what we call an inherently safe robot. Which means it can work right next to a person and, if it bumps into you or hits you, it won’t permanently harm you. Which is good, that’s key, right? So, effective collaboration.

But ultimately the system is really working next to you, it’s not working interdependently. These are systems that co-exist with us. They don’t truly collaborate.

Now, we think about the automotive industry in particular as an industry that’s been incredibly successful at introducing industrial robots. But half the build process of a car, half the factory footprint and half the build schedule is actually still done by people. And it’s not that much more of that work can’t be done by robots. Some of the work is truly too difficult for robots today. It’s very dexterous work. There’s actually a fair amount of cleverness and judgement and innovating in how to assemble the car.

But there are little pieces of almost every aspect of this manual work that can be done by a robot today. Our problem is an integration problem. We can’t take that works that are physically separated from the manual work and have it still make sense for the flow of the factory.

So a robot that can more intricately collaborate with a person, that can almost like dance with a person, to offer just the right instrument at just the right time, can substantially improve the productivity of the line.

Now, the question is, how do we enable a robot to work that seamlessly, that fluidly with a person? And it’s an interesting question because it makes you take a step back and say, well, how is it that people work so effectively together? If you think about sports teams, if we think about nurses and surgeons in the operating room—really, in every setting in our lives, we are able to collaborate with other people because we have the ability to do three things.

For me to work with you, I need to be able to know what you’re thinking. I need to be able to anticipate what you’re going to do next. And then I need to be able to make fast adjustments when things don’t go according to plan.

And it’s this ability that allows a surgeon to put her hand out in the operating and have a surgical assistant put the right instrument there without even a glance, without a word, without a command.

So, our lab has worked for many years on enabling this type of human-machine and human-robot collaboration. We’ve worked on developing algorithms and models that enable robots to infer our cognitive state, infer our human mental state, use that information to be able to anticipate what we’ll do next, and then we developed algorithms for scheduling and planning. So, the robot can use its predictions of what we’ll need and what we do to very quickly adjust its own plan to provide just the right materials at just the right time.

So, realizing this, how do you break this down? Well, we need a robot that can infer what we’re thinking, anticipate what we’ll do next, and execute. Now, in order to make this real, we need a robot that does three things and we need this robot to do three things in sequence. I call those three sequential system capabilities.

To work together as a team, we need the person and the robot, or the multiple people and the multiple robots, to come together and have almost a conversation, have a negotiation about how it is they’re going to work together, who’s going to do what, when and how. We need to form some shared understanding of how it is we’re going to collaborate.

Now, given that, you think about an emergency response team. They come into a conference room, they plan their deployment. Is that the end? Does the team work effectively together? No. And the reason is that it’s unreasonable to ask any team to plan all of the details of how they’re going to work together, all of the contingencies and all of the potential responses in advance. So, beyond being able to plan together, we then need the robot to be able to work with us, to observe us and interact with us, to learn how it refines its plan, how it adapts and modifies its plan based on all of the situations that could arise.

And, finally, once the robot understands how it is we work together flexibly based on all the situations that can unfold, again, we need the robot to be able to use that information to physically work with us—to anticipate and provide the right information or the right materials at the right time.

So, today I’m going to highlight some of our recent works, specifically in systems two and three, and refine and execute. So, diving a little deeper into “refine,” how is it that we can develop this ability to somehow know how it is we trade work, how it is we’re going to respond to any potential circumstance that can arise in the future. So this is a puzzle in and of itself. And we spent a few years in my lab working with a local Boston hospital and studying the work of nurses and doctors on a labor and delivery unit.

And the reason this is an interesting setting—well, there’s a few reasons. One is that this is a very complex workflow. In the labor and delivery unit, you never know who’s going to come through the door, what’s going to happen, how the future’s going to unfold. You need to be able to deal with unpredicted situations.

But beyond that, we have nurses and doctors in training that are actually relatively quickly able to learn the flow of the hospital, to be able to lend support and be at the right place at the right time. And nurses and doctors in training, they do not require millions upon millions upon millions of training examples to figure it out, right? Sort of a week, a few weeks in the hospital and they understand how to be a supportive teammate.

It’s also an interesting environment because we can’t create a simulator for it. We can’t give an AI the simulated world where it can practice and see, if it takes an action, this is what will unfold. All we really have, just as much as the nurses and doctors in training, is the ability to observe what happens in a day in the life over and over and be able to try to learn efficiently from it the way a human team member does.

So, this nurse manager here does a very interesting job. So, the reason that we study this role is because she’s essentially doing the job, this one person, the job of an air traffic controller, on the hospital floor. This one person is deciding which patients go to which rooms, which nurses are assigned to which patients. They control aspects of the O.R. schedule and many other decisions. And the way they learn to do it is not a codified training process. So, it’s very hard to reverse engineer how and when a nurse will make a particular decision, using the information we have today.

So, the question was can we deploy machine learning algorithms to be able to watch a nurse, nurse’s decision-making process, and be able to predict the decision a nurse might make under some set of circumstances. And can that AI or that robot system them potentially offload some of the work of that nurse?

Why is this so important? Well, if anybody’s been in the hospital lately, you may have seen these types of robot systems wandering the corridors. Has anybody seen these robots? No? No? Well, okay. So, that’s a good point. Why haven’t you seen these robots? They’ve been around for 10, 15 years. There’s actually relatively few of them. Okay.

Here is why they are not well-adopted. Now, this nurse here has maybe 10 to 15 direct reports, right? Of human beings. And now, these machines come in and they’re meant to deliver medications, meant to deliver linens. But how do they know where to go and when to be there? Someone has to tell them. And that schedule of their work is very dynamic, depending on what’s coming through the door. So it is just really unreasonable to ask this, essentially an air traffic controller, to now also task and schedule a large fleet of robots. That’s not ultimately very helpful to them.

But a robot that’s able to learn like a human apprentice and even just suggest the right next step for it can significantly reduce the cognitive load for that nurse and can ultimately make these systems a viable solution.

So, how do we make a robot or a machine learn how to make some of these decisions that the nurse makes? Well, we work to understand how it is that humans learn so efficiently in these settings. And one of the things we know is that people learn very well through examples. And we also know that one of the foundations of human multi-criteria decision-making is our ability to learn through paired comparison.

So, that nurse-in-training or that doctor-in-training, as they’re watching what unfolds in the hospital, they’re constantly thinking, ‘Okay, this is what happened in this situation.’ You’re learning this way every day. You’re thinking, ‘This is what happened in this situation. What’s different about that other situation, that other day? What was different that would result in one person making a decision A here and a decision B there?’ We implicitly compute these paired comparisons and we use them to learn more efficiently.

So what we did was take this insight and provide this type of foundation as what we call a scaffolding for the machine-learning algorithm. We gave it the base of how we know humans learn so efficiently. And ultimately we were able to show that, with just even a few dozen demonstrations of a day in the life of the hospital, the machine or the robot was able to make suggestions for what to do next that nurses and doctors agreed with up to 90% of the time. That is a huge success and gets us a step forward to deploying these systems to help us day-in and day-out.

So, I’m going to show you some videos of our robot in the hospital now. This is a robot—it’s not a service robot. This is a research robot. But what this robot was doing was reading the current state of the labor and delivery floor, you know, the status of patients and then making a suggestion to the nurse or doctor.

[Video begins]

NURSE: What is a good decision?

ROBOT: I recommend placing a scheduled Caesarian section patient in room 5. Nurse Meredith can take care of her.

[Video ends]

SHAH: You can see the nurse responds positively.

So we deployed these systems, both in controlled environments, controlled experiment environments, in which we created simulations of labor and delivery floor. And we also deployed the robot in the live labor and delivery floor to show proof of concept that this system could be helpful in the real environment.

[Video begins]

NURSE: What is a good decision?

ROBOT: I recommend placing a new patient in triage room five.

NURSE: What is a bad decision?

ROBOT: A bad decision would be to place a scheduled Caesarian section patient in room 14 and have Nurse Kristen take care of her.

[Video ends]

SHAH: So, ultimately, there’s no ground truth, right? But we’re really encouraged that, with really relatively few training examples for this type of machine-learning algorithm, the system is able to provide recommendations that the nurses and doctors agree with 90% of the time.

Now, the goal of the system in this safety-critical scenario is never to replace the human judgment and decision-making that goes behind the years of training of the nurses and physicians working in these environments. But these nurses are doing the job of an air traffic controller without any decision support, without any aid that a typical air traffic controller would have. So the ability to offload even some of their cognitive capacity through this additional support frees them up to put more of that effort into making the decisions where we really need their judgment and enhance our safety, all of our safety and well-being.

Okay, so that’s a little bit about how we designed robots to be able to take some skeleton of how to work with us and then efficiently learn from observing us and interacting with us in real, difficult, messy environments. Next, we want to be able to deploy robots that can use that information to physically help us. So this is the execute system.

And much of this work we’ve done in assembly manufacturing. And this should be the easiest of all possible scenarios. We have task procedures, we know what’s being built, we put the employees in the factory to build the parts. But it’s quite tricky, actually, because space is tight and a robot that even just makes the wrong maneuver at the wrong time disrupts that line. That line slows, that line stops. And that’s big money, ultimately. So it’s not enough to know an abstract—here are the three steps that will happen in the future. We need fine-grained predictions. We need to know exactly where a person is going to be in space and time. We need to know exactly when an activity starts, we need to know exactly when an activity ends. And that’s the core of the challenge that we address.

Now, I’m showing you here an associate in an automotive test factory with one of our industry collaborators. You can see there’s multiple possible paths that person may take through space in doing their job and there’s a robot here. It’s a fairly restricted robot, a robot on a rail. And what we need to do is predict where in space and time that person will be. And what we discovered in our work was that a machine-learning algorithm can actually predict where a person will walk or where a person will reach on a table bizarrely well.

So, what a machine-learning algorithm can do is predict two steps in advance whether you’ll turn left or right. And it does this by tracking your medial lateral velocity and your head turn. Now, I spent a lot of time walking through the corridors of MIT wondering if I could tell two steps in advance whether someone would turn left or right and I’m not sure. But we can do it here.

Similarly, we have the ability to track the biomechanical model of a human arm and, with about 300 or 400 milliseconds of motion, so just about this much motion, predict with 75% accuracy where a person will reach on a table within four quadrants. That is very powerful. Those are very early predictions that a robot can use to maneuver around us or to synchronize and help us.

Now, the challenge is that in some cases you need to track medial lateral velocity and head turn. In some cases you need to track the hands, the feet. In some cases you need to track objects in the environment. And it takes a machine-learning Ph.D. to tailor your system for every new situation. And if your product changes a little bit, you have to tailor it again.

So we worked in the lab to develop data-driven approaches to be able to automatically stitch the most appropriate predictors together. So we take the machine-learning expert out of it. We feed the system data of people performing their tasks. And it weights the various classifiers and stitches them together in time to develop a very, very accurate prediction in space and time of where a person will be. So you can imagine that this is useful in a number of scenarios. We’ve deployed it in factories.

So I want to show you first a video from this scenario in the Automotive Tech Factory. And I’m going to show you the current state of the art and then what the system looks like when it uses our technique for prediction and planning.

[Video shown]

[Video ends 18:55]

So the robot didn’t need to be scripted to move out of the way of the person. By integrating where the person would be in space and time and enabling the robot to plan in space and time around the person, we see these behaviors where the robot makes way for the person and sometimes you’ll see behaviors where the robot will quickly try to scoot through and get to what it’s doing. But it’s very human-like. And these are applications where every second matters and I’ll come back to that at the end of the talk.

Now, predicting in space and time where you’ll be is useful. Predicting at a high level what’ll happen in the future is useful. But, ultimately, to work in real time with a person, we need to do something even more. We need to be able to understand in the moment what activity the person is doing. What is the meaningful activity that they’re doing. We need to know exactly when it starts and exactly when it ends. And only with that information can we really use our information to predict what will happen a few steps down the road.

And so we take a similar approach. We developed data-driven methods for providing information about the task. We take the machine-learning expert out of it. And the system learns over time what are the key features to track. What parts to track in building the car? Does it track the hands? Does it track the head of the person? And we apply a similar approach to being able to monitor in a very fine-grained way when activities start and end.

So, this can now be used for deploying [a robot], but it’s truly collaborative. It does more than avoid us but can provide the right material at the right time. So you’ll see on the top here, we have a timeline of the various activities involved in assembling a part of a dashboard. The person here, the human, is going to work with this robot right next to the person; it’s a mobile robot. The dashboard is here, circled in orange. And there’s a meter to install into that dashboard from one of those blue boxes and there’s a NAV unit to install into that dashboard from one of the other blue boxes.

And the way this would work today is the associate would walk back and forth from those blue boxes to pick up the next piece and install it. And so you’ll see the robot using its online activity recognition system and prediction system to offer the right materials at the right time.

You can see the timeline up top. The robot’s identified that the person moved to the meter. It’s collecting the meter for the person. The robot then needs to identify where the person is, generate a handover to that person. And that handover process is very subtle. There’s actually an enormous amount of study in the robotics community in how you design a robot to recognize the subtle signals of when to let go to avoid dropping a part.

And so the robot predicts the piece, the instrument that the person will require next. Goes to retrieve it from the other bin. And offers that to the person.

Okay. Our experiments both in this setting, in the real setting, and in experiments that we do with elbow-to-elbow collaboration between people and industrial robots like you see in this picture. Bear out that a robot that can do these small things, of anticipating where you’ll be, what you need and being able to use that information to re-plan can reduce the amount of time it takes to perform the task, can increase concurrent motion between the person and the robot, it can reduce human idle time and it can reduce robot idle time.

And there’s a sense in safety critical domains that you often have to trade something for efficiency. And, in this case, you may need to trade safety for efficiency. The robot needs to move faster around you, it needs to move closer to you. But we gain all of those benefits in efficiency or productivity with an increased average separation distance between the person and the robot. So we’re not trading safety for efficiency. By making a robot that’s smarter, that understands what’s better, we can achieve the best of both worlds—a more efficient collaboration and improved safety.

Our studies show that, based on these measures across these domains, it’s possible that we could, say in an automotive factory, save something like 3 minutes out of every hour in building a car. Okay, what does that mean? That’s $80K, those three minutes. So what does that mean over a two-day shift, or a two-shift day? That’s about a million dollars. What does that mean over the course of a month? $30 million dollars. That’s big money for basically saving seconds here and there. But this is the promise of a robot that can collaborate with us seamlessly.

Now, if you were looking at the trivia at the beginning of the opening, does anybody remember how many industrial robots are in use worldwide today? Go for it—1.8 million, nicely done. Okay, how many robots are in our homes, domestic and service robots? Thirty million. Okay, that’s not including the robots that are in our office complexes, in our work environments. It’s not included connected systems like the Alexa, Google Home. We have enormous potential for these systems to understand what’s better and optimize our lives the way they can optimize factories. And this is what we’re working on in the lab. The days in which robots are separated from us, behind cages, are over. And we’re working towards a future in which we all want to hug our potentially dangerous industrial robot. They make us the most effective that we can possibly be, as well.

And with that, I can wrap up and take any questions. Thank you.

[APPLAUSE]

QUESTION: So, I don’t have my fingers on it right now, but I did read somewhere in  your research that you’re working on systems where you might have a robot decide whether or not to give information or choose to not give information to the human counterpart at a particular time for a particular reason. Which remind me when HAL 9000 said, “I think this conversation can serve no further purpose, Dave.”

Could you give us an example where the robot would choose not to give the human information?

SHAH: Absolutely. So, today I talked mostly about physical collaboration, right? By being able to predict what someone will do. There’s also applications in figuring out what information you provide someone at the right time to influence their behavior, for example.

So, my background is aerospace. I work in safety critical domains, manufacturing, healthcare. And we study teamwork. One of the things we study is pilots’ communication in collaboration in the cockpit. And there’s decades of research in trying to understand how do we train pilots to work together under this sort of stressful environment—especially takeoff and landing, it’s high workload. And what is it that makes someone an effective teammate in this situation?

And it turns out that the most effective teams are very selective in the information they share. They don’t share everything about the current status of the task. They selectively share the information that they anticipate their partner will need. Okay.

So, we can apply the same ideas of predicting what someone will do and what they’ll need to decide whether or not the information we share is likely to influence their behavior and, if it will influence their behavior, how will it influence their behavior.

This is critically important for us to get right in human-machine teams because when we design just multi-robot or multi-agent teams to communicate, this doesn’t matter. They actually share bundles and bundles, thousands upon thousands of messages to each other to maintain their common knowledge of the world, to be able to coordinate in a maximally effective manner. But people can’t process and use that information. So if we’re going to bring robots into human teams or we’re going to embed a person into a team of robots, the robots needs to be able to communicate with us in ways that we can use and understand and a key part of that is not sharing everything but having the wisdom and judgment to share just the right information at the right time.

QUESTION: Thank you so much, this is really amazing work. To sort of talk about the elephant in the room, which is job loss, there are huge, huge estimates—I’ve seen numbers as high as 20% globally of jobs that could be lost and there’s not always great success in terms of retraining people. So how do you address people who have substantial concerns about the future for work and income?

SHAH: Absolutely. This is one of the key concerns of our time, right? And to a large extent, I share those concerns. But one of the reasons that I wanted to do this talk and I come out to do this is to try to communicate that this technology is in our hands. We can design a robot to supplant or replace human work. Or we can change the design problem and design it to understand it to understand and augment what we do. And if you choose one path or the other, you end up with a very different robot and a very different machine or system. And this is ultimately our choice, is to understand what is our true human value and then augment those aspects of what we do.

Now, there are things that machines and computers and robots are good at and things that they’re not good at. Our incredibly human capability is to structure an unstructured problem. Once we have that, the machine can take that structured problem and it can compute and it can crush it. But we have an ability machines do not have. That resource nurse? She or he is solving every day an unstructured problem. In fact, if you try to formulate the optimization problem for what that person’s doing—first of all, nobody knows what the objective function is.

And we look at the same problem for missile defense. Military experts. Even when we know the objective function—you know, avoid people being killed. Avoid your systems being harmed. It takes maybe a half-hour for a computer to compute how to deploy anti-missile systems to protect the ship. The military experts, the best of them, can do it in less than a second and they can outperform the machine that takes a half-hour to compute the answer. Our human intuition and judgment is an incredibly powerful thing and we don’t understand it fully, we don’t understand it well.

How can we enable these machines to use it fully, to better codify and analyze the strategies that humans use together, through humans and machines, increase our joint performance? This is our choice, to design technology in this way, versus designing systems that supplant human work.

MODERATOR: I’ll take a question from Twitter. Back when you were showing the nurse aide robots, Kevin Marshall is asking, were they receiving visual cues from the nurses’ display or the LNB’s data being sent directly to them?

SHAH: Ah, this is a great question. So, in much of our work, we collected data in a high-fidelity simulation environment in which full information about the state of the world was coded in this computer simulation environment.

But, in the live deployment of the robot, we were not able to pull electronically the feed of the current state of the labor floor. And in fact doctors and nurses use a white board to maintain their understanding of the current state of the labor floor. So that robot actually did point a camera at the handwritten white board and read the current state of the labor floor in order to make its prediction.

And that’s part of what makes it a robotics challenge for us. I don’t spend any time talking about it, but it was enormously difficult to get that robot to read doctor and nurse handwriting from a white board.

QUESTION: Thanks. During your talk you used the word “robot,” “machine,” “AI” and you’re kind of commenting on automation. I was wondering if the distinctions are important and maybe just what is a robot in your mind?

SHAH: That is a great question. Even among the robotics community there’s great debate of what is a robot and what is not a robot. There are many standard definitions of what makes a robot. One of the widely accepted definitions is a system that has four degrees of freedom or more. So, kind of four joints or more. It kind of actuates and acts in the world. Autonomous cars are considered generally robots.

In our work, I’m very interested in the physical aspect of a system helping us, a system understanding physically what we’re doing in the world. But in order to do that, we need a system to understand cognitive state or mental state. The two can’t be disentangled. We need to understand both physically what we’re doing and how we’re thinking about it, how we make decisions.

And that cognitive side is equally useful for augmented intelligence systems, systems that can provide the right information, the right decision support. And so our work has applications across, which is why I often talk about machines, I talk about agents, I talk about robots.

QUESTION: Hi, I had a question, at the beginning of the presentation you said that robots and humans co-exist as opposed to collaborate with each other. When I saw that, I thought those two things were the same, so how are they different?

SHAH: So, we have robots that can work next to us, largely. They’re safe enough to work next to us. But they’re not smart enough to work interdependently, to be able to chose its next action based on what it’s observing we’re doing or to be able to physically collaborate in assembling a structure, where the work is sequenced, it’s interleaved. And the sequencing or interleaving of that work is fairly tricky for a robot, because humans are the ultimate uncontrollable entity. When a robot’s working with another robot, they can control a lot about, or pass information deeply about what the other will do. They can deploy the same planning algorithms, they can have a communication protocol.

But how is it that we communicate with other people when we’re working together? We have all sorts of subtle, implicit cues and signals that we provide each other and a large part of our work is trying to understand what is the minimal set for certain applications, that minimal set of signals we have to understand from a human to be able to decode what it is they’re thinking and to be able to track what it is they’re doing, so that we can enable a robot that maybe doesn’t have the full human capability to collaborate with us but can a truly effective team member and enhance us.

QUESTION: Looks like I have the microphone, so I guess I get to speak. Hi, all the way to your left, up top. Hey.

So, it sounds like from what you’ve spoken about the last two questions, the more experience, the more perceptive, the more capable these machines become, the more useful they become. We’ve all seen what happens when Skynet gets turned on and the Matrix. And a lot of people now are saying that AI and computers and robots are becoming so advanced that we’re getting close to a point where people are getting concerned about what might happen if down the line they become too powerful.

What is happening in terms of robotics that you know that is building in fail safes?

SHAH: So, robots, machines, the way they compute and the way they make decisions is, even when you’re talking about deep learning or neural networks, it’s fundamentally different than how we as humans learn and how we make decisions. And we can train systems, with the right training data, with a well-defined task to potentially exceed human performance on a large number of tasks.

But someone has to define that task for the machine and someone has to cultivate that training data that you provide the machine. And this in a way, this is one of the ways that we as humans structure the problem for the machine. We don’t give ourselves enough credit for the art and the judgment that goes into defining those tasks and then structuring the input for that machine to be able to learn.

One of the challenges on the other side is that, when the machine does learn and when it can exceed human performance on some specific or well-defined task, we often don’t have a good understanding of how it is it does that. It has no—often the machine-learning elements are not inherently interpretable. So there’s no way to look under the hood and understand, what is the high-level strategy that the system is employing? How can we vet it? How can we check it? Or even how can we learn from it? How can we pull the insight from that resource nurse and code it and provide it to a novice nurse, to be able to train them more effectively?

And this is something that we think about in my lab. Many other researchers are working on it, too. We need more than statistical correlations. We think in logic. We think in high level strategies and more abstract terms in various levels of abstraction. And so how it is we can design the scaffolding or structure for a machine to learn in a way that it still uses its strengths, but in a way that also allows it to describe back to us what it is it learned is I think one of the fundamental challenges going forward.

QUESTION: You know, we keep talking about the machine learning from us. When you were giving the example of the hospital, what is a bad decision? Don’t put that caesarian patient’s room 14 with nurse Meredith. Have we figured out a way to debrief the robot? If I were nurse Mere…forgive me if I forgot the name right. But if I were Nurse Meredith, I would want to know why they are saying don’t put that patient on that…

SHAH: Absolutely.

QUESTION: You understand, I know you understand me, but I’m going to keep going. Or if I were the people servicing room 14, I would want to know if there are cockroaches in that… Do we have, have we figured out how to debrief the robot?

SHAH: So, this is one of the key uses of a system that can learn from people and codify our implicit knowledge. It’s often—someone as talented and capable as these resource nurses, or someone that is incredibly talented at a sport, it’s very hard to describe in fine detail how it is you’re so good at that, right? People are not good at describing at the lowest level, and even in search-and-rescue, people assume a lot of implicit knowledge that the machine can decode from observing. And so there’s this incredible use case where the machine learns from our true experts, from people that can perform these very, very hard jobs so effectively and we use that to be able to multiply their capabilities, so we’re able to train many more people to do their job at that level.

And so in this area we actually work with, through the Air Force Research Lab, with military pilots. And they conduct missions. And then they come back and they debrief those missions to be able to improve for their next mission. And it might take them hours to develop just a timeline of what happened. And they may never even get to the part where they strategize and discuss and try to improve for their next round. And there’s so many examples of this.

And the key to this is to be able to design the machine-learning algorithm with a structure so that it can explain back to you the strategies that it sees in a way that humans can understand.

MODERATOR: All right, let’s thank Julie, and I’m sure she’ll stick around if you have any other questions.

SHAH: Thank you.