Art of the Possible: Autonomous Real-time Patching

Steve King 00:13
Good day everyone. I’m Steve King, the managing director at CyberTheory and today’s episode is going to explore the world of software flaw detection and beyond. Joining me today is David Brumley, the CEO of ForAllSecure a cybersecurity company whose products are based on Mayhem. The amazing machine that David designed to autonomously and in real time apply patching and continuous penetration testing. In fact, he won the DARPA Grand Challenge by demonstrating that it can fix its own software security flaws at DEF CON 24. David is a professor at Carnegie Mellon University as well. He’s also a very well known researcher in software security, network security and even applied cryptography. Professor Brumley also worked for five years as a computer security officer for Stanford University. So welcome, David. I’m glad you could join me today.

David Brumley 01:10
Thanks for having me on your show.

Steve King 01:12
Thanks for being here. Let’s start with your amazing machine dubbed Mayhem. Can you tell our listeners the story, which is a great one, of how it was developed in the process of delivering that crazy level of performance?

David Brumley 01:26
Yeah, well, it actually started back when I was doing IT. I was, as you mentioned, the computer security officer for Stanford, really, to date myself, it was when it was called google.stanford.edu, as opposed to google.com. So that’s how long ago it was. People just kept breaking into systems. And they kept breaking into them by finding software flaws. And so that started me on a crusade of trying to figure out how we can teach machines to find and fix those flaws first. And so I went back, got a Master’s got a PhD, actually got tenure at Carnegie Mellon University, all looking at how can we teach computers how to hack. And then in 2014, DARPA put out a challenge based upon my research and a couple others that says, “Hey, can you prove that these sorts of results are really going to be meaningful to the world?” And so we took that research, and we entered the contest. And we were able to show that “Yeah, we could teach computers to hack. In fact, they could find and fix software flaws in minutes.”

Steve King 02:24
You know, if, of course, if that’s true, it makes me wonder why the Biden administration hasn’t insisted that everybody install one.

David Brumley 02:34
Well, that would be great business for us.

Steve King 02:36
It certainly would. After Mayhems debut, you won a two year contract with the Pentagon’s startup centric unit, called the Defense Innovation Unit Experimental (DIUX), which is a mouthful, to find coding flaws in both operating systems and custom programs used by the US military. Is this program still going on? And how did me Mayhem perform there?

David Brumley 03:03
We’ve actually completed that. And so if you look at it, right, when you’re in research, and you’re showing something as the art of the possible, you’re really doing it in a controlled lab setting, which is a little bit artificial. And then the DARPA Cyber Grand Challenge, next level of maturity, but very much showing the art of the possible. What this program was about was taking that art of the possible and trying to what they call Bridge the Valley of the Death, right, where you have great ideas, and they never reach into industry to help foster that. And so this was really a two year program to show whether or not the things that we demonstrated at DARPA on this artificial terrain could really be turned into something that worked in real life for real customers. And so that’s when you start to think about what does it take for the system to ride alongside a human? What does it take for this to be incorporated in business policy and procedure and how developers and security professionals work? And so we took two years of that. And then at the end of that they did an evaluation to determine, you know, have you shown with real customers and the DOD, but this wasn’t just a cool research project, and still needs a little bit of time to bake but something that they could use today, I’m happy to say at the end of that they awarded us a $45 million contract. Because indeed, I think we’re being used in every DOD branch we’re being used in the IC, and several other areas to help make software safer.

Steve King 04:30
Wow, that beats a series A round, I guess, doesn’t.

David Brumley 04:34
It does. Actually what was interesting is from 2016 to 2018, when we were doing this, you know, we were taking that research, and we were trying it for the DOD but we actually did also get a series A round. And the reason for that is we went and we were solving problems for the DOD and at the time, and even today, a lot of what Mayhem is used for is to check weapon systems. And all a weapon system is a normal vehicle with some, you know, added protection and some armament. What we wanted to do is start taking this also out into industry. And so we raised 50 million from NEA at the time. And so we’re, you know, we’ve been supercharging, how do we take this research and get it out into practice as quickly as possible?

Steve King 05:17
How are you doing that?

David Brumley 05:18
I think we’re doing well. I mean, it’s really interesting, as an academic who is really just most interested in how do we find an exploit software vulnerabilities to prove that they’re true, but then also fix them at human computer speeds. You know, what does it take to build a product and for example, things that never come up in research, but come up when you’re bringing a product or things like you have to do a little bit of education for your customers. Because people are used to doing things one way. And then you have to kind of post to them, you know, what if? What if we were able to reduce human effort for you? It’ll require you change the way maybe you do business a little bit, but it’ll be in the long run. And so that’s what we’ve been working on. We have some cool customers, we have a number I can’t talk about. In fact, even the DOD, a lot of the customers’ use cases are classified because Mayhem actually finds real flaws. But there’s a couple that we can talk about, for example, CloudFlare, has signed an agreement that allows me to talk about them, we’re CloudFlare is using Mayhem, to protect their constant distribution network.

Steve King 06:18
Yeah, it’s amazing to me that this doesn’t happen faster when Mayhem is an obvious step up, which is an understatement, in both time and accuracy in terms of, you know, trying to determine whether or not the software is solid or has got bugs. There are probably other side benefits to Mayhem’s self healing capabilities. Can you tell us a little bit more about how it works?

David Brumley 06:45
Yeah, I’ll tell you a little bit about it. And I’ll actually tell you the trade off, right, because there’s no silver bullet in the world, this isn’t just something that, you know, is exactly the way we used to do it, but a little bit better. That’s not what the challenge was about. So back in the old days, and I’m talking about last year, people really what they thought they wanted was an application security product that would tell them where all the flaws are at so kind of they were in this mental mindset of this thing is going to tell us where all the possible problems are at, we’re going to put a human against that list, you know, kind of check it off, like your honey do list. And then we’re going to have safe software. And that vision has been wrong for a number of reasons for a number of years that I mean, I could go on and on about it. But really what Mayhem did is it said look, we’re never going to say that we find all the problems, but what we’re going to do is we’re going to say every problem that we find we can prove it, we have a witness and actual working exploit that would trigger the vulnerability. And the longer you run Mayhem, the more and more of those vulnerabilities, it’s going to be able to find and prove. And so what it’s actually doing is you can kind of think of a program like a recipe, right? A program is instructions on how to cook something, Mayhem is going through these incredibly long and detailed recipes and finding all the places that you end up with baking something that’s bad, and showing you “Hey, if you do these steps, this is how you reach into this bad state.” What we did in the Cyber Grand Challenge is then we would fix them. And what we found is part of our go to market actually as a product company is the automated fix is something that technologists are enthusiastic about. But there’s a variety of actually legal barriers that prevent it from necessarily being the best approach today. For example, in Cyber Grand Challenge, when Mayhem found a flaw, it would automatically fix it would rewrite actually the program binary, the executable that was running on the system. But here you’ve run into an interesting problem. Suppose Mayhem is being ran on a commercial product. And we change the code of that commercial product. Well, you’ve licensed that product from a vendor and now it’s been modified. Now that vendor may be upset, the vendor may no longer warranty it and so like that automatic fixed we haven’t quite got there yet, mostly because of kind of this legal barrier of what do you do when you modify software that you yourself didn’t write?

Steve King 09:01
Yeah, so it’s not that you can’t do it. It’s that legal barriers exist that prevent you from from doing it, in terms of whether it’s a good judgment call or not. Right?

David Brumley 09:13
Absolutely. That’s where we’re at. And so as we built up the company and tried to bring this out to practice we focused on how do we find and prove vulnerabilities. Because what we found is developers are just exhausted everyone talks about shift left and developers have to stop releasing buggy code and all these sorts of things. Really, you’ve taken someone who loves to code and you’ve yolked him with this security responsibility. What Mayhem has helped them do is really two things: every time we say there’s a bug we prove it or vulnerability we prove it and the second is we actually help them better test their software just to make sure you know good stuff works because at a high level when you’re doing application development, you want to test good things work like you know what I wrote it for really happens as well as no bad things can occur and Mayhem actually automates both for them.

Steve King 09:58
Yeah, it’s sort of Like having 1000, white hackers all organized around various different components of code, but not correcting it, just identifying it, it seems to me. And I know, you know, those poor folks in DevOps and DevSecOps are, as you point out, indeed, overwhelmed by all of this newly discovered sort of focus on what they do every day, and as a former programmer, it does hamper your style a bit. But you’ve got two products now. You’ve got this Mayhem for code, which I think delivers the automated, integrated, you know, accurate security testing. And then you got this thing called Mayhem for API, which seems to provide similar automated tests needed to build quality APIs. Can you tell us about fuzzing which is kind of at the core of that technology and the impact it has on DevSecOps, as we were just discussing, and the role it plays in both your solutions? And then potentially across the industry?

David Brumley 11:02
Yeah, absolutely. So if you go back and look through, you know, how people have approached this problem of trying to find flaws in software, right, there’s a couple different approaches. One is you can always look for the known knowns, this is saying you’re using an out of date open source library, and you should probably update because that open source library has a bug. And companies like Sneak and SourceClear and Black Duck do a great job of this sort of stuff. So it’s not really what Mayhem is focused on. Of course, we can find vulnerabilities in that, but it’s not that (unintelligible). The other way of doing it, which really has been the industry norm today, has been what we would call static analysis, where you take the program and you’re essentially running a grammar check over it. And just like a grammar check, if you’ve ever had a Microsoft Word document, ran grammar check, you know, you see a lot of suggestions that you don’t want to take. And that’s kind of the same experience programmers have. Mayhem, as you mentioned, uses something we call fuzzing. And fuzzing is actually just as old as static analysis. Static Analysis really came out in 1970s, fuzzing actually predates that, even a bit. Where the original idea is if you give a program a bunch of random inputs, right, like so instead of a human interacting with a program, you have a bunch of monkeys just typing. Surprisingly, they found about 30% of apps, you would trigger bugs, actually pretty serious ones. Now the researcher has moved way beyond monkeys, we actually, inside Mayhem, use hardcore latest research in modeling programs as mathematical equations actually playing something called SMT solver, some things that can solve mathematical equations to figure out where these equations can kind of capture where things are vulnerable, and then where they’re true that would give us a way to trigger them. So we’re modeling the formal semantics of it. So that’s what fuzzing is doing is kind of conceptually, you can think of as a million monkeys, except for it’s not monkeys, it’s actually, you know, Deep Blue, it’s the chess engines running on the program, systematically exploiting them with the advanced computer science. And so when we first started doing this, and for the DARPA Cyber Grand Challenge, this is what we focused on. We focused on compiled applications we focused on, you know, this would be your go or your rust or your cc plus plus. And the reason those are especially important, especially in the DOD context, is our critical infrastructure runs on those energy, airplanes, automotive, everything runs on compiled programs. But we also, of course, can’t miss the fact that a lot of the world is moving towards web and SaaS services. Now, these aren’t compiled programs. These are usually actually a system a system. So when you talk about a web API, you’re really talking about, there’s a little API piece of application logic, and it connects to a database. And maybe that database has some other things feeding it as well. So really what we would call a system of systems. And so we built Mayhem for API to bring the same benefits that we had for Mayhem for code looking at a single application to an API where you can just point it at an API. And even if there’s, you know, three or four different computers that have to cooperate to bring that API to you. We can check it.

Steve King 14:09
Is it being used today for NCI at all? I mean, we have no visibility. And I mean, it’s one thing for Biden to issue and Ex. Ord. and all the rest of that now, all of that was all good. But we have generally no visibility into what is going on around the critical infrastructure world in terms of protection, detection, what have yous. Can you talk about whether mayhem is being used for that today or no?

David Brumley 14:41
Mayhem, within the Defense Department is being used to check weapon systems. So this would be things like airplanes and so on. We also have customers and unfortunately, because of NDAs and respecting customers’ privacy, we can’t name their names but we are being used in aerospace, we are being used in automotive, to go and check things and what we’re seeing actually are new requirements coming out for each of these industries. For example, automotive just had one come out called ISO 21434, which is about safety and security testing, where Mayhem really is one of the few solutions just kind of plugs in and does it. So we’re starting to see that. When we’re talking about what’s best for security, you can look at it from the technical perspective. And that’s great. But as a startup, one of the things we’re always looking at is the business perspective. And it takes time just for people to understand, oh, there’s a new technology, we have to figure out how to adopt it, we have to figure out how to buy it, we have to make sure we have budget for it, all those kind of things that go into actually using it. That being said, it’s been, I think, incredibly strong uptick. So we’ve really only kind of had the product launch in 2020. And we have about 20 customers using it today.

Steve King 15:48
I’m sure you pay attention to what’s going on in the world of cybersecurity on an intimate basis. You’ve seen, of course, the attacks on Colonial and the water systems, and JBS etc. What do you expect is going to happen here pushing out maybe six to 12 months into the future?

David Brumley 16:12
Well, if we look at it from a policy perspective. So Biden has surrounded himself with some really top notch experts. That’s the first thing that I want to comment on. People that I’ve actually known for a long time, like Chris Inglis, becoming the Director of National Cyber Policy are big deals.

Steve King 16:28
Yeah, I agree,

David Brumley 16:30
Deep expertise, really deep technical expertise, and have been just public servants for a long time, very smooth sailing. So I think that we’re going to continue to see more and more policy. The first thing that we’ve seen is really the software bill of materials, becoming an important part of it. And this is like the smallest unit that they could be doing really what they’re trying to say is when you deliver a software, you should know what its ingredients are. And so that’s going to happen. I think one of the things I’m interested to see is how much they lean into regulation. So historically, what’s happened in the energy sector, in particular, since we’re talking about the Colonial Pipeline, is administration’s have wanted them to self regulate. So all the energy companies, for example, would decide what they thought was critical infrastructure and then agree amongst themselves, they should protect it. But there’s kind of a perverse incentive there. Because they would, for example, say, “Oh, that computer’s not really critical.” And they would say it not because it’s not critical, because it’s vulnerable. They didn’t want to count it. And so I think there’s a really interesting balance that has to be done here. Where in anything you do, you don’t want to have huge amounts of government oversight, just because of the speed of government is super slow. And you’re not going to keep pace with technology. But it’s certainly not enough today. Like they haven’t struck the right balance. And so I’m kind of interested to see how this goes.

Steve King 17:48
Yeah, so am I. So you mentioned the fact that we’ve got some great people up there now. And like I said, I concur with your assessment. Certainly. I’m just amazed that Well, while you’re right about the speed of government being, you know, syrupy slow. I’m just amazed that if you’ve got a solution, like Mayhem, you know why it doesn’t get rolled out on a on a kind of a national level. I mean, you know, we managed to get to the moon in a few years, and we managed the Manhattan Project, in whatever was 27 months on some very loosey goosey nuclear science, I don’t see why we can’t have a similar program, here to figure out how to ward off the onslaught from our adversaries. What are your thoughts about that?

David Brumley 18:36
Well, I’ve long advocated actually creating a Manhattan-style program within the DOD, especially around critical infrastructure. I couldn’t tell you why it wouldn’t happen. I know, for example, after Mayhem won the Cyber Grand Challenge, and the following year, two things happened. First, Congress actually put in language into the appropriations bills that said, DOD shall start using Cyber Grand Challenge technology. Second thing that happened was an election. And as we know, Trump won the election. And so that language was basically ignored, right, because Congress will say to do things but they don’t always necessarily get done as far as how appropriations works. And I think we need to revisit that about saying, we need to invest in more cyber autonomy out there. Because at the end of the day, like, it’s not just application security, even though in my mind most compromises – and DHS backs us up with statistics – are due to software flaws. Really a lot of security is a cyber workforce problem, where, you know, theoretically, if for every developer, we could pair them with an application security expert, it would be secure software, very, very good. But you don’t have that amount of talent. For every computer network out there. We know IT people are just stretched thin, keeping it up, let alone having enough security people. And so I think we really need to double down as far as a Manhattan Project and building out more autonomy here, and hopefully we’ll see it.

Steve King 19:56
Well. That would be terrific. And we’re watching closely here. I’m conscious of the time here, David. So I’ve got one final question. You know, we talk about DevSecOps a lot here, as we’re rolling out our CyberEd.io, online training and education platform at ISMG. You know, we know that DevSecOps can be impacted positively by, as you say, you’re having a security expert sitting side by side or by automation, integration and speed. While you acknowledge these issues, you say they’re symptomatic of a larger root issue, which is inaccuracy. What’s your position on accuracy at ForAllSecure? And how can you guarantee that deliverable when so many others haven’t been able to?

David Brumley 20:43
Well, the big differentiator for us is we’re saying every time we say there’s a bug, or a vulnerability or something fails, we can prove it. And so that’s what keeps your limited human resources really focused on what they can do best, which is create a problem solving, right? As opposed to trying to go through a big laundry list and figure out what’s real and what’s not real. And so as I mentioned, the way we do this is every time we say there’s a problem, we actually give you an input, so developers can debug it the way they would any other problem. If you think about it, actually, DevSecOps, is in a way what Cyber Grand Challenge was about because it’s about autonomous pipelines, you know, if you get past the test phase, you can auto deploy. And so that’s what Mayhem is about is building that up with the idea that you don’t want an extra human step in there, always verifying everything and automate everything. And so I think that’s really how Mayhem works. It’s how it plugs in. And the way we guarantee it is every time we tell you, there’s a problem, we can actually just, just like if someone reported a bug, you’d ask like, “How did you trigger it?” Mayhem can actually answer that question. It’s the first product that really does that.

Steve King 21:43
It’s really amazing. I wish you and ForAllSecure all the best here. You’ve got an amazing product. We don’t normally, you know, do these podcasts to promote companies. But I think your situation is remarkable and extraordinary. So I’m more than happy to help let the world know what you’re doing and how cool it is. Thanks for explaining all of this. I’m sure our viewers want to hear more. So maybe we can get back together in a few months and talk about what’s happened between now and then. And any progress that Mayhem’s sort of achieved or traction during that period.

David Brumley 22:27
I look forward to it, Steve, and thanks for having me on the show.

Steve King 22:30
Well, great. I want to thank you, David, for taking time out. And I’m sure you have a crazy schedule. And I think we had a nice 30 minute chat about this and want to thank our listeners for joining us in another episode of CyberTheory’s exploration into the complex world of cybersecurity, and digital realities. Until next time, I’m your host, Steve King, signing out.