EP 5 — Travis McPeak: Securing the Modern SDLC with Security Guardrails

Travis: Hey, amazing to be here. Thanks for the invite.

Harshil: You've done a lot of interesting things over the past few years. Tell me and our audience a little bit more about what's next. What's exciting that's going on in your life?

Travis: So today is my second day on an exciting new journey for me. And really most recently I was at Databricks. That's an incredible company. Good opportunity. It was very hard for me to leave, but I'm fortunate enough to have the opportunity now to build something with my bare hands. So build my own thing. If it's good, it'll be my doing. If it's bad, it'll be my doing too. So it feels very motivating for me. What I'm doing is really without being too specific, I'm trying to make developers lives easier. I think especially in a DevOps world, developers have so much on their plate. They have to be kind of mini experts in everything, right? They have to deploy, develop, test, operate, do things in the cloud. And so my goal is to make their lives as simple as possible in an area that I have expertise in, which is cloud and cloud security. So really making systems that make developers lives easier when they use the cloud, is so exciting, man.

Harshil: First of all, I just love this journey of security practitioners, becoming founders of security companies, and solving the pain points that we all saw. It's just an amazing story. But also the other thing that I also love is making security easy for the developers that's I think the key part of how to actually build scalable security. And you've done that at several companies before this. So it sounds like you're in a perfect position to go and solve this problem.

Travis: I will say that your journey has been a huge motivation for me as well. Just seeing you start a company and all the excitement behind what you're doing motivated me to take the uncomfortable route and start my own thing as well.

Harshil: You're too humble. Fantastic. So I would love to dive a little bit deeper into why all of a sudden we are seeing more interest in making security easier for developers. And I guess it's not a very recent phenomenon. Right. And, folks, when you were at Netflix, you guys gave a lot of talks and presentations about why that is so important, the paveroad concept and all that stuff. So if you can help us understand why is that important in this day and age?

Travis: I think we've seen security been shifting kind of a strategy over the last at least a decade, certainly accelerated in the last few years. Back in the good old days. Right. Everyone had their specific roles. Developers wrote code, and then somebody else tested the code and security would do a security sign off. And then now with quicker release velocities, we can't do that. The waterfall model doesn't work anymore. And so it used to be like, whose security job? Security is security's job. And today whose security's job? Everybody. Everybody's got to be responsible for security, but it takes a long time to learn about security. There's a lot of kind of nuances and got you. And so really, from my perspective, where you can add value as a security person is making tools and processes that empower every person to do security effectively without having to build that deep expertise.

Harshil: That's an interesting way to look at this. Right? I mean, I don't think anybody would deny that security is everyone's responsibility. I think the key difference is earlier we all used to just talk about this and expect everybody in the company to just follow. But now we have an opportunity to actually build tools and systems that enable them to do something about security. And by them, I mean developers. Right. Or any other team within an organization. So this has changed recently, and this has also led to, like, if you're following this mentality, I guess the type of people you would hire in your team would also be different because now you're not hiring people who are just abstract consultants for performing testing all day long. Right now you're hiring people who can build tools, who can build those paved roads and so on and so forth. Is that easy to hire for that type of skill set?

Travis: That's right. Yeah. So the hires that I'm most excited about are versatile. They have a security background, and then they can also build systems and you can definitely find them because as opposed to a security person that comes from a security background and just builds that skillset over time, sometimes the people like we're talking about here actually come from a developer background and they get exposed to security, and then they want to keep going with those problems. So, for example, Databricks, one of the more recent hires I did, it was a few years in his career, and he was amazing in the sense that he had software engineering skills and was able to ramp up well on our problem. So when you do this from my perspective, you actually expand the talent pool, which is really important because as you all know, we don't have enough security folks in the industry to just be stealing the talent from each other all the time, right?

Harshil: Yeah, that is true. And since moving into this role in my new company, I've also realized that we all talk about how difficult hiring security engineers is. It's actually equally difficult to also hire for good software developers. It's not that easy, but I guess at least the talent pool is much bigger. Right? There's huge volume of good software developers. Convincing a few of them to learn security makes them into really good security engineers because now they come with the software background and they also understand security to an extent.

Travis: Yeah, that's right. I think the number of people that need to understand the ins and outs of cross-site scripting exploitation. If you're building systems that prevent against that problem, you only need one person with the skillset on the team, and the rest of the folks understand the problem well enough to build a system around it.

Harshil: That's great. So how do people think about stopping this? Right. So you bring up a good point in terms of, like, you need some level of deep security expertise and some combination of systems building and systems design thinking as well, people who can code and build systems. So when you are building your team at Databricks, when you join, like, how did you think about what type of people you hired first or what type of people you already had versus what was the gap in skills that you introduced?

Travis: Yes. My favorite profile. And this is in part just my bias from what I saw was successful on Netflix. But I really like if you have a spectrum where you have, like, strict software engineer on the left and strict security engineer on the right, I like right in the middle. I like good software, good security, because they're so versatile. If you happen to have a time where you need, like, operational support for a team, they're designing something, you have to get the security right. The person is equally good there as they are for quiet times when you can build automation and prepare for the future. I just think a person like that, it's like a hammer. You can do a million things with it.

Harshil: There's also this relatively recent phenomenon of teams hiring builder-type people within the software security teams or product security teams and then calling that like a security engineering function. Right. So you're building systems for the security function within the organization. There's also some companies who use product security as another team name. So now we end up with either AppSec or ProductSec or security engineering. How do you distinguish between those different functions and what does that actually mean?

Travis: I have something funny that I've given. It's ironic given my previous title, but I actually don't 100% know what product security is. It means different things in different organizations. But I guess contextually, you're going to have security people whose job it is to shore up the security of your product. Right. So this is different than security functions like corporate security, where they shore up your workforce and your G suite and things like that. It's really like the software that you're building that you sell to customers, making that secure. That's kind of where the line for product security is for me.

Harshil: And what about security engineering? You see that as a different function. Like if you had enough resources, spin up a new team, would you consider spinning up a new team, separate team of the security engineering team versus an AppSec team?

Travis: I would call the security engineering team, like the people that are building software for the purpose of security, whether that's product security or internal security, the engineers that build security and then AppSec is yet another thing. We have like a confusing Venn diagram situation here. But application security is a part of product security where they focus on a specific level of the stack. So that's what's they call it, platform security. But the folks that are responsible for the low-level infrastructure that people deploy apps on top of. And then at Netflix, they also had cloud security, which is responsible for working with like AWS or GCP or whoever your cloud provider is, in showering up resources. And then the application security is the level on top of that where they are responsible for making sure that the applications that are being written by developers are secure.

Harshil: When I talk to a lot of other apps like teams, the one common thing that comes out is nobody disagrees with the fact that they have to automate their own things. They have to build their own systems just because every organization looks slightly different. Right. But in most cases, they actually don't have enough budget, resources or access to the skill set that they can do the things and build the things that you guys used to do at Databricks or Netflix, for example. So when you have a smaller team who's sort of limited in terms of how many people they can hire, how many developers they can hire within the security organizations, how do you suggest they should think about automation, customizations, and building your own stuff?

Travis: Yeah, that's a great question. Different organizations are going to have different security budget, like how much they're willing to spend in response to risk. And that budget will go into the security folks that you hire. Now, let's say you're budgeted for five people, right? That's going to cover all of security. So you know that you need a corporate security function. You need to have laptops patched and things like that. You need to have folks on application security and cloud security. Now, if you have the five and then you split it up. Now maybe like one and a few different functions, your apps, that person one of one has a lot of surface area to cover. And so for them to build software that solves a specific problem is going to be very expensive. They may spend a year or multiple years building that thing, and then they need to maintain it. So this is really where great vendor products come in. A lot of the security challenges that an organization faces are not specific to the organization. Every other organization that is in the same market as them has the same problems. And so a lot of times the way I think of it is like a vendor solution will cover 80% of your problem and then 20% is specific to the company. And so if 80% is good enough for you and your staff with one person, then a vendor makes clear sense. You're not going to be able to write the automation. This is like a situation that I've talked through with a lot of founders when I help them as a security consultant or advisor is like, hey, you've got two people we really shouldn't be building bespoke solutions here. Like, look for vendors in these spaces.

Harshil: And I guess that was one of the inspirations for you to start a company because most other people are facing the exact same problems that you sold for, right?

Travis: Exactly. Yeah. So, I mean, like, what you're doing my thesis as an investor or now as a founder is take the things that keep getting built again and again at different companies that are commodity. Right. It's the same problem, but everybody builds it because there's no vendor in the space and make a vendor product that solves the 80% use case. And what you get there is, let's say at Netflix or Databricks, we do have enough talent to build something, but we prefer not to. We would like to pull that engineer back and have them work on problems that are specific to the organization. So even if we built something internally, if a vendor product does most of the same thing, I would prefer to pay for it all day, every day.

Harshil: I think that's one of the challenges as a practitioner. What I saw was when you walk the halls of RSA or BlackHat or whatever, which I haven't done in a few years. But a lot of people building security products over the past several years don't really understand the practical problems that we deal with on a day to day basis. Right. And a lot of products just throw in buzzwords all day long and try to sell it brute force. But when we see practitioners who have felt this problem, who have solved these problems before, take that and try to commoditize those problems, I feel like it creates much more relevant solutions that a lot of people can start adopting very quickly.

Travis: Yeah, totally. I've been in a position where I've seen a lot of pitches for companies and startups building things lately. And one of the things that comes up a lot is like, here we built this thing. What is it? It's a dashboard of problems. Like, I don't want any more dashboards a problem. I have so many problems. Like, if we can't actually solve the issues, I don't want to know about new problems that I don't have the solution for.

Harshil: Hey, it can be solved with AI, ML and Blockchain. Right?

Travis: Okay, good. Yeah, I'll take five of those.

Harshil: Right. So now we talked a little bit about secure by default. We didn't actually talk about secure by default, but we brought up paved road, which is kind of secure by default, and that kind of operating methodology of how you make security as the easiest path for the developers. Right. So let's say you hired a couple of people who are building those solutions in house for you, making your development lifecycle secure by default and making security the easy path for developers. It's almost transparent. Have you really seen that take effect, or does that actually move the needle with developers, or is it still a pie in the sky kind of a story?

Travis: Oh, yeah, absolutely. So one of the big success cases that I saw on Netflix was Spinnaker. So Spinnaker is a deployment system that is open source. I believe Netflix originally created it, but a bunch of different companies use it now. Actually, Funnily enough, every company that I've either worked at or interviewed at in the last three years or four years uses Spinnaker. So I think it's more prevalent than I actually knew it was. But, yeah, what it is. So if you're a developer and you need to deploy an application, there's a lot of things that you would need to go learn about. Right. So, like, how do I do failovers? How do I do traffic load balancing, how do I do red-black deployments, things like this? You need to be a deployment expert to know about these things. And Spinnaker makes that easy for you. So you basically click a button and you say, I want a new application. And Spinnaker has a bunch of reasonable defaults about these best practices that you get for free. And so I think this is like to extend that approach for security is what I'm most excited about, where a developer doesn't have to think about how is my S3 bucket protected, or what kind of permission should I have on this queue? A system like that, like Spinnaker, could just pick same defaults for you, and then you don't have to worry about it. I think it's an economic thing. It's like something the tragedy of the defaults or the tyranny of defaults or something. But it's basically like if you have a default option like, do you want insurance? And it's default yes. Or default no. The difference between who actually ends up with insurance is like 80% to 20%. I don't know the exact numbers, but defaults are so powerful, right, that if you think carefully as a security person about how you can make the security system just automatic for a developer, that's such a powerful concept.

Harshil: That's actually a really good way to drive adoption. Right? It's just if it's transparent, they don't have to do anything to get security. They're just inheriting it. But I guess somebody would have to think about it. Somebody would have to build it out. Somebody would have to make it relevant and contextual to the business. What have you seen in terms of who does that? Is it the security engineering team or security team, or is it more of the platform DevOps developer experience? Whatever team you want to call it somebody who controls that. The deployment pipelines and Spinnaker, for example, is it them who's building those controls?

Travis: It could be both. So I'll give you a couple of examples. One is like Spinnaker itself because so many developers use it in Netflix. Now, the security team would target Spinnaker as a point we layer our security controls on. So we as a security team actually went over to Spinnaker and implemented features that would be on by default because it's like a watering hole. That's where all the developers are. Another case is we had a tool that was an internal tool called Newt, was like the Netflix new project creation tool. And it's really cool because it's got templates. You just say, I'm a developer, I want a Python microservice running Flask, and then Newt will create all of the stuff CI deployment job and a build. And like your Git repo and everything. And because of that central place, somebody I don't know who had put Bandit so that it ran automatically on the Python projects. And so as a developer, you don't think about it, you just get Bandit running for you, and you may be able to opt-out of it, but if you don't, then you're going to automatically have security linting for your project.

Harshil: That is so cool and so powerful. Just a few weeks ago, I came across as another project. I think it was built by the team at Spotify. It's called Backstage. Io, and it's very popular in modern development teams, and it kind of does that bootstrapping for you. So you just tell it, I want to spin up a Python project and so on and so forth, and it will create a repo with all the right templates, with all the right CI files. And that's a fantastic place to add your security controls in it. So it's all transparent to the developer. They just natively inherited as well. In your experience, did you have to do anything different to drive adoption of it, or were you forcing controls at Spinnaker level or at the CI level.

Travis: No, that's the great thing about defaults. Right? You're not forcing it's a suggestion. And most people leave the defaults alone. So if you were like, hey, I really don't want Bandit, or in the case of, like, Spinnaker, I don't want this setting for my deployment pipeline. You can go and change it, but most people don't. And so by default, you get whatever 80%, 90% of new projects are covered with these controls.

Harshil: Yeah, fantastic. And I'm guessing if it doesn't work for whatever reason and they could make their own choices, they could make their own decisions in terms of removing these controls, that there is incompatibility issues, or if it doesn't work for whatever reason.

Travis: Totally. Yes. And this is where, as a security organization, the person providing these defaults, we need to make sure there's a killer user experience. Going back to my earliest point, developers are so busy, the last thing that we want to do is make their lives harder. So we can't just be like, throw a million security tools in there and hope that the developers don't complain too much. That's not going to make our customers happy. So we always have to have that like, product first mindset and make sure that what we're adding is making people's lives easier.

Harshil: Yeah, that's great. I think a lot of it also has to depend on the company culture as well. Right. So for example, if it's a much more regulated company, then obviously they have to take a little bit of a different approach in terms of actually enforcing things because of whatever requirements, government regulations or industry-specific requirements. In that case, it can be done differently. But for the vast majority of things, as you said, you don't want to make a developer's life even more difficult than what it actually is, give them security as the easiest path and drive adoption that way.

Travis: Yeah, absolutely.

Harshil: So one of the things that, as I've come to know you, I've noticed that personally, this is a separate topic outside of work, but I know that you're very much focused on maintaining personal health, and you work out quite frequently as well. With this busy day to day of being at Netflix, being a data breaks now, doing your own thing, it can get busy. And security, by its nature, is a very stressful role. Right. So do you have any thoughts on, like, what do you do and how do you motivate yourself to spend time towards that?

Travis: Yeah, I love this topic. So first of all, I find taking care of myself to be like a net positive outcome, despite the time it takes. So I work out for an hour a day and I get that time back in being more productive, like having clearer thinking. So for me, it's a net win. But the other little trick that I play is I do what's called pay yourself first. So this is like a finance technique where if you want to save money, like say you want to save like $2,000 a month, then you take your paycheck and the first thing you do is you send two K to your savings account and then you don't have it anymore. So I do that same thing for fitness, meaning I wake up in the morning and the first thing I do, whether I feel like it or not, is I exercise for an hour. That way it doesn't matter how I'm feeling in the rest of the day. I've already exercised. I can't pull that time back. I find it, one, to be a good way to get the day rolling, and two, a good way to make sure that I always get it in.

Harshil: I love it. So it's not a decision that you make every morning. The decision has already been made. You're just going to do it. That's right. Just roll right out of bed, right into exercising. That's so cool, man. And I believe I ran into you at least once going up the Hill by where you live.

Harshil: So before we wrap up, just to summarize this, think about yourself. A few years ago, if somebody who is listening to this podcast, they are ambitious leaders training themselves, educating themselves, learning how to be a better leader, more senior leader within the space of AppSec. Like, what advice do you have for them? Yeah. So first of all, I totally remember being an engineer and knowing that my passion was going to be leading people. And that's such a hard transition. I have a ton of empathy for people that are making the transition because it can take a while. You kind of have this catch 22 situation where until you've led people, people don't know if you're going to be a good leader or not. And they call it the blast radius and kind of like cold, calculating managers speak. But blast radius is like if you're an engineer or an IC and you don't do well, the blast radius is your project. If you're a people leader and you don't do well, the blast radius is your team. And that gets more and more bigger impact as you go up the ranks. And so there's this kind of reluctance to give new people that haven't led before an option. I would say one, keep your head up. Two, spend time learning about the kind of hard parts of management before you get into it, do some of the tasks that managers do even if you're not in the role. So, for example, how does the team organize themselves? This is something you'll be responsible for as a manager. How does recruiting work? How do you build the reputation of the team? These are all things that individual contributors can do that would also help the manager and demonstrate a skill set and prove to yourself that you actually like this work. So those are all good ways to get started and then the third thing I would say is in the security industry we have a talent shortage so all things being equal we're in high demand and the people at your company should be willing to support you in your career goals and so don't expect it to happen overnight but if you've been asking for an opportunity for a year or a couple of years and it's not happening then don't be afraid to look outside in other places. There are a lot of companies where they have like a tech lead manager or other like kind of smoother paths into people leadership. So take a look at some of those opportunities to maybe get something that you can't with your current employer if they won't support you.

Harshil: That's fantastic advice. I think growing our own team, growing the people that are already within the team that's such an important thing being able to hire is 100% agreed. It's incredibly important so learning those skills, growing with the rest of your team, helping everybody else within your organization, being a leader that's really good advice. Thank you so much, Travis for being on this podcast. I really appreciate your time.

Travis: Thanks for having me. It's awesome. Hopefully, you'll have me back soon.