The Future of Offensive Pentesting with Mark Goodwin
44:40 Share

Offensive Penetration Testing, or Offensive Pen Testing, involves actively probing a system, network, or application to identify and exploit vulnerabilities, mimicking the tactics of real-world attackers. The goal is to assess security weaknesses and provide actionable insights to strengthen defenses before malicious actors can exploit them. Bishop Fox is a private professional services firm focused on offensive security testing.

Mark Goodwin is the Director of Operations at Bishop Fox, and he was previously an officer in the U.S. Air Force where he did cyberspace operations. Mark joins the podcast with Gregor Vann to talk about Bishop Fox and the future of offensive pen testing.

Gregor Vand is a security-focused technologist and is the founder and CTO of MailPass. Previously, Gregor was a CTO across cybersecurity, cyber insurance, and general software engineering companies. He has been based in Asia Pacific for almost a decade and can be found via his profile at vand.hk.

Hi, Mark. Welcome to Software Engineering Daily. Hey, thanks for having me. Yeah, so great to have you here today, Mark. You are at the company BishopVox, which we're going to hear...

all about and understand about what is called the Cosmos platform and really just what Bishop Fox does in general. I think one of the first things would be really just interesting to hear about. Obviously, some details I don't think you'll be able to go into, but just your background is very interesting and sort of how you then ended up at Bishop Fox. So maybe if you just sort of introduce yourself a little bit in terms of, I don't know, straight out of college or high school, like what did you do? Yeah. Again, thanks for having me on. My name's Mark. I've been with Bishop Fox for four and a half years.

I graduated college back in 2012, 2013.

Immediately went to the Air Force to be an active duty officer where I got to do offensive cyberspace operations. So a lot of late nights, a lot of long days, and learned a lot about threat posture. How do we go after hard targets in ways that keep the toolkit safe, keep the platform safe, and go after nation state objectives? So did that for six years, six and a half years.

And at the end of my time in the Air Force, I started applying and Bishop Fox was starting this new service called Cosmos. And it was all about continuous offensive security at scale. And, you know, definitely very exciting, very...

I consider it the future of offensive pen testing to be able to stay up with nation state actors, follow emerging threats, and identify attack surface that customers may not know about. So like I said, I joined Bishop Fox in March of 2020, started as a senior analyst running the analyst team, which is all about discovering attack surface. Did that for about a year and a half.

Led the operator team, which is all about taking that intelligence that we derive, that attack surface that we see, prioritizing potential vulnerabilities and providing true positives and true negatives, right? So true positives, here's a finding that we know exists, that we've validated and we've proved you're vulnerable to.

with a finding, or we give them a true negative to say, we went and identified that this host is out there. We thought it was going to be vulnerable and it wasn't. So you know you're good, right? Your defensive team is doing the right things. They're applying the patches. This misconfiguration that we thought we saw didn't actually exist.

So we're able to provide that like, you know, safety blanket to say, you know, even if you're not getting findings, you should be getting something to say, we're looking at your attack surface and we don't see anything. Yeah, super interesting. So I'd love to understand. So you've given a great sort of quick overview, I guess, of Cosmos.

My understanding is that Bishop Fox started as more like consultative company and the platform aspect has kind of developed over a significant amount of time. I believe Bishop Fox started back in like 2013 or around then. I believe it's older than that. I believe we're an 18-year-old firm. So yeah, they have been doing consulting for a long time. That has been...

That is how we've made our name. We bring on some of the best offensive mind in the country, across the world, and let them work on really interesting projects. We dedicate time to training and developing tools that is going to make the offensive cyber community better. I have to shout out Joe DeMessi with his Sliver tool.

tool kit that he's built. Pretty exciting. And we use that almost daily. So yes, Bishop Fox started as a consulting. It's still our primary way to get after securing customers, keeping them safe and protecting their customers' data and trying to find vulnerabilities. We have our consulting firm and then we have the Cosmos firm

And they're both, you know, continuing to grow. Amazing. What was the, I guess, do you know kind of what was the moment when it was thought actually bringing, turning sort of this knowledge into a platform made sense? And like, what was the kind of the thought process behind that? Yeah. So when I joined in March of 2020, we had been doing, we had called it CAST, which is like Continuous Attack Surface Testing.

And then we had been doing that for almost a year. And I think it started from this mind of like, you know, how do we take our insanely skilled consultants or operators and give them a suit for Ironman, right? Like we don't want to build Ultron where we're just automating everything, but how do we instead enable and make our operators better? And so that was kind of the driving force. So it's on our very old documentation that, you know, from the way back times,

We have that, you know, we're not building Ultron, we're building Iron Man, you know, to propel us forward. And so part of that was looking at enterprise customers, right? Their attack surface is huge. How do you look across all of it when they just need help identifying what is the most important thing, right? We only have so many hours in a day, in a week, in a year. How do we find those things that advanced actors are going to go over, right? Hackers by trade, very lazy, right?

We know that if we can take the knowledge of 10 operators or 10 consultants, consolidate that data, make it easily attainable, take off some of the scanning that every external pen test starts with. What if we instead offload that to automation? So it is on a weekend or smart scanning as we identify new targets, it gets scanned and then it prepares that data for us. So I'm not...

wasting cycles waiting for a scan result to return. Instead, that data is there ready for me. And then when I go to pick up the work, because we've had five other super sharp consultants or operators do a similar investigation, we get a jumpstart on what we're looking at. Yeah. So I think this is where it gets super interesting. So my understanding is the platform has, it kind of has like three parts to it. So you've got the automated attack surface, you've got the automated application,

application pen testing and then does this understand also automated internal testing and pen testing is that correct? Right now we're not at the automated internal pen testing yet that's definitely on the roadmap though how do we do that smartly how do we make sure that we drop into the right location and again it's for Cosmos the entire point is scalability right I need to be able to do this on five targets 500 targets or 5,000 and way beyond that right so right now the main focus is that web app pen testing and

doing the research on customers attack surface, we know it makes up the majority of their attack surface, right? There's going to be SSH and FTP services out there that may be vulnerable, and we're going after those. But most of the company's attack surface is going to be web based. So that's what we spent the first four years really focusing in on making sure that we can automate what we should write like, hey, this is an exposure, you need to go fix this, preparing the work for the team of, hey, we've identified that there's a version mismatch or

There's a potential SQL injection here. We're not going to throw that. We want to keep that to be a human decision, right? Because at the end of the day, the last thing we want to do is knock over customer infrastructure, right? Like that's one of the worst things that could happen.

So using this insight, using this, you know, prioritized attacking, we're able to say like, hey, we're going to automate these things that we don't care about. You need to fix it, right? Directory listing is something that you should not have. Like that's not a good configuration because of the potential to put something sensitive there. But we don't necessarily have to go look at that, right? We can build automations that will look for directory listings, tell you about it, and then look for things that were exposed and identify like, was there something sensitive here that needs follow on work, right?

So, you know, continuous web app pentesting is what we're started on. Continuous external pentesting is kind of that next thing of, all right, now that we've got web app pretty much locked in, now that we know what we're looking for, how do we look for other services, you know, SQL and other endpoints that may be a little sketch to be out there? How do we start targeting those and take care of those?

So when it comes to the automated pen testing, and I might just sidebar for a second, I mean, this is what could also be classed as effectively automated red teaming. Is that a good way to think about it? I think it's definitely nuanced.

I think there are some portions that could be considered that. When I think of red teaming, again, with the background that I have coming from the military, I always think of red teaming as this like, you know, very high stakes. I need to move silently. I need to, you know, I'm competing against that blue team, right? If I'm a red team, I'm competing against the blue team. I don't want to get caught.

So Red Team will have to operate and act and make different OPSEC decisions, right? For our continuous external pen testing, for our continuous web app pen testing, I hope you find me. Like, that would be a good thing, right? Like, if you have smart filtering or smart logging to say like, hey, we see this thing, this isn't good, drop those packets, like, that's a win for you. So that's kind of the nuance, I would say.

But the methodology is still there, right? Like, I want to be safe. I want to make sure we're not knocking over infrastructure. I want to make sure that we can get the data that we need out without causing logging to occur because it makes our job easier.

Yeah, I think that's a good nuance. I mean, some of our listeners, whilst very technical, security is probably not something they actually have had to touch a lot. So yeah, the sort of term of red and blue teams, yeah, red being the offensive and blue being the defensive. And I believe there's even at this point now, purple teaming, which is... Yes, there is purple teaming where, you know, the blue teams come together like, hey, we want to see if we can identify this attack patterns, right? The offensive landscape is huge. You're not only competing against White Hat,

Or even gray hat, you know, security researchers, you have black hat actors out there that are willing to cause harm and willing to like ransomware is on the rise and at an all time high, right? So purple team is this way to address that where you take red team folks and say, run your playbooks with us, right? Like tell us when you're going to launch an attack so that we can look at our logging and figure out like, do we have these systems in place to catch that?

So yeah, sorry about the nuance, right? Like red teaming to me is a very like specific subset of offense, but I would say everything that we're doing is on that red team offensive side. No, that's great. I think it's really good to be able to explain these, as you say, it is nuanced. So

So I'm just kind of interested to understand maybe more, how did Cosmos evolve? So I mean, it's Cosmos, I think you said it's now been in, I guess, production for four to maybe five years. Is that correct? Yeah. Could you just walk us through almost, I don't know, year by year, if that's possible? Yeah. Where did it start and like what was added and why? Because I'm just super curious how you take sort of this human process and then transform

decide what and when that's going to get automated and brought into this platform. So yeah, if you could walk us through that. Yeah, it's an ever changing process. I know, you know, since being here in 2020, what we do now is not vastly different, but it is significantly different than what we did, you know, back then. When I joined the team in 2020,

We're operating out of just the goods, right? We're only going to provide findings for things that are highs or critical severities, right? Like this is the pulling your folks, like they don't get to take a weekend. They have to fix this now. Everything else was a notification of, hey, you should probably fix this. And as we continue to work with customers, we realize like there is a wider range of severities, right? Like it goes from informational all the way up to critical for a reason.

So around 2021 or early 2022, we started reporting low to critical. We still don't do the informationals just because there's so much noise associated with that. And again, Cosmos is all about the like validated findings. If we're writing you something, it's something you actually need to go take a look at. So after about being here for around a year, we started reporting these low to criticals. And so that was more work coming in as we were getting more customers. And that's where we started looking at, all right, how do we

identify those vulnerability types that we should automate? Like, does it really bring value? I gave this example earlier, but does it bring value for me to look at a directory listing and say like, yep, there's folders in there with files, right? Like that doesn't really always equate to something good, especially if it's something that is relatively benign, like a CSS folder, right? Like very rarely is there ever going to be anything sensitive in there.

But you do need to be aware of it so you can fix it in case, you know, you have an intern or even somebody super senior that goes to drag and drop and puts it in the wrong folder, right? Like there has been findings that we've reported where like, hey, we found passwords in a docs folder, right? Like that's not good. You need to make sure you don't do that. So those become two findings. And so it really became, you know, out of survival and scalability, what do we need to automate? Right.

And so it is those like these low level, medium level of this is an exposure that you need to fix. This is a system configuration problem. And we're going to save that harder follow on work for the operator cycles, right? We let our analysts make sure the targets in scope and then our operator will do the work to make sure that like, hey, here's the full business impact of what we found in these files as we continue to grow.

Around 2022, we started automating subdomain takeovers. Again, this is one of those vulnerability types that doesn't take a lot of time, but a customer lifecycle with Cosmos, you get onboarded into the platform, my analyst team goes out and identifies all of your attack surface that they can find, right? They use open source intelligence. They use publicly identifiable information to figure out like, all right, this person

is in your org, this person has registered this domain through Whois information, so we believe this domain follows.

We do this work and we would always get this huge spike of trash DNS records, stale DNS everywhere, domains that were for sale that used to be registered and had clear ties to a customer, or just, again, misconfigurations where you have subdomain takeoverable subdomains. And so because we were at a point where we were getting a lot of new customers coming in, we looked at that and was like, "All right, let's prioritize subdomain takeovers." And so we spent some cycles

to report it out as fast as we could because there's this other entity out here that we're working with called Bug Bounty, right? You know, there's a lot of Bug Bounty organizations out there that are also trying to help keep customers safe. And so, you know, we bring in a mature, validated findings. We have, you know, quality assurance checks where there's a lot of like

not a lot of bureaucracy, but there are a lot of checks to make sure that everything you get still is that Bishop Fox standard. Where with bug bounty, there's not always that, right? Like it can be five people just trying to get first report in, so they're not considered a duplicate. And so that's how we've kind of driven our automation approach. It's

All right, what are the low weight things we can knock out? And then what are through a customer lifecycle? What are the high impact things that take up too much time that really don't need to be there, right? Like if I can write a script that can go out and validate this vulnerability, do I need to go look at that? Or can I use my creativity somewhere else? And that's what we always try to lean back on is, is this a creative exploit or is this just something that

With enough time and repetition, we can train and build a good enough tool that can tell a customer about it so that they can go fix it. So yeah, I think a lot for the listener base, the subdomain takeover, I think is super interesting and also very relevant given some news that came out recently around a company called Watchtower, who's actually based in Singapore, being able to take over the .mobi, who is records, I believe, company.

Can you elaborate on why was subdomain takeover so important for Cosmos to focus on? And what does that even mean? And what can someone do if that is brought into effect, I guess? Yeah, it's not the first order. It's the second order effect, right? You know, if I was hosting something, clients.example.com, and then for whatever reason I misconfigured or the application, like the hosting provider that I was using lapsed,

And that clients.example.com instead gets hijacked or taken over by somebody else. It can now become a phishing angle or, you know, a way to like harvest credentials from customers that are trying to log in. And so it's bad in that, you know, it hurts customer, like it would hurt our customers, like public perception and making sure that like we're keeping our customers, customers safe, right? We try to approach it from that angle.

But when we talk about subdomain takeover, is there a reason why the subdomain is more vulnerable? Or say, if you have access to the DNS of a domain, then you can just create whichever subdomains you choose. So are we talking about both effectively domain takeover and subdomain takeover, or how does that look? That's a great call-out.

It's both, right? Like, you know, Azure subdomain takeovers, you're not going to be able to take over a domain that points back to an Azure. But if they misconfigure their Azure instance, then it would be possible, right? Given that the right things appear when you do a dig, you're able to investigate like, all right, this is vulnerable because...

of the configuration that they have. And then you can do work through the API gateway to take it over, right? There are other applications where you go out and you, it's a third party that you're hosting on. And if the third party is vulnerable or you didn't complete a setup,

You know, Shopify is a great example, right? Like if you don't complete a Shopify setup, you're able to come in there and scoop it up. And now that's your subdomain. Congratulations. So there's a lot of examples out there and they kind of run the gamut of, yes, they are harder because you actually need the domain. And that's why we also report on like, hey, this domain is for sale, right? Like it used to belong to you. It would make sense that either you

Make it very clear that it no longer belongs to you or you go out and you purchase it just defensively. There's a subset of offensive security called like brand protection. That's not something my team does, but our customers ask us about it and we try to lean in and help where we can. But because it's like more defensively focused, we don't do it. But we're able to say like, hey, you know, the common things to look for are typos, squatting, you know, off by one character, those kind of issues. Yeah, that's a very good way to explain that.

Introducing Hyte, the only autonomous project management tool. Backlog grooming, bug triage, keeping documentation up to date. Those aren't why you got into product building, right? Well, Hyte handles all that grunt work for you. Using a first-of-its-kind AI approach, Hyte proactively takes care of time-consuming workflows without you lifting a finger. Hyte recognizes when you've agreed to trim scope and handles mapping the necessary edits back to your product brief.

When new tickets are added to your backlog, Hype combs through them, adding feature tags, time estimates, and more. And it's not just you. Everyone on your team manages projects, tracking updates, scoping work, balancing priorities. But whether or not your product succeeds shouldn't depend on project management. With Hype, autonomous workflows handle that mundane upkeep so your team can focus on building great products. If you're ready to stop managing projects, it's time for Hype.

Join the new era of product building where projects manage themselves. Visit height.app slash SEDaily to get started. So when we think about, you know, you mentioned having enterprise customers and yeah, if you go on, you know, bishopfox.com, you can see a whole bunch of names there. How do you approach that when it comes to like companies with such large attack surfaces, you know, whether you maybe want to bring out any examples that you can or just sort of talk at high level about a sort of industry approach.

I've worked on a tax surface, but I'd say probably this was a tax surface of fairly small companies. Maybe the largest I think I worked on was Singapore Airlines. It was large-ish, but I wouldn't say I think for probably the kind of cases that you work on, it was what would be defined as large actually. We did find a couple of interesting things like

tons of staging sites and that kind of thing. And to your point, you know, it's not that critical, but it's, there were various things where you sort of said, look, if someone is looking for as much information as possible to try and say spoof being Singapore Airlines, like they've got a ton of resources now. So you might want to just like lock that one up. So yeah. Talk to me about like maybe some of your largest attack surfaces and kind of what were the actually, because I think Bishop Fox talks about sort of the needle versus the haystack, you know, we find the needle, not haystack.

What were some of the needles, I guess, in enterprise? Yeah, that's a great question. You know, our largest attack surfaces are six to seven digit targets, right? So that is a scheme, IP or domain, and then a port, right? So HTTP on an example.com and then HTTPS example.com on 443, that would be two targets. So we look at it as a whole though, because we've seen through

investigation and the reporting that we've done that port 80 and 443 may serve up different information. And so we want to take that unique approach to each one of them. The way that we find that needle in the haystack goes all the way back to my analyst team.

So they are the ones that are tasked with identifying a tax surface, right? So we do this breadth first, and then we go depth. We start with breadth of customer A has a lot of subsidiaries. So let's go out and identify each one of those. We use things like SEC filings if it's a US-based company. If it's not, we get to research like, all right, how does Germany do their filings for what customer owns different companies?

And so it's a lot of fun investigative work there to figure out this customer that has the service actually has 10 other companies underneath it. And so we do this iterative approach until we get to the point where we're like, all right, we've identified all of the main companies that they would own that have a domain. From there, we start using reverse who is information to figure out like, all right, this company...

generally uses these email addresses or they have this as their org name or their tech name and we're able to just search back through

current information, as well as historical information to figure out like, all right, did they have these domains? And then they privacy protected them. We're going to bring those in and have a conversation with the customer to say like, we found these, are they still active? Because they're private and we don't want to include them if they're not yours. But if they are yours, we do want to include them. So we try to get that whole approach of, you know, we call it domain-based investigations, right? Like we're going to go out

And, you know, yes, we're going to cover your CIDR blocks. We're going to research ASN filings and make sure that like we're grabbing slash 24s or slash 20s and bring those into the platform. But we really want to make sure that we're identifying your domains because that's, again, how most of the things talk to each other on the Internet. From there, we're going to do sub-demand enumeration with lots of different tools. The cybersecurity community is not shy when it comes to creating a tool and throwing it out there. And then we get to go through and figure out like, is this useful or not?

Is there, you know, a nugget in here that's good that we can take and make better? And in some instances, we'll go back and do a pull request to help the community. In other instances, like, you know, we've built a lot of things in-house that, hey, this works really well. And we're going to do this for our service to give back to our customers to make sure that we're finding as much as we can. We let the platform help with, you know, reachability checks, resolving checks. So we never cut off a target too soon, right? Like we give a target several chances to become validated and become something that we look at.

So it's a really long-winded way of saying we do a lot of prep work to build out a haystack that is less haystack and more needles, right? Like I know if I'm looking at something, it's definitely theirs. And then from there, I leverage my T team. They're like my threat enablement team.

And they create scanning engines. They create, you know, things that we're going to go look for that are threat informed. You know, we use the CISA known exploitable vulnerabilities list, right? Like that is one of our driving forces because customers care about that, right? I may not care about this thing that a security researcher found, right?

because it's probably not going to be out there. But I know that my C-suite or my senior executives are going to be asking about these things. So again, we always want to partner with the customer, meet them where there are, you know, get those baseline things taken care of.

And then we can start working on, oh, and by the way, here's this, and here's this, and here's this. And we continue to get better that way. So it really goes back to, you know, good attack surface discovery and then looking for the right things so that when the operators pick it up, they're guaranteed to have a needle. That makes a lot of sense. I mean, when it comes to the, I guess, continuous...

aspect of this and sort of continuous i'd almost like analogize with like being proactive versus reactive in the sense of you're able in theories as of as it sounds continuously

understand the attack surface and I guess match that up with, especially with any newly declared vulnerabilities. How does that work? I mean, it must be a lot of, I'm just talking total layman terms here, but like a lot of filtering going on where you're like, we looked at this yesterday, so we're not going to look at it again today. Or like, how does that work? Yeah, definitely. That is one of the hardest things that we've had to solve for.

We say that keeping attack surface under control requires constant vigilance, right? Like it is a whole party team, right? We do that by my threat enablement team. They track emerging threats. And so again, we use the CISA KEV.

to help guide that. But we also look at things like Microsoft Patch Tuesdays. What are the things that they're reporting on that may not be a news article yet, but it will be once other people start researching. And then you brought up Watchtower earlier. They do some great write-ups. And so it is looking across the cybersecurity landscape to figure out who else is doing really cool research. I know Bishop Fox, we frequently do blog posts of like, hey,

I have to shout out the capability development team. They're awesome. They're always researching those next things. And so we really try to leverage what they're finding as well as what other researchers are putting out there to figure out, even if there's not a proof of concept and an exploit to go with it, what are the versions that we need to identify? What are the things that we need to tell customers about? And so that's where that continuous cycle comes in. We're

continuously looking for attack surface to bring in. We're scanning customers to make sure that we have a right picture of their attack surface. Because if we're only scanning once a month,

That's 30 days where something can change. And so we need to scan more frequently than that. We need to make sure that we're staying aware of the changes in the nuance of versioning to make sure that if an emerging threat drops for version 5.2, we're not going out and reporting to customers 5.1 or 4.8, right? Because again, that's not delivering a good service. And we want to keep this high signal to noise. Like if you're getting a notification about an emerging threat, it is

hey, this is important. Try to do this before your Friday kicks off, right? Which is unfortunate because a lot of emerging threats drop on Fridays. It's a classic joke. Humans are humans. Humans are humans, right? They got to wait till Friday. But that's how we get after it, right? Yes, there's a lot of filtering.

With AI and LLM becoming a thing, computers are good to talking to other computers. So knowing that a CVE comes in and it has a CVSS, a score that says this is a local privilege escalation.

we can use an AI to help. It could be a simple regex, but we are leveraging LLMs to help make sure that like, all right, why is it a no? That way if a customer asks and they give us a random CVE string from 2022,

We're able to look in that and say like, hey, you know, we're not going to cover that because that's not an external unauthenticated. That's a local privilege escalation. So like you're going to care about that once they're already on your machines. That's not something that like we're going to identify from where we sit externally at scale. And I think there's mention of sort of like a live collaboration feature on the platform. And I think a lot of what you've just been speaking to

The client needs to be involved, you know, to be able to, you know, because I think quite a few things there you say, we know it's yours, but perhaps like, can you clarify X, Y, Z? I'm, you know, from my experience, like we had a scan done by someone on past company and they found, you know, a Notion instance where,

with the same name as our company, but it wasn't ours. But they absolutely thought it was ours. And that platform did sort of enable me to say that is not ours. Like I can guarantee that. So is that what the collaboration aspect is? Yeah. So our platform allows customers to do that with or without us, right? Like they're able to say it like through the platform, they're able to see their targets. And I can say like, actually those things,

they don't belong to us. And we're able to not scan those things and not report on those things. But we always tell our customers, hey, we work best from an informed perspective, right? Like if you want to get the most out of this service, lean in because we're here. We're here to help. Like we want to keep you safe. And the best way to do that is open communication. You know, we've had some customers come in and with folded arms, like, hey, we're not going to help you. You figure it out and we'll still deliver value. And we're still going to do a good job. But those customers that meet us where they are,

and they're able to say like, this is everything we think we own. What do you see? Right. That is always a better conversation because we're able to say like, Hey, yeah, like we found all of those things, but you've missed, you know, this section or this company business unit that you may not know you own yet. But according to the news and lawyers, you do own it. Why are some customers, I mean, like that's an analogy or not analogy, but you know, come in full with arms, like why do they have that attitude? Security has been around for a while. Right. And

I think we're at a stage right now where we have to work like crazy.

offensive security has to get back some goodwill, right? Because it is so easy to walk in, punch somebody and be like, hey, go fix it, right? And frequently that's what it used to be like, right? And so in people that have worked in the industry long enough, it is like, I don't want to give you anything because if you find it, I'm going to get in trouble for it, right? But instead, if it is like, hey, we're going to find it and we're going to make sure that we work with you, it goes better, right? And so part of our investigation life cycle always

It should always end with a remediation test, right? Like, you know, we're going to work with you until it's fixed. And I think that is like the one thing that we try to shout out to customers when we get to talk to them. It's like, hey,

You know, customers will fail a retest and we'll give them specific reasons why. We'll jump on calls with them to help. Maybe not troubleshoot because, you know, like we're not developers, but we are able to attack it with our mindset and help them work through the problem. And so most customers that come in with arms folded by the end of it, you know, they see the value, they see my team and how much they care and we get through.

I think that's a really good call out. It's just, yeah, you know, in days gone by, pen testing kind of done by a team that maybe was never seen before. And all that happens is someone gets this report on their desk kind of saying like tons of problems. And quite frankly, they're just like, why did we have to find these problems? Could we? Yeah. Could this not just happen like next month or next year? You know, I'm about to leave. And now it's going to be the next guy's problem. Exactly. I think it's just as you as you call out, it's.

the mindset of what a service such as Bishop Fox can do, which is it's collaborative like

end-to-end and security if it's the right security partner it should be a collaborative exercise end-to-end and and obviously confidentiality is just such a huge part of that which is you know saying look we'll be as open as we can as a client with you but at the same time it's we have to trust that you know this stays within the walls of of our relationship and you know even as working at a security company previously like we had a third party security

company come and audit us. You know, there's just, that's just the way it works, but you have to obviously trust, you know, if one company is auditing another security company, you have to really trust that. So I think it's a really good call out. So one thing I'm interested in is within the industry anyway, it's no secret that it's a very stressful job, like anything to do with security and finding vulnerabilities or like helping remediate. Would you say like introducing Cosmos within Bishop Fox has that sort of enabled the

I'm not saying it was a stressful environment, specifically in Bishop Fox, just more in the industry. Has Cosmos enabled sort of it to become a more manageable profession, I guess? I think the short answer is yes, right? The long answer is always it depends. When you get attack surfaces for extremely large customers, you see the amount of work.

that exists, right? We call it our backlog, right? That's a pretty common name for the way to refer things. But to make sure, it's always that added pressure of, did we prioritize all the vulnerabilities that we see? Because at the end of the day, because we're not a scanner and we're not fully automated and we want to keep that bar to tell a customer to go fix something, we want to go look at it. There's a backlog of work. And so it's that the stress of the unknowns of like,

I think we prioritize all these right. And I know that we're working in what we think is the most important thing. But what if, you know, there's this random offshoot and there's actually a critical vulnerability that may be hidden as a low. And, you know, the way that we approach that problem is like, all right, let's attack it from all sides, right? I have my operator teams going priority first. But then we also look at things that like, all right, what's our oldest vulnerability?

potential flag. Like what's the oldest thing out there? And let's go make sure that we're staying in line and not letting things get too old. Because the worst thing that would happen is something is vulnerable. There are passwords out there and then they take it down on accident, but then those passwords don't get rotated, right? Like that's still bad. So we want to make sure that we're attacking it from both angles. So that helps with the stress at night to know that

If it's not the most important thing, we're going to get it on the back end and then we're going to be able to tell customers like, hey, we've identified this a while ago, but because of priority thresholds, we haven't found it, but now we did. And now we can work together to make sure that it is remediated and we're able to give those timestamps to customers like, hey, if you are really that concerned, you can spin up an IR, an incident response and make sure that nobody beat us to it. But that is one of the things that we look at is when we land remote code execution, we're looking to see

Did anybody else beat us here? Because that is incredibly important because it helps build the picture for the customer of risk, but it also gives business impact, right? If I'm throwing an exploit that puts a file on disk and I get there and there's 20 other files, like, hey, you need to fix this like tomorrow. So that's how we approach it. Yeah, it's still stressful, but I think it is more manageable knowing that we're trying to approach it the best way we can. On a traditional consulting engagement, you

You scan and then it's kind of on you to figure out like what looks the most important. And because we're leveraging, you know, five plus years and lots of operators and analyst input, we're able to say like, we believe as a service, we're looking at the most important things. So it takes off that stress off the individual and really puts it, you know, on the shoulders of the leaders to make sure that we're making those right choices. Yeah. It's not as if Tony Stark is not stressed just because he's inside the Iron Man suit. Exactly. I like that.

I think that's a good call out. I mean, technology will help. It will, at the same time, as you call out, by now getting to almost have these sort of x-ray vision that can lead to maybe more sort of thinking about what to prioritize, et cetera. And that can potentially just be a different, it's just a different stress point potentially. Moving on just to kind of

The final area before I just ask about some future things, regulation and things like SOC 2. SOC 2 has become so, dare I say, popular. You've got companies like Vanta who've made it, in theory, I wouldn't say easy to get SOC 2, but they have definitely streamlined the process big time.

How has that sort of impacted Bishop Fox generally? Is that sort of being a great driver of business or is that kind of actually being more of a problem to have to like think about how to adapt things like Cosmos to be in line with regulation drivers or like just how's that sort of evolved? Yeah, that's a really good question. A driver for business. I don't know if I can speak to that. That might be more, you know, more of my sales team's perspective on is this driving them to

to help with customers that are SOC 2 compliant. I know for us, like the work that went into it to make sure that the Cosmos platform and all of the operators, analysts, and engineers, like that we are doing the right things. I know that that was like good for us to make sure that

you know, the common saying is like, you know, everybody's building a plane as they're flying it. But I would say SOC 2, like it forces you to figure out like, hey, here are the non-negotiables and how do we make sure we're doing these right so that we can continue to fly and do the right things. So I don't know if that completely answers your question, but that's kind of my perspective on it, just with my limited interaction with the work that went into it. Yeah, no, for sure. So I think the internal aspect is very interesting because again,

perhaps it's not always clear to maybe non-security industry people that just because you are a security company doesn't automatically mean that you've got, you know, sucked to or something. You should still go look and check. That is correct. It was a lot of paperwork, a lot of interviews, a lot of making sure that we followed best practices. Yes, exactly. I guess I was just also interested if with, let's just say current clients, as opposed to sort of anything to do with driving new business, but with current clients has, do you get questions about

to do with, well, if we don't fix this, does that make us uncompliant with SOC 2? Or is that, do you get those questions? Or does that actually go to kind of, I guess, that would go to a Vanta or something? I think those questions get driven internally by the customers, you know, infrastructure and IT teams. You know, we're able to say this is a problem and, you know, you have problems

PII or PHI exposed. And so to me, that means you're already failing and you got to go fix that. And some of it is they're just not aware that it's out there. None of our customers currently or historically have asked, "Hey, does this vulnerability, does this finding affect our SOC 2 compliance?" Instead, it's just like, "Hey, how do we help take care of our customers' data?"

Yeah. Okay. Makes a lot of sense. So just kind of, I guess, looking ahead, what can you share about sort of how, where Cosmos is going? Like, what are you guys thinking about in terms of, when I talk about new features, I guess the customer kind of sees features like they have minus settings, they have a UI that they interact with. But I mean, a lot of, I imagine the development is kind of more like under the hood and what you actually expose to the customer is...

It doesn't maybe always look new as such, but what can you share about that? Yeah, this is actually a really great question because we are at this precipice of moving off of our old Cosmos platform that has gotten us to where it is, gotten us to where we are. It's successful. It's able to generate information that my team can pick up. We work the cases. We do that work and we report it out to customers. Once we kind of launch into our next phase of our Cosmos evolution,

There's going to be a lot more push for self-service for customers so that they can go out and take the data that my team has found and dissect it and look at it in different ways, in ways that make sense, where they're able to determine confidence values or why this asset exists in their endpoints. Like, why are you showing this to me? All of that confidence information is going to be made readily available to the customer so that they can self-service and figure out like, oh, I'm going to do this.

you know, this subsidiary, they're making it too easy for everyone to go out and find our assets. We're also going to, it's kind of like two answers for reporting, right? So the Cosmos is kind of the portal and the service and everything that goes into it. And then we have different service lines. Our Chasm, which is kind of the thing that started all of this, is our continuous attack surface management. That's going to continue to grow and improve. You know, like I said earlier, we focus historically on like web apps,

and FTP and SSH. And we're going to continue to grow that scheme and that list of things that we're not only showing to customers through the portal, but we're actually providing investigations, right? Hey, you've got anonymous login for this service that you should definitely turn off, right? We do that for FTP. We do that for other logins, but to make sure that we continue to grow that so that the Cosmos Chasm service is kind of that, here's everything from a threat-informed perspective.

We recently just launched our Cosmos application pen testing. We call it CAPT. And that's web app pen testing that is delivered with some upfront testing, similar to a consulting engagement. But then they continue to test those assets through the lifetime of the contract, looking for things like emerging threats specific to frameworks and doing tail testing to make sure that like, hey, has anything changed from the last time we hit this application really hard?

So that's our Chasm, our cat. And then we have a Cosmos external pen test. And that's meant to be a point in time external pen test, you know, similar to consulting, but delivered through our Cosmos platform where we take everything that my analyst team on Chasm has found and we've used that as the scoping. And so, you know, the future of Cosmos is continuing to be operations led discovery and

relying on the customer to work with us and partner with us and provide feedback, but to continue to grow and improve what we look for, what we report. And then there's a whole new world out there with AI and LLM of how do you discover things faster? How do you help rip through SEC filings so that you can generate that subsidiary list faster?

faster with better confidence. Yeah, I think we don't have time to dive into, I guess, what could be done with ALMs and AI today on this one. But I think you've mentioned it a couple of times today, which is a good call out, which is effectively replacing things like Regex with an ALM lookup. Now, obviously, from a cost perspective,

That's expensive, but at the same time, in many areas, that's going to give much actually clearer data and be able to react kind of faster effectively than a regex. So it must be quite an exciting time for you guys thinking about integrating those. 100%, yes. I'm very much looking forward to it. And then, you know, I'd be remiss to say, as we continue to grow, I think emerging threats are going to be the thing that continues to set us apart and set continuous pen testing apart.

How do you stay up to date and do what threat actors are doing? Not only with the old things that we know work, but also the new vulnerabilities and the new things that are dropping through either, you know, white hat security researchers or, you know, this APT got busted by a government. And now that TTP, you know, that tactic or technique is now available. How do you leverage that to keep customers safe? I think that's going to be super important as we continue to grow.

Definitely. Well, just as we wrap up, I sometimes ask either two or one question more back to you yourself, Mark. So what do you know now that you like would like to have told yourself when you were starting out in this industry? Ooh, man, that's tough. You know, it's tough because I started as an officer, you know, so many years ago where I

We were trained to lead and think a certain way. And a lot of that has done me well, but it's also good to know to be technical and to be able to work alongside some of the sharpest people I've ever met in my entire life. And so, you know, something I would tell myself going back 15 years is embrace that hard training and know that it's worth it. Because from my experience, those individual contributors will lean in and partner with you with a leader better than

if they're able to speak their language and understand the problem set that they're trying to solve. I've had a lot of leaders and managers in the past who, because they didn't understand the difficulty, would oversimplify the problem set. And I think that's something that leaders need to be aware of. And then, you know, for individual contributors, it's continue to embrace the hard thing because that's where you learn the most. Okay.

I love that. I really like that. I can definitely side with having not gone through any, I guess, military background myself, but was in a company that was started by US military operatives. And I think it was just having that mutual understanding is what would drive things forward very well. You know, I don't go for a military background, but I understand a lot is learned there. Meanwhile, I come from a technical background.

and i hope that sort of a lot was learned on on that side as well and i think the sort of the collaboration the combination was what made that work so it was a great call out mark thank you so much for coming on giving up your time in an evening over in north carolina today i really appreciate it yes sir thank you again for having me on man this was a blast yeah thanks a lot and we'll catch up soon i hope yes sir thank you