MLOps at JFrog with Bill Manning
50:24 Share

JFrog is a DevOps platform that specializes in managing software packages and automating software delivery. One of its best-known services is the JFrog Artifactory, which is a universal artifact repository. JFrog is also focused on rapidly emerging needs in the ML Ops space. Bill Manning is a Senior Solution Architect at JFrog.

He joins the podcast to talk about his background in startups and venture capital and his current work in ML at JFrog. This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him.

Bill, welcome to the show. Well, thank you so much for having me, Sean. I'm really looking forward to this today. Yeah, absolutely. Thanks for being here. So I'm excited to talk more about JFrog and DevSecOps and some of your areas of expertise. But first of all, I wanted to dive in a little bit of your background. Who are you? What do you do? Explain your role at JFrog.

Yeah, absolutely. So I've been with JFrog coming until almost eight years now. Prior to JFrog, I've been doing this for a long time, started my career around 97. Yes, I'm not old and that's what I do. But like I've had a very successful run. I've had three acquisitions. One of the things I've done since I began my career was always pick different technologies every time.

First company I had was a first web-based CRM platform. A lot of the people that were there went on to form things like Marketo and SugarCRM and Salesforce. Next company I did was email encryption and security, and we sold that to Cisco in 2006. Third company was IoT. Before IoT was even a thing, it was called 4Home. It was the connected device company. We sold that to Motorola and Google in 2010. I was a venture capitalist for a bit. I was with Vodafone Ventures. I was senior media partner.

Did another company called XTV about media consumption, took that public in Australia. Did some other work around after that with some startups. I'm also a mentor. And then I joined JFrog a little over eight years ago when they were midway into the company. And I had some friends here and they said, we'd love for you to come in and give your input. And since then, I've been a solution engineer, solution architect, managed teams here. I do everything from a little bit of marketing to sales to you name it. Currently, right now, I'm transitioning over actually into machine learning.

And so our JFrogML platform, which we just launched this year with the acquisition of Quack, I have now joined that team on a go-to-market strategy. And on top of that, also to education and things around that. So that's me. Yeah.

That's awesome. Yeah. I mean, it sounds like you have a whole breadth of experience so you can kind of be the ultimate gap filler regardless of what the problem area is. You can jump in there and add something to it. Jack of all trades in a way. I learned in my startup years how to do everything from legal to marketing, you name it, because I had to. No choice when I had to do it. Yeah. I mean, I had a similar, I founded a company and

you know, and I was the CTO of that company and had mostly experience in engineering before that. And really the way I got my business degree and my, you know, marketing diploma was because there was no else to do those things. And you're just like forced to do it. And I think it's,

taking me on a career path I probably never would have done if I hadn't been a founder as part of that experience. You got the practical MBA, as I call it, right? You got the hands-on MBA. The things that they teach in the MBA school, you learn the hard way. Yeah, exactly. I made a lot of stupid mistakes. Oh, yeah. I'll be the first to admit it too. And I'll be like, oh, that was dumb. Yeah.

So you had quite the run career in terms of picking the right kind of companies to be part of or companies that have had successful exits. What is your strategy there? How have you been able to pick these companies? I'm a very weird being in a lot of ways. A lot of friends say that about me. People have known me in the industry. One guy from a certain venture capital firm, I won't.

say who he is, but he calls me his lucky rabbit's foot. I seem to be able to sniff out technologies in advance. And sometimes I actually have had some of the technologies that I've worked with or, you know, with companies we've started or big things I've done, or we might have been slightly ahead of the curve. I have a tendency to be able to sniff things out and I don't know what it is. There's no real strategy. It's a feeling more than anything. And it also stems from just my inherent, you know, the reason why I got into this industry in the first place is the evolution, the change. It's not stagnant. It's always flowing.

And that was a huge kick for me was like, do I want to be in like here? I am 52 now. But like when I was in my early 20s, I was like, do I want to be the person that's doing the same job and gets a pension at the end of 50 years? Or do I want to be on the bleeding edge of stuff all the time?

And always be learning. The thing is, is I like the fact that I'm in an industry where I have to constantly keep my skills honed. I constantly have to be learning what's out there. You know, if you don't do that, you're not going to survive, especially in this industry. I mean, like I said, at my age, you know, we're in an industry of ageism. And the thing is, is I hate to say it that way, but it's true. But, you know, that's the thing is where I stay relevant is by staying relevant, you know, staying up with the trends, learning the details.

Not honing on skills that I learned a long time ago as being bedrock and solid, but always evolving and changing. Yeah, I've certainly worked with people, especially on the engineering side in the past. They have their toolkit, and they don't necessarily want to expand that toolkit. They want to go and they apply that toolkit, and they can do it very effectively, but that's what works for them. I think I'm more also...

Aligned with you, where I've always been driven by sort of the educating and pushing myself to figure out, I want to work on the edge of technology, essentially. And that's why I spent a long time, too long in school, to educate myself and really work on the edge of technology.

Come on, let's think about it, right? I mean, you know this. The thing is when you read about something, you start learning something, and then you start applying the practical portions of it, and you get that first light. We used to always say in our first – every startup I've done, I'm like, what's the first light where you get the product to be yours, that first moment where you all kiss it around and look at each other and say –

damn, we did it. You know what I mean? Like this is the start. We know this is something. And I still love that. Like I said, right now I'm working on a couple of things, you know, outside of JFrog to kind of keep my skills up. And when every time I have one of those little epiphanies and actually one of the companies that I've had was epiphany, but one of the epiphanies is like when he gets to that moment, you go,

Okay, yeah, you know what? This is practical. I get it. What's next? And then in my head, whenever I've done this in the past, my floodgates open on what's possible. And also, too, what are the restrictions? What are the constraints I'm within and how do I break down those constraints and barriers?

Yeah. So what's kept you at JFrog for eight years now? Number one, the people. I'm going to be honest. I love the technology. I love the platform. I get to work with some of the biggest tech companies in the world. That's one part of it. But the thing is, is I love the people I work with. You know, like every other company, you have to be able to work with these people you're around, right? You need to be able to get along. And we have very interesting hiring policies here. You know,

Are you a frog or not a frog? Right. I thought in the beginning I was like, oh, that's cute. You know what I mean? But now I've learned over time that that's actually a real thing. We have quality on who we bring into the company. And the thing is, is that, you know, I'll tell you some of the hardest situations we've had here. I've had some of the most delightful and fun discussions because, you know, in some cases when things get tough, humor is the best way to alleviate some of that strain.

And being able to have people that are in the same similar mindset where you can have that is exceptional. And so for me, having people of the same mindset around me has been one of the major things that has kept me here. Secondly, like I said, I get to work with some of the largest companies in the world and always be on the edge of creating something that will do something. And the thing is, too, is I always make the joke whenever I talk to, say, customers or prospective customers, especially if I'm a consumer of their products, that

I want to be able to say, hey, you know what? I'm here to help you make yourself better, compliant, safer, because I'm your customer, right? I want to ensure that my own personal data, my own personal safety is a top notch. And this gives me a chance to have some sort of influence into that.

And so it's a lot of different things. And also J frog has afforded me the ability to be out there, right? The big joke is sometimes people say I'm one of the faces of J frog. I do a lot of our public speaking and webinars and things like that. And I enjoy that for a person who gets stage fright. I enjoy it. And,

And the thing is, so there's a lot of different aspects. But like I said, it also encourages and nurtures employees here to take those steps and keep going. And I also love being able to mentor others like I was mentored when I was younger and bring my perspective. And there's a lot of people here that I think that I love having those conversations. I love the diversity. I love every little bit of it. And that's one of the things that keeps me here, to be honest. Actually, this is the longest I've ever been at a company. I've been one of those ones that's constantly evolving and going.

But this company constantly evolves and goes. And so it falls in line with my mantra. And it gives me the ability to also excel and exceed with the product. And like I said, work with large companies and work with companies that are making a difference. I always make the joke that people ask me, like, yeah, from Chick-fil-A to SpaceX, you know, like we have this whole large swath of things that we do. And that's exciting.

Yeah, that's awesome. But you know, John Maxwell, I think said, you know, people quit people, not companies, I think is a quote. And it's true. Like,

I think very few people are going to stay at a company that is up and to the right if they hate their day-to-day and they can't stand their coworkers. It's just going to be a miserable, maybe some people are willing to do that, but it's going to be a miserable existence. And the great thing about working in technology, especially on the technical side, you have a lot of options. So you don't have to kind of put up with that stuff if you don't want to.

And I could tell, you know, you seem very passionate about this. I can understand why they have you out there as the face of the company from time to time. Where was JFrog in terms of like their size and development eight years ago when you joined versus where it is today? Oh, absolutely. So when I joined the company years ago, I mean, I was employee number 123 and now we're up to almost 2000 employees. You know, we were like 35 million in revenue that year when we announced it. And, you know, and since then, you know, you can see where we're at now, you know, heading towards 100%.

massive revenue and scale. When I joined, we were just shy of, I think, about 2,000 customers utilizing our platform, which was exciting. And now we're almost at 8,000. The thing is, is that over time, watching this grow and expand and have different locations around the world, it's fun to be in the roller coaster as it's going. And in this case, the roller coaster doesn't have an end. It keeps going. But it's got thrills and chills and ups and downs. I

But the thing is, though, is that you're there constantly and watching JFrog grow and being there in the front seat with some amazing people that have gone along for the ride with you and having that shared trench story. You know, you're in the trenches together. Times are tough. Times are good. You know, for me.

It's been exciting watching the growth. And the thing is, we've adapted and we've always been kind of at the bleeding edge of what's next in terms of that. And so we're able to meet the market where it meets. You know, in some cases, we might be a little ahead of the curve, but we're laying the groundwork, right? And that's the thing is, is that like now, like even for me, jumping into, you know, this whole idea of JFrog ML and ML Ops, and it's exciting. You know, every company is going to have AI eventually integrated at some point or some sort of machine learning into their application.

It's already starting. It's been starting for a while. But be able to actually have and I look at where that is compared to what DevOps was years ago, where it was this massive playing field of all different technologies. And with Jfrog, one of the things that also attracted me was the universality that we've actually embedded into our messaging, which is like I always equate it to we're the most perfect conveyor belt.

Right. Raw materials on one side. That's the raw code, the third party transit dependencies and that and some sort of manufactured product at the far end. And we're in between there. We provide that layer of consistency. And the thing is, you have all these different tool sets. And the thing is, is I always had this map that I always show. I'm like, here's the terrifying realm of DevOps. Right. Every single tool in every single category from ERP to like CICD to security. And then platform engineering came in.

It was like, okay, let's start wrangling this down into some sort of consolidated platform. And it's the same thing going on right now with ML. You look at all the different tool sets, all the different things for tuning and RAG and all these things, and we're trying to address that same level of market. So for me, watching the company evolve and say, hey, you know what?

We want to be that layer. We want to be that base foundation, the pillar of technology that you build your software on. And whatever you choose, because every software company is different, and that's exciting too, by the way, is the fact that this company has evolved with that and preached about the idea of we're going to help make it more efficient, more effective, more secure. We're going to be able to deliver your software better.

You're going to be able to go ahead and update those devices in the IoT realm. Now we're saying, okay, you know what? We're going to bring compliance. We're going to bring speed and accuracy and efficiency now to ML and ML ops. Because the thing is, let's face the hard facts. 85% of ML technologies out there that are developing, 85% of those never make it to production as they're doing this. And how do you increase the fact that that should be lower? It should be half that or even less than that. And you should be able to say, okay,

Let's build something. Let's evolve it. And like I said, for me, JFrog has been an extension of that, of that natural trend of the actual industry and our ability to be that consistent base layer that allows them to do what they need to do as a company and ensure that they have a consistent set of tools, accuracy and efficiency. And I'm fortunate enough to work with people that want to provide that to them.

Yeah. I think to your point, there's so much new tooling essentially going on in the ML space right now. I was looking at even just with agents, like the agent market landscape chart the other day, I think it was maybe from Menlo Ventures or one of the VC firms.

There was easily 200 logos on this thing. And agents is like the new, new thing in Gen AI. If you expand that to the entire plethora of stack, it's like thousands. I work in the space and it's hard for me to even stay on top of it, let alone somebody who's just like thinking about, oh, like maybe we should kick off an AI project, like where do you even begin? And even now, right? Even those are starting to segment, right? So even those layers, that landscape are starting to segment.

And that's the thing is, is that initially when people like have this mass, there's always a way, right? There's always this massive umbrella of what something is. And then inside of that umbrella, then things start to segment and factions start to come about. And like I said, agents, in my opinion, is the new battlefield. When you look at it, right, everybody's vying for the perfect thing. And the thing is, is that when you look at it, it's like you said, it's now when

Whenever technology, this is the way I always look at it, right? Is that when there's a segment of technology that gets more involved, people see that this is actually a value add. They want to get into this space. Ideas start happening. It's not a bad thing, right? I mean, this happened with the dot-com bubble, right? It was like, you know, initial thing that happened where people were like, hey, you know what? This is new. This is exciting. Retail, commerce, education, information. And then it got into these segmented markets of little bit players all over the place.

Then when it's in the next part, like mobile, same thing, all this. Now, ML is the same way. It's like here started off with a small consolidation of things. But then people go, I got an idea. And then the ideas start to spread. And then you're going to start to see is that 200 logos probably shrink down to 15 or 20 logos and then eventually work its way down to like five or six major players. Right. So it's going to shake out. There'll be mergers, activations.

People will drop off. Some tools will come across as strong, but they're inferior. You know what I mean? There's that natural shakeoff. And that's what the other part, like I said, one of the other things I love about this industry is that if you want to look at natural selection and natural ability of like, you know, our industry is a living industry.

And I love that, the fact that it does change, it does evolve. You don't really see that with CPA work. Yeah. And I think, you know, I've been in industry long enough and you have as well, where we've seen multiple cycles of these. Essentially, whenever there's a new sort of shift, there is like this massive expansion, essentially, of players. Even you go back to social and people who are around for this might not remember it.

But like there was endless, you know, essentially takes on social networks. You know, Facebook overtook MySpace and sort of, you know, leading the charge there. But there was a lot of competitive players that had their own, you know, takes on that. And now we've really like consolidated the market to, you know, a handful of players.

I was like, do you remember Friendster? Yeah, exactly. Friendster couldn't scale, basically. Their MySQL database fell over, and they didn't have the tech talent to scale it, and they basically died. Right, and that's the thing is actually if you look back at it, I was talking to a friend about this recently. It's so funny you brought this kind of stuff up because I was talking to a friend about this recently. We said, actually, if you look at the design and everything that was around Friendster, it was actually a superior platform in terms of usability and design.

But, like you said, the back end of it couldn't support the heavy weight in which they enacted. They had a bunch of brilliant front end, a bunch of brilliant UX and UI and interoperability people. But on the other side of it, they couldn't compete. And the thing is, by the way, remember that was early. There was no real cloud scale. There was no real Kubernetes. There was no real…

way to provide an infrastructure without physical service. I mean, I make the joke of like trying to explain to younger engineers. I'm like, you don't understand. I go, I had this one startup where we needed servers. And I literally went to some really sketchy location in Fremont to some warehouse where some guys sold me 10 servers. And I had to scrape the DOJ stickers off.

off the servers and wipe them out and bring them back to my server location because that's all we can afford. And now it's like I can go in and be like, I can just auto scale anything I want. I mean, it's amazing. Once, you know, I've always been a big fan of adoption of things like cloud native and all that. And like I said, one of the things I really enjoyed about coming here at JFrog was, you know, we're governance board members of the Cloud Native Foundation helping define

that roadmap for all these companies that have that successful scale, not to fall into the Friendster trap of tipping over at that 1 million and one user, right? You know, plus one that just killed it all. But on the other side of the two is, you know, to work with a guy like we have this guy, his name is Remus. Remus is one of the co-creators of Helm. He works here. Amazing, right? You know, it's like royalty, in my opinion, like other people are like, oh, that's cool. I'm like, that's super cool.

Like, I get to see the guy who's like, what was your inception? What were you thinking when this was coming about? How did this orchestration? I had so many questions. I was like a fanboy. That's awesome. This episode of Software Engineering Daily is brought to you by Jellyfish, the software engineering intelligence platform powered by a patented allocations model and boasting the industry's largest customer data set. You know, one of the biggest shifts in the past year or so is the adoption of Gen AI coding tools like GitHub Copilot.

and engineering leaders are trying to figure out if their teams are actually using it, how much they're using it, and how it's affecting the business. Are they shipping more? Is more roadmap work being done? How do you know beyond anecdotes and surveys? That's why Jellyfish introduced the Co-Pilot Dashboard in 2024 in partnership with GitHub.

and since then they've analyzed data from more than 4,200 developers at more than 200 companies. Want to know an interesting finding? Developers are shipping 20.9% more with Copilot. The folks at Jellyfish have a ton more insights, and you can get started seeing your own team's data so you can plan and manage better. Learn more at jellyfish.co slash copilot today.

Yeah, I mean, I think that's one of the great things about working for some of these, you know, larger tech companies in the Bay Area. It doesn't have to be in the Bay Area, but you do send a run into some of these people who've invented technology that's like widely adopted as an industry standard. And they're just there as an employee and you can like talk to them and pick their brain. So I want to transition a little bit to get into, you know, some stuff on DevSecOps. So

Just to start, like for those that are maybe less familiar with the space, like how do you define, you know, DevSecOps and where does it kind of fit into a modern software lifecycle? Oh, absolutely. It's essential in my opinion, right? The thing is, if you're not adopting this ability of constant, you know, here's the thing is, you know, we've always talked about whenever I discuss DevSecOps and I get a lot of talks on this, you know, over time, I always start off with just the speed of build.

Right. Like when I started in the industry, we did quarterly builds, you know, we did quarterly releases. I mean, you know, we did builds and we did testing, but we only released our software, you know, once a quarter, you know, and it was a big announcement. Hey, we spent we spent, you know, 90 days working on this piece, you know, this upgrade to what we're doing. And yeah, you know what? We build it occasionally, you know, we test it and things like that. And then, of course, you know, everything started to progress, you know, built up.

Cycles got faster as automation became more available. And then, of course, you know, in 2006, when we started getting into the whole idea of DevOps, right, you build it, you run it. And, you know, tools like Docker and orchestration and all that, you know, once again, it was a Wild West show. You know, you looked at all the tools that are out there to do something. But the thing was is that there was this essential idea of DevSecOps, you know, being able to have faster release cycles, being able to have security enabled into it, right? Originally, it was just DevOps.

You know, the security came and said, you know what, we should really do something to make sure that things are safe, secure and compliant. And the thing is, is that having these iterative cycles, having the ability to create things that allow you to build more rapidly, deploy more rapidly is essential. Right. I don't see organizations.

living without it. It's funny is in some cases you'll see technology, well not technologies, but more like trends like DevOps or others ebb and flow and change or disappear and they evolve into the next thing. Look at Agile, right? I mean, Agile came about and now people is like, is Agile dead? You know, it's all these things, but DevOps and DevSecOps is essential for speed, accuracy and go-to-market.

And having the security behind it ensures your ability to perform for your customers, non-dependent of silo of industry that you're in. But by providing not only the best software, but the most secure and compliant software, which is, you know, we know, as you know this, you look at the trends and the numbers, you know, like the number of increased in supply chain attacks that have happened in that compromised people's software. We always go back to solar winds. It's still, you know, it's a use case that we studied for decades.

And that was a fourth level transitive dependency in the cycle. I mean, it was deep, deep, deep into the recesses of this actual application. And it caused $100 billion worth of whatever problems around the world. But the thing is, is that companies need to adopt this because it allows them to automate more.

Do more. Make sure the things that they're doing are correct and they're compliant and safe and secure, ensuring the stability, ensuring security, ensuring resiliency. You know, all the words that you hear when people talk about DevSecOps are true.

You know, the thing is that you need to adopt them in a way that works best for your company. And at the same time, it adheres to some sort of level of standards that can also be applied that when you bring people in, you're not bringing them into some jarring sort of thing. You're bringing them into something that's similar or something that falls in line, at least with minimal deviation, what the definition of a DevSecOps is.

Can you have DevOps today without security? Like, are these really two independent things or should it just be, you know, one concept? No, they should be the same thing. Here's the thing is, I see this a lot. There's a lot of companies that are like, we'll talk security with them and they'll be like, well, we need to bring the security team in.

And some of the questions I always have when I email the security team is like, how much delay, how much hindrance does it put on your organization by having a separate security team? The thing is, is that here's an example.

You know, everybody talks about shift left. Where do you put the tooling where it matters most? And where it matters most actually is at the developer level, right? This is your entry level. Now, you're not going to have a security person sitting over their shoulder every time somebody codes. You know, this isn't a bullpen, right? Or like one of those like margin call bullpens, you know, where everybody's sitting around in a cube working and there's a security guy behind them going, you know, you shouldn't

do that oh you know what that's actually a incorrect algorithm you're doing there oh you can be susceptible you might do a sequel injection you know nobody doing that over your shoulder so being able to have devsec ops as part of it security integrated in at every phase here's the thing people you know it's not a point solution security should never be a point solution

It should be an iterative solution that goes through the entire SDLC from the developer level. Even we have here, we have a product called curation, right, which is like a firewall for your organization to intercept requests. And if those requests come in, these are rules and definitions by companies that say this is a compliance issue. This is a longevity issue. Like, is it operationally risk averse? Are there malicious packages? But it should be all the way down to runtime.

And the thing is that every segment along the way is susceptible in one form or another to some sort of security attack. So when people say, is security separate from DevOps? I say no, because if it is a single point of failure, then you're failing.

Because the thing is, is that there's a lot more space in that spectrum of what SDLC, software development lifecycle, and that should be attributed to every segment of the SDLC from shift left as part of the developer experience down to even code, right? Being able to actually go through and say in Git, we had a big announcement with GitHub this year around things like autofix, like how do you autofix potential security threats and issue? We worked with them on that. Co-pilot integration, how do I...

Query and find out before I even do my job if something I want to use to build my software, one of these libraries that's essential. You know, 85% of what I do as a developer is someone I don't know, right? These third-party transit dependencies. Is that going to affect me? Part of the, you know, CI process. It should be part of that too, right? Constantly iterating through. And as you build, it should be doing security checks.

looking for things like secret detection, are my applications configured correctly? Did I forget to enable TLS on a connection that I'm doing to the service? It should be part of the QA process even. Even as QA is QAing it, they should also be successfully testing it too. Even when it gets to production, you should give one little chef's kiss of security before you deploy it, and then once it's actually been deployed into say something like a Kubernetes set or node, you should be constantly monitoring that too.

The thing is, we're a CNA. So at JFrog, we're a CNA, which I adore. I think it's one of the coolest things we do. And we have this massive research team. I love going through – it's such a nerd thing. I love going through their research site and seeing what they've discovered. And you'd be shockingly surprised how much –

security has now become a normalcy, which it should always been there. It should never have been those guys in the black glasses with the black hats and the shirts that hang out in a room and come out of nowhere and say, "You need to fix that because your code is bad." The thing is that there's not enough humans. Usually it's like 100 developers to one security person. It's crazy ratio.

So the thing is security should be part of it. It should be automated. It should be behind the scenes. Plus it allows the security people to focus on what's important as opposed to a bunch of just minutia that's out there. Like they get inundated by this stuff. Like they're flooded by CVEs constantly. How do you make sure that they're focusing on the right ones? How do you focus on if something is contextually relevant to the issue that's actually being brought up by, say, a vulnerability? How do you know if you're being affected by it?

Yeah, I think one of the other challenges as well, when you have security, I think this goes for privacy as well as these like, sort of separate teams that are like satellited in at the end of like, you know, a launch to make sure that they, you know, bless the thing, it becomes somewhat of a antagonistic relationship as well, where they can be seen as sort of the office. No, because they're coming in at the very last

point after people have invested all this time to say like, no, you need to go back and redo these types of things. But just to play sort of devil's advocate with this, does moving releases to DevOps, to the engineering team, to those that are building, we're also moving some parts of security there. Are we overloading essentially the engineering teams with too much responsibility by also giving them sort of security to think about?

You know what? I'm glad you brought that up, right? That's actually, you know, that's the thing that we, most organizations, most people I've talked with over time in this industry that are friends of mine and other colleagues. And, you know, I've been to so many conferences and we've had this debate, you know, how much is too much? And my opinion, and this is very controversial, but my opinion is it's never too much.

But it needs to be done intelligently. Okay. So it's not just, is there an issue? This is the nuance. Because the thing is, say I'm a developer. Now, security, you're telling me now I'm a security engineer besides being a software engineer. So not only do I have to sit there and come up with creative ideas and aspects behind the features and things I'm trying to create, but now you want me to be a security guy? Well, how do you do it in a way that's not intrusive? How do you do it in a way that...

it feels almost gamified in a way, but also too as a way to help the coder actually become a better coder. And that's the thing, it's a fine nuanced line you have to ride. And I believe it's every developer's responsibility to do this. I mean,

Even in the days we didn't have things like SAST, right? That lets me know that you're a bad coder. You did something terrible, right? You shouldn't have done it that way. You know, it would take usually some sort of pen testing. It would take some sort of automated testing maybe down the road or maybe a physical test. I mean, I remember the days when we didn't have the automation. You know, we literally had to go through and record the clicks, right, on the website you were doing. And then it would do the injections and testing like that was a very manual process.

By giving the developer the ability to look at their IDE, with the world that they live in, give them all the potential tools that they need to do their job in the world, their house, right? I mean, your IDE is your house. You're in your house. You have different rooms in your house. You have your source code. You have the code you're working on. You have the dependency libraries that you're monitoring. You have all these things. But what if you were able to add that layer of protection to that, to highlight for you, take the ambiguity out of it,

And that's the beauty of being able to use tools like Copilot, right? I have a, you know, with Copilot, the stuff we're doing, we did with GitHub. The best part about this is, is as a developer, I can go in and say, show me a better way or maybe bring up and show me some of the code. And then what they came to us for at JFrog was, well, what kind of libraries can I use? What are the safe and compliant libraries?

Why don't I query in advance? Why don't I go find these things in advance so that will reduce that actual cognitive load I need to do in the security realm? And that's the thing is, so when it comes to it,

I think that there's never enough, but it has to be done in a way that it's not like suddenly, like I love the fact, what did you call it? The Ministry of No? Or the Department of No? Yeah, Department of No. I'm going to go with the Ministry of No, though. I like that because that sounds like a very, it sounds like a very, but the idea is very, like I said, you know, when we get back to the idea of like, you can't have a security person hovering over your shoulder, whispering in your ear, like creepily saying, you shouldn't do that.

You know, like that could be bad, right? This is one of those ones that, you know, I've shown developers, like I said, that's my background. You know, I started off as a coder and, you know, done everything. And like I said, I wish I had half the tools.

I always go to my IDE. I bring a VS Code, which has always been my one of choice, you know, after the acquisition, you know, because it was before. Anyway, but I go in and I say, look, I can still do my job. But at the same time, I now have a bunch of rich information that allows me to do it, number one, better, right? Takes a lot of the time that would take me to write certain algorithmic functions and stuff.

And do it for me so I'm not just, you know, missing a parenthesis or, you know, whatever. And then on the other side of this, making sure that the things I need to do my job properly, the 85 to 90% of the things I consume are safe, secure, and compliant. And it puts the onus on me, you know, it's 100 times more expensive to find a security issue or a security object or some sort of potential threat, nefarious component, whatever, in production than it is at the developer.

So, given that we have the growth of an interest in DevSecOps and also no shortage of security tools in market and more secure ways of developing software.

The problems or challenges that companies face with security incidents, data breaches are not only not going away, they're actually getting worse. So why do you think that there's still this problem despite better tools and practices in place? So it's interesting because one of the things that we addressed here, and this is actually when we were working on it a couple of years ago, my eyes opened.

I blazed when I heard what we were going to do and I looked at it because one of the problems is, is the deluge, right? Yes. This is a systemic problem. It's been a problem for decades, by the way, right? This is not new. It's just become more prevalent. And I think the thing is, is that I don't think we had the tools to,

controls to see at the time the level of the potential exposure that companies were having. And I think that's the reason why the numbers are so high in a lot of cases. I just think we're more aware, right? We're now more aware. We know that the problems have happened. The knee-jerk reactions are out of the way now. We're now on standards.

right you know when you look at things like i said like solar winds or log for j right log for j i mean let's go back and look at that for a minute right i mean that was the you know it's the darling of that library you know everybody knows log for j and then suddenly it became the most toxic mess on the planet you know and the thing is the software building materials standard came from that and solar winds right that was like you know the government's reaction was like you know we got to do something about this i don't have to say something i'm not

I curse like a trucker sometimes. But this is the thing that we need to react around it. And when we look at this, it's not that it's increased suddenly. I just think we're more aware of it. We have data to back that. And the thing is, is that for us, what made me excited was, is that this constant delusion that developers are like, you know, we started talking to customers and we started looking at reports that were like,

CVEs aren't that too important in a way because there's so many. It wasn't that they're not important. It's just that how do you constantly – you have an avalanche coming at you of CVEs. How do you pick the ones that you want to address? And what we did is we created this thing called contextual analysis.

And with contextual analysis, we actually look at the actual threat parameters of, say, an issue. And we say, are those parameters, are those conditions being met that would cause the potential nefarious exploit? And this cuts down the amount of actual CVEs and the amount of potential threats and security issues that our companies we work with have to go after. I showed one customer. They gave us a doc remit.

And I ran through our thing and I came back with like 458 CVEs, like something ridiculous. And we're like, how long is it going to take you to go through that? And just to find out if you're being affected, not even what level you're being affected, but are you being affected? And they go, probably not. We'll probably go for the critical ones and the CVS score 9.7s and above. And I said, what if I could tell you that 92% of these are not affecting you?

with certainty. Those conditions for that exploit are not being met. Now, you have a small segment of security threats that you can address that's manageable and effective.

Now, like I said, that's the reason why I said, even if there's an increase in security vulnerability attacks, doesn't mean you're being affected by it because in most cases, it's a singular condition, right? It's a condition that gets met with a parameter, a variable, something that's passed into it, whatever, you know, that causes the exploit or is a chain to exploit, it doesn't matter. But the thing is, is that you

Yeah, there's a sheer scary number out there. That's terrifying. 240 something percent increase like year over year. It's something stupid. Like you go, that's terrible. But you're like, well, you know what, though? Just because there's an exploit doesn't mean you're being affected by it. It's not going to affect you directly. That's the thing is like when I look at this, I said, yeah, you know what? Yeah, it's terrifying. Those numbers are terrifying. But here's the thing, too, by the way, just quickly, is it's not only are you being affected by it, but if you are being affected by it, how long have you been affected by?

Also, too, what other products in your product line might be affected by this? How do you have an audit trail? How do you have accountability behind those components and pieces you use to effectively protect your company or ensure that you have been protecting or to mitigate a disaster that might be PR coming out when somebody finds out that you're being affected by this and X number of millions of user data has just been exposed?

These are terrifying threats that companies think about all the time. And like I said, just because there's a sheer number of increased checks in the supply chain doesn't mean it's affecting you, but you should be prepared for it. Yeah. So I know that JFrog works with some of the largest banks in the US. When we think about DevSecOps and regulated environments,

I guess, like, what tooling does JFrog provide those companies that help solve some of these problems that we're talking about? Oh, absolutely. And that's actually some of the things we were just talking about, right? We actually provide various different levels. And our banks that utilize our platform use large swaths of what we do. You know, our security product called JFrog Security is actually segmented into four different products.

We have our standard JFrog X-ray, which is different, by the way, compared to any other security tool. Most security tools are action-based. You need to do something to get some sort of results back to enact some sort of action plan to fix that.

With our X-ray product, because of the way Artifactory is symbiotically linked to Artifactory and X-ray, it's constantly scanning. It's constantly evaluating. And it's also, by the way, remember, it's not just one security facet that it should be affected by, right? It's not just CVEs or CVSS, right? There's malicious packaging. On top of that, there's licensing and compliance. You know, read those licenses, right?

You'd be surprised at how many crappy licenses there are out there, right? That are just made up licenses that say you use us and all your code belongs to us. Small print, right? And then on top of that is also two things like operational risk. How old or outdated or abandoned are the libraries you're using? Over 80% of the libraries that are out there in library world, depending on like the Python, NPM, whatever, are old, abandoned, or outdated.

I mean, that affects you too. If you have an issue with a library that you're utilizing that hasn't been updated since 2011, good possibility that you're going to have to go hire a company that will rebuild it for you with the fix in place. There's a couple of companies out there that do that. Or you're just going to have to deal with it because it's never going to get fixed. So you can write rules on that too.

Now, we also have advanced security, and advanced security allows us to go through and filter through the actual CVEs that are affecting companies, right? Are you being affected by this potentially nefarious component? Also, too, you know, are you doing anything in terms of secrets? Are you exposing? I'll give you an idea. I downloaded a Docker image. I ran it from Docker Hub, pulled it down. I ran it through our security product and found out I had access to this developer's

AWS keys, his GitHub, everything. I had access. I contacted the guy. And he's like, that's impossible. It's clean. I go, did you check your root history? And you go into the image and you type in history and it had everything. Everything. And he's like, oh, my God. On top of that, we also tell you, are there services or applications that can be potentially threatening in, say, an image that you're doing?

We also do infrastructure as code analysis. I messed up once. I was learning Terraform. I did all this stuff. I made my first. I was patting myself on the back. And none of the ACLs, none of the security actions I put in were there. And when I ran it through, I looked ahead. Authorization equals none. Totally rookie mistake, but our product found that and let me know. I was like, oh, I'm not crazy. But.

But we also go to the frontline defense on this for them, right? We have curation, which is like a firewall because x-ray and our advanced security is once those binaries, once those libraries, once those nefarious components are inside your environment already, potentially.

Zero days, right? Our curation product acts as a firewall and intercepts the requests for that. It says, I want to bring this library in, but what I'm going to do is I'm going to do a dry run to see, you know, what things are coming in and stop it before it begins. And

And then we have companies now that are using, we released this year, runtime. So in other words, now we can trace that security issue from before it comes in, while it's in, giving the developer everything they need in the IDE with SAST and all that, all the way down to is this thing in my runtime environment right now? And how much of a potential threat is it?

Right. This is the stuff that we're bringing to banks. And the thing is, is that by having a single holistic security solution allows it so that you're not trying to correlate data after the fact. Right. This is correlated data in the same platform, allowing you to have that consistency. Like people ask me if I were to delineate everything we do in JFrog down to one word, I would always say consistency.

We provide a level of consistency to organizations so they can reduce the tooling, which means that they don't have tool sprawl, which also means that there's security gaps and holes. The more tools you have, to be honest, the more holes you have because those are interoperable, right? They don't work together. They're non-interoperable, actually, with each other. So you have to do manual correlation. And anytime, I'm sorry to say this, but we get humans involved, unless it's an innovative kind of thing, we're actually one of the linchpins in something falling apart.

Yeah. So you mentioned the guy who accidentally had a bunch of credentials, you know, in image history. And I think that's like one of those things where you read about like some data breach that was due to weak credentials or something like that. And you're like, oh my God, like how could these people be so like dumb? But like, it's such an easy mistake to make as people. And, you know, like you said, like, you know, a lot of times people are the linchpin in this. Like,

I guess like on the, in terms of security issues that relate to like social engineering or a person being the falter point, like are there things that JFrog or a DevSecOps like practices help prevent things like that?

Well, it's funny as you say that, right? You know, I had a chance to meet like Kevin Mitnick when he was alive, right? I actually, I have one of his, I have one of my prized possessions in my home on my shelf is I have one, I even have it on a little stand. I have one of his lockpick business cards. I don't know if you ever saw those because Kevin Mitnick was the quintessential social engineering hacker, right? I mean, that's how, I mean, he started with the bus system of LA when he was what, 11, you know, and then worked his way up. And, you know, the whole idea is most of his hacks were done over the phone, right?

That's a harder thing. We don't prevent that. Right. I mean, let's be honest with the code side where the where the ones and zeros, the binary side organizations, when it comes to the social engineering aspect, you know, that comes down to, you know, we've all taken those courses. You know, are you compliant? You know, Sally sent you an email and sent you a request to say, hey, I need your social security number, your mom's maiden name and your first dog. Right. You know, it's like, oh, I'll answer that.

You know, those kinds of things are, you know, somebody calls and like, all right, I was watching, I decided for some reason to start watching hackers when I was walking, was working out this morning, you know? And, you know, he's like, oh yeah. He's like, he's like, look at the little box next to the computer. He's like, and my BLT drive crashed and,

I'm AWOL on this and I'm laughing so hard watching just 1995. But the thing is when it comes to social engineering, that's more of education. That's more of concise guidelines, putting in processes and things like that. We do it from the automated side. We're the robotic side of the house. We're the automated thing. We're the people who give you the notifications that you're doing something terrible and you should fix it.

Hackers and Kevin Mitnick. We're going back in time. You're really taking me back to my early days on the internet.

By the way, I still have a free Kevin sticker in my office at home. Awesome. Well, relearning, you were talking about how one of the things that separates or differentiates JFrog is sort of this more like proactive approach versus reactive. And now you're making, you know, investments into ML as many, many companies are like, how does that potentially help with proactive like

We're entering this era of agentic AI and agentic systems. Suddenly, could we actually have some of these security tools that are actually not only monitoring and alerting, but going and fixing the problems?

I'm glad you said that because actually this is one of the things that we're doing with the, you know, we just introduced, you know, it was actually an acquisition like we talked about before. It was a company called Quack and now it's our JFrogML product. We also just released our FrogML, which is an SDK to help around this. But the thing is, is that

What we've done is we've taken our knowledge of, say, DevSecOps, our ability to do this, MLOps. We're like, no, this is just an extension of DevSecOps. It really is. When you draw the cross parallels between standard software development engineering and ML engineering, there are, yes, they're different, right? But there's still similarities, right? You're still using Python, right?

or are, right? To do, you know, your data analysis, your EDA and all that kind of stuff. You're pulling in data sources that are out there, right? Hugging face, look at hugging face, millions of repositories, you know, a repository of millions of LLMs and things like that. We proxy those in our company already. We did this before even the acquisition. And we use our security product to scan for things like malicious packaging and licensing and

Things like that, making sure that you're not bringing in data sets that you're learning to train your potential models with something nefarious, something malicious, something that could cause a skew, a bias, a hallucination in the data. But also, too, remember, things are built on Docker. They're actually deployed. I mean, whether you're working with a GPU or you're working with a standard CPU, you

And then being able to go down and have some sort of level of accountability. I mean, we talked about SBOM a little bit over there before, right? You know, the reaction was software building materials. Now there's MLBOM, right? The idea of like, what did I use to train this model, right? There's going to be accountability behind that too. And now I think the industry is now taking it more seriously. And so for us, you know, we're providing tool sets that are proactive. You know, we're building a proactive modeling security in this case.

We just released our machine learning repository so that people can customize, build, tune, and have a place where they can version models as opposed to shoving three things into an S3 bucket, which makes no sense. We're going back to the days of where we stored third-party transitive dependencies before products like Artifactory came out, right? And the idea is simple.

Now, we need to take those approaches. You know, ML is in a Wild West phase like DevOps was in the early days. Look at the number of tool sets. You know, you talked about that.

Just Google. Just be like ML tool sets or ML things. And you get those maps like we used to have. Remember the maps you look at DevOps and it's like here's all the quadrants of crazy crap and there's logos you can't read because they're like the size of a dot, right? These kind of things. And then on top of that, they have the size of a dot. And now we have the same thing with ML.

Yeah, absolutely. And also, like, you know, I think as much as some of these technologies can bring to automating and help us even, you know, automatically fix some security issues, we're also introducing potentially a lot of, you know, security vulnerabilities, especially ads like, like it is the Wild West. We have so many, you know, new technologies.

Tool stacks, they're going to all have these dependencies, could open up a lot of potential supply chain attack scenarios, as well as just like tools that are going to the market. And security is not job one that they're necessarily thinking about. They're just like, hey, I need to get something out there in order to meet demand and stay on top of sort of the hype cycle. Yeah.

Absolutely. And the thing is, is now we're going to go into the next phase of this, right, which is accountability. And now with the accountability side, people are going to start demanding more. And I'm going to say when I say more, I'm going to say more information on what comprises the things that we utilize and interact with.

You know, look at some of the things that are going on, even like at OpenAI, right? With like, you know, ChatGPG4 and now 5, right? And like the iteration of the models. I know that we've already gone past, you know, I don't think they have told us everything on sentience yet. But, you know, the thing is, is that,

Yeah. Not only is it preemptive and not only is it iterative through the process, but I think we're heading into the accountability phase, which is we want to know how the soup's made. People are going to want to know from a legal perspective what kind of challenging data has been attributed to the marvel in which I'm interacting with. It's like there's still a lot of things that I

I think is shaking out. It did the same thing with DevOps, right? And then DevSecOps. And then we started getting into ML and MLOps. And, you know, what's the next phase? I'm excited. I don't know. I've got some ideas, but I like the idea that I don't know. And I hate to say it, though. Something is going to have to happen to catalyst that next step. It always does. Yeah.

Well, Bill, I want to thank you so much for being here. This was really fascinating. I love your passion. It's really energizing. Thank you for having me. I really appreciate it. You know, like I said, we've all been through a lot in our careers, right? And the thing is, is that I love the ambiguity. I love the future. I love the uncertainty. I love a little bit of that chaos engineering that, you know, that our life is. So yes, I'm glad to be here and I appreciate it. And this has been a really fun talk. You have some really good questioning and I appreciate it.

All right. Well, thank you so much. Cheers. Cheers. Have a great day.