#268 Scaling AI in the Enterprise with Abhas Ricky, Chief Strategy Officer at Cloudera
43:23 Share

The data is becoming much more powerful because now you can use that as enterprise context and apply large language models or any of the new AI frameworks that might be on top of that. So I do think that there is an increased focus on the quality of the data, as I said. Welcome to Data Framed. This is Richie.

Data and AI are brilliant, so naturally you want to use more of them at work. The problem is that at some point either your technology or your processes will fall over and things will stop working. So today we're going to look at scaling your operations. I want to know how to manage costs as you grow, which areas to focus on to have an impact, and how to deal with the security challenges of data sprawl.

I also want to know how useful some of the big trends in data management are, like hybrid computing and treating data as an asset. Joining us today is Abbas Rikki, the Chief Strategy Officer at Cloudera. Abbas is responsible for creating the company vision, building the business and customer target operating model, and communicating that with key stakeholders.

Previously, he served as Chief of Staff and VP for Business Transformation at the same company. And prior to the Cloud Era Hortonworks merger, he helped scale Hortonworks' go-to-market efforts as Global Head of Customer Innovation and Value Management. He was also named a Global Shaper by the World Economic Forum and was listed as a Founder of the Future Under 35 by Founders Forum.

On top of his day job, Abbas was founder and CEO of the non-profit AMRI Foundation, helping underprivileged children integrate into mainstream education. So, let's hear what he has to say about scaling data and AI capabilities. Hi Abbas, welcome to the show. Morning, great to have me. So, to begin with, what are the most important trends you're seeing in the world of data?

Yeah, so there are a couple of things that we're starting to see. So as you know, for the last 10 odd years, the majority of the focus has been about cloud applications. There wasn't technology in the last 40 years where you have three very large providers, each of them with $100 billion of compute or more, growing at 30%, 40% KBAR.

And that fueled the data application business to next levels. But that also brought in new challenges around data security, data governance, data lineage, and all of those capabilities. So over the last five years or so, when realistically the Transformer library paper came out from OpenAI, there has been an increased focus on data as an asset.

And therefore, majority of the large enterprises we speak to, or I speak to on a weekly basis, they all want to say, hey, I want to be able to structure the data that I'm going to use to train my large language models on.

And whether it's agency applications, whether it's RAG applications, whether it's fine tuning, the core premise is you want to be able to trust the data that you're going to use for any of your AI applications. And I do think that the trust in the data is a big, big subject that's coming through. The second thing we're starting to get through is the fidelity of the outputs that we're starting to talk about. Oftentimes that depends upon the quality of the data sets that you have.

And therefore, in AI parlance, we call that as enterprise context. There's an increased amount of focus on getting the high-value enterprise context, whether that's for AI applications, whether that's for agency applications or there are. And I believe the third one really is to the original point that I said, which is there's a lot of data sprawl that is happening. Because remember, architecturally, we went from data lakes to lake houses, right?

to now a slightly different architecture for AI technologies. But along the way, there have been application developers, there have been AI practitioners, there have been SQL developers, front end and back end, and they've all created their own systems and tools. And there's been a lot of data sprawl that has happened. And that creates issues around data security, that creates issues around cost. Total cost of ownership is a big element.

So that is something that is coming to the fore more and more. And large enterprises are trying to tackle that with the best tools and services they have at their disposal. Those all seem like, well, very important things. So yeah, you want to be able to trust that your data is going to give you the right answers and you need to be able to find it rather than just working out which application it's sitting in or which database it's sitting in. So you mentioned the idea of data as an asset there. Can you just tell me a bit more about what that means?

Yeah, so technologically, a lot of the people have said they wanted to get to data mesh architectures. And that's

a source of capabilities, right? So that includes query federation, for example. That includes adding attributes on the data itself so that you can treat data as a product. Because at the end of the day, that's where majority of the application developers want to get to so that their job becomes easier. But on top of that,

I'll give you a real example. There's a very large family I work with. They have close to one x-bytes of data in the management. And that's a very large number, structured, unstructured images, all kinds of data.

But for two, three years, that was all storage. And you could pull some of it out for analytics. You could pull some of it out for your next best action or your sentiment analysis or whichever the use case might be. And then you worried about making sure you were compliant with the policies. If you were regulated in this case, they were. But also elements in Europe, as you know, GDPR in California, CCPA came through the floor.

So a lot of the value was around making sure you can use the data in parts to get to a specific outcome. But with the onset of AI, what has happened is the data has been completely the primary focus because of the democratization that is taking place. So for example, if you were a SQL application developer and I was a data scientist, two years, three years back, you would have to come to me every time you wanted to change a parameter on a model.

Now you don't need to do that because you can take the enterprise context, aka data asset, and apply it to the large language models without even having to come to me as the AI practitioner or the data scientist. So you don't need to evolve or upskill necessarily, and you can get different outcomes for different use cases with the same dataset.

So the data is becoming much more powerful because now you can use that as enterprise context and apply it to language models or any of the new AI frameworks that might be on top of that. So I do think that there is an increased focus on the quality of the data, as I said, but also making it available, getting it to the hands of practitioners is equally important. And data as a product definitely helps you get there.

Absolutely. So I like the idea that, well, you know, just storing your data somewhere, keeping track of it, that's fine, but it's not necessarily going to get to adding value to your business. You need to make sure that you've got it in a place where you can use it easily and then that way you can take advantage of it. And you mentioned generative AI. Obviously, it's working its way into almost everything these days.

But I get the sense that there are some organizations, you know, they've sort of got things together. They're quite advanced. Other organizations are still just getting started. Do you have a general sense of where enterprises are at typically in their AI journey? So I'll respond to the answer with,

Two facets. So first one is depending upon the journey that the customer has been on or lives and space they've been on. So let's talk about leaders. So leaders are organizations who've had very large or relatively large engineering teams in-house, skill sets. And they've been able to solve the data engineering problem rather succinctly, but also within the realms of cost, governance, operations, etc.,

So those people are primed to take advantage of the generative AI application. So for example, there's a very large bank that we work with and they started off with a low-hanging fruit. They build a text summarization application, they build co-pilots or something that is like a chat Q&A for the service operations. So they spent the first year doing that largely with the systems.

But this year, they're actually getting to scaling, but also they're going to agenting applications. So I say that move from assistants to agents. Now, an agent is largely a capability that allows you to do function calling. Back in the day, some people used to say API. But an autonomous agent actually gets from

not just data to insights, but also to action. Because it takes actions on behalf of the human loop that would have been enforced by analyzing a set of parameters, understanding the intent behind what the action should have been, and being able to do that. So in this case, this bank, this last one here,

They've productionized various use cases such as loan underwriting or such as tax invoice reconciliation through all agenting applications they might have.

But also, even the LLM calls example that I said, 70% or more of their LLM calls are now being done through agents. So they spend this year productionizing agents, but also incorporating generative AI as part of the normal processes, which I think is a super important part for scaling generative AI. It can't be this standalone piece in the corner as an innovation arm. It has to be embedded as part of the normal R&D cycles. So that's for the leaders.

Now coming to the laggards, and I don't want to have a negative sentiment with laggards. I just mean to say people couldn't get started early enough. I do think they're still at a slightly earlier stage and they're still starting to get to a point where by playing with the model of the choice, whichever flavor they like, proprietary or open source, they're still getting to build applications with whichever AI frameworks there is.

and they're still starting to get to their first AI application. But those organizations, enterprises, or customers of ours

We oftentimes say that you have to make sure that your architecture is built for scale in the future, but also you're getting to a point whereby you have the core data platform. You have the core ability to serve up the data before you can do any of the model hosting or fine tuning or RAG applications or so on. So I think a lot of those customers are at that point in time, and that's what we're seeing them to get towards.

Okay, so there's really quite a wide range. And of course, yeah, it's definitely okay to be just getting started now. Everyone's a bit like a beginner at everything to begin with. So you mentioned the idea that you need to sort out your data before you really get into the AI stuff. So tell me what needs to happen there. Like what do you need to do to your data infrastructure before you can be successful with AI? Yeah, so I think the AI lifecycle is...

a very good representation of multiple steps that you would have had done when you were building a cloud application as well. But it's slightly different in that you need to have solved a data engineering problem before, because you still need to ingest the data. You still need to do data prep. You still need to do data writing. You still need to do data engineering, and then serve it up to whichever platform you're using to be able to apply

any applications, any model, any frameworks, any agents that might be on top of that. So what I meant by that is

As part of your AI application lifecycle, you have to make sure that the data ingest prep, running, et cetera, has been done and you're ready to serve it up to an environment or a platform whereby you can then start to apply AI or machine learning application building capabilities on top of that. That is slightly different from when we were doing lifecycle analytics because when we were just doing lifecycle analytics,

The whole part of the lake house was that you had the data ingestion prepping, but you were also finding applications on top of that. And that is what was giving you outcome. And therefore, I use the word when you go from data to insights. In this case, you're going from data to insights to value to action. And that last mile is equally important. And it's super important for us to be able to get through that.

I really love the idea that there's a step-by-step framework there. So you start off with improving your data, then you try and get some insights out of that, and eventually you want to move towards getting some kind of action from that data. And then maybe you're sort of adding on the AI and you've got agents doing that all automatically. So it sounds like there are going to be different challenges at each step of the way. So I guess to begin with, what are the big challenges with preparing your data, particularly in the enterprise? Absolutely. And I think...

The big context is whether you're doing lifecycle analytics or whether you're doing AI, it's a team sport. I've always said that AI is a team sport. So for example, the example that the use case that I just talked about, which is AI lifecycle, you have to get to a point whereby your data management platform serves the data at the highest levels of fidelity, but with the security modules, with the metadata that you require to be able to go and apply forward.

you need to have models that you can apply to the data that you have that gives you the highest fidelity output, but also within the cost framework. So for example, you don't oftentimes need large language models. You can potentially deal with small language models or even large action models as well.

And then when you're starting to get to do vector embeddings for semantic search querying, or you're trying to do inferencing of certain capabilities, you also have to have a governance layer around that. So not only do you need finding tools, but you also need monitoring capabilities, observability capabilities. You need to have MLOps capabilities around that.

And then finally, when you start to deploy and build agency applications, you need to have agents, but also the orchestration around them, because there are a bunch of them depending on which tool you use. And then you need to have an application hosting UI capability that you can deploy all of that. But across all of these things, you need to be able to get to a point whereby you have

an infrastructure layer so that you can run scaled-air workloads on GPUs on an environment of your choice, private cloud, public cloud, your desktop, your mobile, at a price point of your choice. Why? Because compute is so prohibitive in today's world. And by the way, last I checked, electricity isn't becoming cheaper. So it's going to be there that way for a period of time. So that's the AI module. Now to the second part of your question in terms of the specific challenges around data to get that data prepped.

Well, there's a few. Number one is data scroll. That's happening, as I gave you the example, for application developers. We need to make sure that we are able to contest that but also have a process

and a set of processes within the systems, within the organizations, within LOB IT, lines of business IT, so that there is no concept of shadow IT driving that through. The second one is getting to a point whereby you can access the right data sets. So I talked about agents. One of the customers told me about the hardest thing to do is to be able to expose the data

to those agents because that is the core fundamental for us getting to our higher fidelity output on the other side. So getting that and making that available to whichever applications you use it in the format that you want, in a platform where you want, is not an easy thing. And obviously lastly is

Everyone wants to be able to know who touched the data, who changed it, when did you change it? What did it look like before? What does it look like now? That's all things around governance and operations. So data lineage, metadata governance, technical metadata, business metadata, having a capability of understanding

the flow across systems when you move the data. So those are the three broad areas in which I'd say data operations as a concept is becoming more and more popular. And that's where that's feeding the first part of the question you'd ask, which is how does data operations make it inherently useful or increase the outputs for the AI lifecycle that we just talked about? Oh, wow. That sounds...

an awful lot more complicated than I was expecting. There's like so many different layers involved there. So you said you got the infrastructure stuff, like data lineage, ML ops, you've got monitoring, you've got all sorts of stuff. And that's before you even get into like, are we actually solving a business problem as well? So maybe before we get into like the problems later down the sort of life cycle, perhaps it's worth talking about who needs to be involved. I guess it sounds like there's gonna be so many different teams involved. Who needs to be in charge of all of this?

I don't think there is one person who's in charge of the whole thing, like we were doing in the whole cloud application era. Naturally, we'll have the data stewards who will make sure everything around the data operations is running smoothly. But obviously, all of this starts with the use case. All of this starts with the business use case.

It's very rarely, and actually it's a wrong thing if you start to build an application and then force it onto the lines of businesses. So it should start with the line of business owner. So for example, if you're a bank or a telco, and if you want to run a sharing model because you want to get better insights in your most loyal customer, that's where it starts to. That person will own the business definition and the conversion. Somebody on the team will own the conversion of business definition.

requirements into technical capabilities that the IT teams will have to do. Depending upon the org structure they might have, oftentimes you will have AI practitioners sitting within the lines of business as well. And then obviously they will come in

and they will want to try and build an application to test it out. And sometimes if you need to scale it, they'll lop it across the wall and say, "Hey, IT, can you actually get into enterprise-grade security, enterprise-grade governance, and make it an enterprise-wide application?" So in that case, obviously, the AI practitioners, but also the SQL application developers sitting within line of businesses, which you and I call it ILOBIT, will need to be involved to get that test through.

But when it gets to enterprise-wide usage, it's not just the IT teams who are making sure that that's getting through. You have the security teams involved. Because in a large bank, in a large token, a large insurer, etc., there's elements around data security, which is super important, specifically in the world of generative AI. And we've all seen the multiple lawsuits that have been filed against OpenAI, for example, in the New York Times was one of the conversations.

But do we have policies whereby which kind of data sources can we use within a model? Because that is something that needs to be driven by a compliance officer. Do we have policies around what can we actually put on the internet or can we use somewhere else? So there are policy requirements whereby security teams and compliance officers will have a role to play as well. And then lastly,

it's the core engineering capability that you have in-house. There are the people who are building the picks and shovels and doing the plumbing to make sure that your platform looks cool, and not to mention the practitioners who are actually using them. So there's a, as I said, AI is a team sport, data is a team sport, and you have to be able to make sure you have the right organizational structures in place to be able to drive the most efficient processes forward.

Okay, yeah, I absolutely agree that it's a team sport. And it's interesting that you should always start with the business teams and like, what's the business problem? Because otherwise, I guess you're not going to get any value from it at the end. It's going to be a mismatch. And then move it on to the data and AI people. And then it's only when you start scaling that the IT people and engineering teams tend to get involved. So you also mentioned the legal challenges and making sure that things are secure.

When do you need to start worrying about that? Should that be like at the start? Should it be after you've come up with a business case? Yeah, when do you need to worry about the legal and security challenges? I'd say worry is probably not the right word. I think when can you start to plant in corporate elements around the legal ramifications there might be? And I think right at the start when you're building a plan, and the plan is not just a business plan, but also a technical plan or a solutions plan.

And the reason I say that is because, so right now you have Dover available in Europe, right? And you have the White House Precision Initiative from the guidelines around AI came through. More and more countries and more and more organizations will be subject to different levels of regulatory requirements and pressures that will come through. And that is something that a large part of practitioners

practitioners will need to deal with. So I do think that you have to be aware of what the ramifications are for specific policy directives, whether it's governmental or intergovernmental. I do think you have to incorporate the potential legal risk and exposure that you might get into if not applied through that right at the start and also throughout that.

But the big thing, which is, this is an ever-changing paradigm. There's a model every Sunday, there's a framework every Monday, there's an agency, company, and an application that's every Wednesday. So naturally, people who are on the policy side are also evolving. They'll be coming up with newer guidelines

over a period of time. So not only do you have to do it right at the start, but also you have to make sure you're planning that through along the way. And the one thing that I will say, which is not long ago, we had multiple multi-trillion dollar companies out there who have a lot of buying power, but also the ability to dictate how the practitioners engage with them.

So in cases of partnerships for smaller organizations, for example, if you want to be a strategic partner of choice with somebody, you have to make sure that you're taking care around elements such as indemnifications for libraries. And those are some of the core pieces that will come through to the forum more and more as people move from building RAG applications to financing. And I think most people are starting to do that in six months if they haven't already.

So that is one of the core things that will evolve just like everything else. Okay, yeah, that certainly makes sense, particularly if you want to partner with larger organizations, then you're going to need to make sure that everything is working correctly from a legalist perspective. And there's only so many companies who have chip providers. There are only so many companies who are providing with the compute capabilities. And at the end of the day, what's the monetization advantage

unit economics lever. It's compute. That's what everyone's after. Whether it's the large LLM providers, whether it's the hyperscalers, we're providing cloud applications and now we're also getting into some of them getting into ship manufacturing space, or whether it's the very large GPU providers who have just started

a software business, they all have to compute. So I do think that, as I said, very large organizations have extremely outsized buying power. So doing generative AI at scale is a capital-intensive game. And I've said this before, you have to take a leap of faith. You can't say that, oh, I'll get somebody to do this, and then there will be an ROI after a year and a half, and they will bring in the consultants, and we'll have the plan.

I think the train would have left the station if that's the case. And therefore, with so much capital and so much investments going into it, there will be a myriad of legal exposures that organizations can be subject to. And we need to make sure that we're aligning to the new paradigm along the process, before the process, and after the process.

This is really interesting that you say compute is the thing that's most important because I think, well, at least the first sort of couple of decades of this century, there was this sort of sense that, well, computing is getting cheaper. And then now, I guess, is Gentrify just this big game changer that says, well, actually, you

applications are expensive to run now, you need as much computers as you can, or you need to get it cost effectively. How has the story changed, do you think? What I meant by saying it's not the computers that are most important, I meant computers are the denominator for which

people are monetizing against. That's the metric that people are using. When you monetize, whether it's a large language model, pick any one, whether it's OpenAI and Threepenny, most probably we're here for a lot of language models. Whether you look at large hyperspheres, whether it's Google, Microsoft, or Amazon, through the services they provide, or whether even NVIDIA that we as CloudAware, we just launched an inferencing service with NVIDIA through NVIDIA NIMS.

So they have their models and they have a microservices package because the models are optimized to the GPU performance that they have the data. But you also have industry-specific APIs. But if I were to take that and give to a customer, the value proposition for them is, A, you can run scaled AI workloads on GPUs in a form factor if you choose the best TCL. In certain cases, you can save millions of dollars a compute per month.

But the second one is you can do what I call as private AI. And then the idea is you can take any of the AI applications that you have and you can do it in public cloud, private cloud or desktop. And you can do that with the model of your choice, open source or closed source.

This is a value proposition for the customer, but the way the vendors are monetizing that is through compute. So that's what I meant. Now, to your question on is compute becoming cheaper? Well, there have been significant advancements in quantum computing. There have been a large set of organizations, including a slew of billionaires, who've been, I would say, sponsoring companies

and alternative sources of energy. I was at the also General AI conference last week in California, and a tier one CEO said that he believes right now

In the next three generations of the models that will come through, it might take up the same amount of electricity as you would take for a small town for a week. So it is pretty compute intensive. So in spite of the advancements of what quantum computing and the normal large companies have done through, it'll be a while before compute becomes available.

less prohibitive is what I would like to say. And therefore, I meant that it's a capital intensive thing. So don't mistake it for the fact that people aren't working on it. They are. Whether it's nuclear fusion energy and Bill Gates talks about it as well or anything else. But I just think that it hasn't become mainstream yet for the average manufacturing company or the average retailer across the world. And therefore, people will have to intend with the services that are being provided at the price point that are being provided today for a foreseeable future.

Okay, yeah, I can certainly see how if you want to vastly improve on this cutting edge foundation models, it is going to get incredibly energy intensive. Is that something that organizations need to plan for now? A big increase in their electricity bill? I would say the first thing organizations need to plan for is which use cases do you actually want?

to build an application for. Because not every application requires another. You can just use a small language model. Not everything requires X billion parameters because you don't need to boil the ocean. So the first thing is identifying which use cases you want to be able to deploy through prioritization metrics, business values and execution. And then after that, once you've done that,

There will be certain cases in which you would want to use large language models. And obviously, you need to make sure you have the data to be able to train them. Because it's only compute intensive, it's running against a wide variety of data sets. So you might not need to. And therefore, I said that there are action models that might be more effective. But in case if you do, obviously, you have to plan for, I'd say, a tad bit increase in your IT budget.

insofar as factoring in the compute cost is concerned. But the bigger thing is, just like cloud applications, when cloud applications became popular, SaaS as a service became popular. Vertical SaaS offerings came to the fore. A similar thing will happen here.

But the winners were the ones who had in-house teams who were developing, playing with the products, fine-tuning them, not as we say fine-tuning here, but in the literal English way, trying to get through with that. So even in today's world, there will be armies of people within your own organizations who would want to get to the latest technology, but will also want to

play with it and therefore you will have to factor in some amount of compute cost, some amount of the GPU capabilities that you want to have internally, even if you have outsourced your preliminary set of use cases to an SI who will build that for you or

a strategic partner of choice for a software vendor. So net-net short answer, yes, but how much it depends upon your use case, the amount of data you use, and where it goes with it.

Okay, that does seem very sensible is sort of using the minimum viable model for any given use case. So earlier on you were saying that there are sort of very simple AI applications like text summarization and it goes through to very complicated things like agents. So is there an equivalent way of deciding well these particular use cases need a simple LLM, these other use cases need something a bit fancier or if you've just got to try and see what performance you get. So

So I wouldn't say that there are easy ones or hard ones. Like if you ask practitioners, all of them are complex ones. But let's say, for example, if you're a bank and if you're a trader and you come in the morning and you read the insights and it takes you two hours, for example, to compile the insights from different reports or different research works. For that one, for example, you can actually build a text summarization tool. You can just have a co-pilot application and all it does is does text summarization in

in two minutes. So your productivity is 99%. You don't necessarily need to your original personal computer in that case, unless you have like volumes of data that you're actually scrolling through the internet, which you won't be in a normal case if you're a specific part of the trading desk, because you have a limited amount of information to try to get through with that.

But on the same front, if, for example, your use case is slightly different, you're doing drug discovery, canonical drug discovery research for the next cancer drug, you want to look at as much canonical and empirical data you might have for the last decade or even more.

And you will have multiple strata of different chemical composition that you are applying through. And in that world, you obviously want an LLM that has the capability to handle the majority of the input requirements that you're feeding in, which a lot of people call as parameters, where you must have a 7 billion, 8 billion, etc.

By doing so, you will get high fidelity output, but obviously it's compute intensive as well. But the use case in that case is life and death. In fact, we have a very large pharmaceutical organization, over $60 billion in revenue, and they're doing a lot of their drug discovery work with us. And they're actually building generative AI applications in production.

And they have been able to move from not just a data platform to knowledge bases so that they can get the insights they need. And they've been running different types of language models on top of us. Now, I agree, whilst it's compute intensive, the outcome is significantly important and it outweighs the cost that you might be going through. Because if you can get ahead of

for a specific drug by X number of years. That's an incredible achievement, not just monetarily, but also for humanity in terms of helping and provide cure. And we have other organizations who are running

All of the immunization testing during the COVID timescales in the years 2020, 2021, 2022. And for them, for example, they'll want to use that as well. So I think it depends on the use case and the outcome. And you have to have, obviously, a cost equation. Everything's a business case. But on top of that, it also is the quality of the data that is available to you and the skill sets that you might have that you can leverage to build applications going forward.

That's really interesting that you've got two very distinct use cases that both make sense. So just summarizing a report or a meeting is like, well, you've just saved someone like 15, 20 minutes, but it's pretty cheap to do. And then on the other hand, you know, you're improving the drug discovery cycle. That's going to make you millions of dollars, but obviously a lot more expensive to do. So it's about weighing up that sort of cost of implementation versus the value you're going to get.

I'm curious as to how you go about scaling things. So you mentioned a lot of companies have tried playing around with prototypes. Some of them have got things in production. What happens when you want to go really big? You found your use case. How do you go about scaling your AI application? There are three pieces that large organizations have to be aware of. Number one, scaling the generative AI infrastructure is an important piece. So it's not...

Just the fact that you will use any application there might be. So I'll give you an example. There's a very large financial services organizations. They have roughly around two dozen generative AI applications in production.

But they're processing 75 million words through their LLMs on a daily basis. They're transcribing 600 hours of calls through the call sensors that they're getting through. And they have like 1,200 direct users and 5,000 daily indirect users for...

the general output you're getting from LLMs. And in that case, you have to make sure that you're scaling the general AI infrastructure as the first piece, because otherwise that is a challenge. Second thing, you have to embed the general AI within the processes I mentioned about. And embedding into the process is not just having a separate team, but also there are different levels of complexity that you need to be able to get to so that you can provide LLMs as a service.

Because you need the system integration with the various tools. You have the applications for different capabilities, but also the serving capabilities. You have a set of orchestration capabilities that you have to focus on. And then there's model management as well. So to be able to get all of that in a manner in this financial services organizations example,

Every 30 seconds, they received the customer feedback across eight different channels. And to be able to feed that back into your social media campaign, to be able to get that through ratings that you might have because people are posting about your CSAT, to be able to respond to the complaints you might have,

that impacts your customer journeys, that impacts your brand perception, that impacts your products and the value. So as you can see, you have to embed that as part of your normal processes. It cannot be a standalone thing. And lastly, I would say you have to get to a point whereby

You are getting to advanced technologies like agent-tick applications. Now, a lot of organizations have used techniques such as reasoning, but also fee-short learning and summarization. And that obviously helps with getting information in real time. But integrating complex reasoning with agents is the next frontier.

Doing it in an autonomous fashion is where the world is headed towards. So I think I'll give you a real example. The same financial services organization that I talked to you about, they have vision language models. What is a vision language model? In plain English, it's a pair of eyes that you can use for doing processing. But they also have agents to streamline complex multi-stage tasks. So the example that I gave you, tax invoice, reconciliation. So they have an agent who does tax

They reach the tax documents. They have an agent that extracts additional sources and applies on top of that to apply the context. They have an agent that does benchmarking against the other relevant tax data. They have an agent that writes a memo. They have an agent that fact checks the memo. They have an agent that formats the memo. So you need a system that orchestrates and regenerates a lot of these agents so that when somebody asks the question, what is the source of this wealth document?

you should be able to get through that, not just through the LLMs that you're using, but also in an autonomous fashion that you can do high volume processing to go through with that. So to summarize those three things, scaling a generative AI infrastructure,

Embedding that in your processes and getting to advanced technologies, aka complex reasoning with agents is the core thing. But here's the key, the benefits is huge. Not only do you get productive improvements like two days to 15 minutes, you get consistency because agents enforce consistent format and quality across all the documents, across all the six use case that I talked about. Validation, agents provide document references in the example that I said.

to make it easy to validate the information. And lastly, and very importantly, you prevent hallucination because the agents can proofread them and you can get any out-of-contact detection and there'll be like a flare for automated hallucination rectification that you can act on.

That's absolutely fascinating that even just something that sounds fairly simple, like writing a document, you've actually got a lot of different agents working together in, I guess, a swarm. Is that the collective noun for agent? Yeah.

So yeah, you've got something writing the memo, something formatting it, something proofreading it, and something doing citations, all that kind of stuff. So it's not just one giant AI, it's lots of smaller AIs working together in tandem. And that's the whole neural network concept, right? Whereby you want to get to a point wherein...

you have autonomous agents taking over a series of human actions. We never document the gazillion steps that we would take to do a simple task.

We just do it because the brain functions and we're wired to do that. But that's what we're training agents to become. So as you can see, when you're applying that to a business use case, there's a series of them. And not to mention, there will be regenerative agents, agents that will be regenerated as well, depending upon what you have before. Therefore, you need a platform to orchestrate that and make sure that you have these very complex reasoning capabilities coming together with a deliberate use case.

But as you can see, I'm super passionate about that. And I do believe that agents will be the next version of applications. And that's where the world is headed to. All right. Wonderful. Okay. So just to wrap up then, do you have a recommendation for what should your first agent be? Again, it might sound cliche, but it depends on what the use case is.

So it depends upon the use case and that's how you should determine that. That's my simple answer. Having said that, there is one topic that we haven't discussed and that is hybrid. I do think whether it's cloud applications or whether it's generative AI applications or whether it's agency applications, you want to be able to do

all of that at the point of residency of the data. And I've often said that, I actually had a LinkedIn post a few courses back where I said, you need to bring the models to the data, not the data to the models.

And the interpretation for that is you need to be able to invest in capabilities that allows you to do hardware acceleration because you want to be able to run scale AI workloads on GPUs, you know, from factory features of the best TCO. The interpretation of that also is you need to be able to run these large language models

in private cloud or public cloud. Because if you're a large bank, if you're a large telco, there are certain use cases that it's never leaving your data center because there's sensitive information, because you need to run 24-7, 365, cybersecurity, and a host of those. You need to be able to do that. And obviously, by the way, you need to do it with a wrapper of governance and operations with the same level of

fidelity, the same level of robustness around lineage metadata, et cetera, on public cloud and private cloud. So I do think whether it's LLMs, whether it's agents, whether it's frameworks, a lot of these will get to a territory whereby very large organizations will say, I want to be able to do hybrid AI. And that means I want to be able to build applications, AI applications, in a form factor of my choice, but also ported.

depending upon where the data resides, because otherwise it's super expensive. So that's the one core thing that, say, large organizations and practitioners should be wary of if you aren't already

That's wonderful. Yeah, so that seems absolutely fascinating, the idea you bring your data and your models in the same place. And if you've got big security concerns, that means you don't want it in a public cloud, you want it somewhere private. And so, but everything else, I guess, goes public cloud. So that's why you want the hybrid setup. All right, super. Any final advice then for organizations wanting to get better at AI? Three things.

AI is a team sport. You need to make sure all of the pieces are in order. Second, you need to get to a point whereby you're able to trust the data you're going to use to train your models and applications on to solve other data.

And third, you have to embed general AI within your normal processes to be able to scale that. Otherwise, it's a science experiment which never scales. All right. Great. Final wisdom there. Thank you so much for your time, Abbas. Thank you for hosting me and thank you for having me here, Richard. It was a lovely experience.