The Future of Visuals: Nvidia's Text-to-Video AI Generation
08:11 Share

Today, on the podd cast, we are going to be talking about being next step in A I. We have ChatGPT that is doing text generation. We have things mid journey that are really effectively doing image generation.

And now the next generation is video generation, and that is right around the corner in the video recently just announced and showed off a new text to video technology. And today on the podcast, we are going to be diving into this. What is current capabilities are and what the implications of are the what the implications of this are on the A I space at large.

So the first thing I want to say is that um a new research paper and microsite from videos toronto AI lab called high resolution video synthesis with latent diffusion models came out. That's where I guess this new research paper is kind of h what this is all built on. But essentially, it's giving us a taste of the incredible video creation tools that the video is about to launch and that are going to be the people are going be able to, uh, january.

And obviously, the video is for those that don't know, it's a it's a technology company that creates mostly chips. So most of these these big A I models open eye and a lot of these other guys are training their A I models on individual chip. So it's individual's best interest to help develop A I technology.

Um so a they could take advances of IT, but b my assumption is they are gonna. You create some good video adding video creation technology so that it's here because it's obviously to be incredibly resource intensive and that is going to drive the sales of all the individual at chips AI training chips. So that's my opinion.

Um it's really interesting. no. And obviously a incredible breakthrough. So latent diffusion models or L D, ms are essentially type of A I that can generate video without needing really massive computing power, right? Like obviously, this takes a relatively computing power, but in the past this would a bit insane and now it's becoming more manageable.

So the video said that its tech does this by building on the work of text image generators. So in this case, stable the fusion um and they add a what they call a temporal dimension to the latent space to fusion model. So that sounds fancy.

But really all that thing is that essentially it's generated A I can make still images that you could generate unstable the fusion or majority or something like that move in a realistic way. And then I will upscale those images or increase the quality using some super resolution techniques. And this just means that you can essentially, you can produce a four point seven second long video with the revolution tion of a twelve eighty by twenty forty eight.

So that's actually pretty great resolution. Um and you can also do longer ones if you're in a lower the resolution of IT. If you doing like five hundred by new ten twenty four, I can do IT can make the videos a little bit longer I think a couple minutes or yes, I think um maybe two, three, four times as long.

So you know, I think the immediate reaction that I have to all of this and seeing these because I have a couple of interesting demos. They got one work like a storm. Trier's is on the beach and his vacation and the waves and IT looks like there's vacuum tube become out.

The back was lagging and kind of just attaches to a shadow behind him. So that's kind of funny. It's obviously like glitch in the in the video editor or an image editor.

And then IT turned into videos, the the back in tubes kind of moving, but they have one like that. They got one word just like this, you know, stuff to bear playing an electric guitar. And so I think that this is obviously, you know, a really big move, just because we can see where this is going in the industry as a whole.

But I think like for where the technologies is out right now, it's immediately people going to use this to make gifts. I think that's a big thing, right? These are five second videos that you can generate so immediately this can be used to create gifts.

And then in the future um I obviously i'm going to shock us, does this uh get Better Better and people are able to make a lot more robust longer videos. So you know I like you're able to essentially me in these videos are really simple prompt. So this give me something similar to what you get on generally just say, a storm troper vacuum on the beach and boom IT generated an image.

And look, everybody i'm looking at now, or you could say, you know, a teddy bear is playing the electric guitar, high definition for k kind like maid journey where you have like commas and all these extra things, you could do the thing um and yet so I think right now that makes the text of video technology that india is currently demon. It's really just like I say, it's most suitable for thumbnails and gifts and um those kind of smaller things, but obviously can be useful. And I think this is a really big step in the direction that we would like to go, which is going to be people being able to generate a lot longer video scenes.

Um and I think that we are problem. I can have to wait too longer given the um given the speed of the industry right now. Now before we started seeing those really complex and and bigger videos coming out.

So I think IT is interesting and important to know that NVIDIA isn't the first company to kind of show off some of this ai text to video generators. Um recently google um made a debut with fani, I think I was called um and essentially they had like a twenty second clip um video that they could create based on some longer prompts. So in their demo, IT was showing um something that was I think over two minutes long.

So google did IT a little bit longer. I mean, from what I thought of that, I was I would say probably a little bit less quality, but I would be interesting to see where that goes. I'm really happy there's a lot of different companies that are competing for this because obviously, don't just want one that complete corn is the market.

So i'm happy that google in the video, a lot of these guys are working on IT believe that I was gonna start to see a lot of cool stuff to start up runway, which helped create the text image generator stable diffusion. I'm also revealed its jenie A I video model last month and so along. When they did that, they had like A A video that if the prompt for IT was the late afternoon sun peeking through the window of a new york city loft, and I watched the video to IT looks a lot like a gift.

IT looks kind of glitch. This the shadows are kind of uh, not perfect, but like he really is not not crazy to think that this is not very far off before. Um this is a lot this this will be a lot more realistic and a lot longer. And you're going to be able to you know essentially have an idea for you know I want a star wars movie maybe with um James Cameron playing the lead role and b blab bligh it's set the thousand nine hundred and twenty years and instead of light sabbath they have guns and alcohol darva like you are able to say something like crazy stuff like that and it's going to just create the video.

So I think to be really interesting, some people that are talking about this are talking about the fact that when a some of these videos, studios released movies around the world you could have the main character uh reflected and looking like A A celebrity from different geographical locations or just like a genetic person from different geographical oca's so that i'll be really interesting there's a lot of interesting use cases. This A I technology obviously once a hits video is going to be absolutely insane. Um and IT doesn't seem like it's that far.

While people are pushing in the direction, advantages are being made in the direction. So it's really interesting. Um and I in addition, all of this we have a lot of just like video editing software that's coming out that is integrated with A I we have adobe firefly um which uh just came out that is going to kind of tackle that um in programs like I don't be premier rush, just gonna be able to type in the time of day or season you want to see in your video and adobe's AI is gonna do the rest.

So I think it's can be pretty interesting um obviously right now read after about what it's capable, we see more like creating GPS. Um but I don't think it's can be far to um I don't think can be a long shot to have these things creating more fully ly fledge video. So we really interesting space continue to watch in the future.