don't expect new chips , more transistors and ram . there will be a "flagship" with a trillion transistors and all the ram that most of us wont be able to buy and all the affordable configurations will be a.i. model upgrades.

"AI" is a buzzword, even bigger than 3D before.

I watched that entire 1 hour long video from DF - very interesting. Yes the main point was that DLSS 3.5 generated (path traced and ray-reconstructed de-noising) frames are now *better* than could be generated natively. And their view that old style rendering technique (rasterization) are inherently "fake" at every step of the pipeline and so old style rendering is the actual "fake" frames, and AI based rendering is more real and produces more real quality truth than old style fake rendering pipelines. ie. we have now passed the point where the old ways of doing it are now much worse than this new way in every way - there are no longer any downsides mostly. And I would 100% agree with them.

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling? Sure, some details might look better, but the whole thing about AI upscaling is that it's giving a best effort to fill in missing data. It can (and does) do an incredible job, but when going across frames, I don't get how an AI could do better. But let's say for a moment you're enabling DLSS 3.5 without upscaling, so, the input resolution is the same as the output: while I get how an AI could help with things like denoising or smoothing edges, I don't understand how it could otherwise make a more realistic image than with it off. The AI has to be trained on what "more realistic" is supposed to look like, so how is it supposed to do that in a fictional universe? What trained it to know what a better render is supposed to look like, if all it can be trained on is rastered renderings? That's like giving a definition to a word by using the word in the definition. Or, that's like trying to describe what the color red looks like to a blind person. So, how does it make sense to generate a more realistic image when the AI doesn't have real-world images?

schmidtbag:

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling? Sure, some details might look better, but the whole thing about AI upscaling is that it's giving a best effort to fill in missing data. It can (and does) do an incredible job, but when going across frames, I don't get how an AI could do better. But let's say for a moment you're enabling DLSS 3.5 without upscaling, so, the input resolution is the same as the output: while I get how an AI could help with things like denoising or smoothing edges, I don't understand how it could otherwise make a more realistic image than with it off. The AI has to be trained on what "more realistic" is supposed to look like, so how is it supposed to do that in a fictional universe? What trained it to know what a better render is supposed to look like, if all it can be trained on is rastered renderings? That's like giving a definition to a word by using the word in the definition. Or, that's like trying to describe what the color red looks like to a blind person.

They are saying its better entirely due to the denoising improvements. When you turn RT on the game fires a ton of rays into a scene but not enough to generate an image without a ton of noise. Natively, the game has human tuned denoising algorithms that run and try to clean up the picture for the final image. These denoisers are imperfect, they need to be hand tuned for specific scenes, remove lots of detail and create issues with other effects in the game (GI on certain objects). On the other hand the AI denoisers are able to run slightly faster than the human ones and they also make much better decisions about how to denoise the image because they are trained on data sets that contain much higher ray counts than the native image is even generating.

geogan:

I watched that entire 1 hour long video from DF - very interesting. Yes the main point was that DLSS 3.5 generated (path traced and ray-reconstructed de-noising) frames are now *better* than could be generated natively. And their view that old style rendering technique (rasterization) are inherently "fake" at every step of the pipeline and so old style rendering is the actual "fake" frames, and AI based rendering is more real and produces more real quality truth than old style fake rendering pipelines. ie. we have now passed the point where the old ways of doing it are now much worse than this new way in every way - there are no longer any downsides mostly. And I would 100% agree with them.

So, you simply believe what some guy from Nvidia has said? Becuase i take what employes of companies say with a gigantic bucket of salt, for obvious reasons...

schmidtbag:

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling? Sure, some details might look better, but the whole thing about AI upscaling is that it's giving a best effort to fill in missing data. It can (and does) do an incredible job, but when going across frames, I don't get how an AI could do better. But let's say for a moment you're enabling DLSS 3.5 without upscaling, so, the input resolution is the same as the output: while I get how an AI could help with things like denoising or smoothing edges, I don't understand how it could otherwise make a more realistic image than with it off. The AI has to be trained on what "more realistic" is supposed to look like, so how is it supposed to do that in a fictional universe? What trained it to know what a better render is supposed to look like, if all it can be trained on is rastered renderings? That's like giving a definition to a word by using the word in the definition. Or, that's like trying to describe what the color red looks like to a blind person. So, how does it make sense to generate a more realistic image when the AI only has a fictional universe to refer to?

One remote possibility is that the AI is so advanced that it manages to understand what the game wants to show before the image is generated, improving the image so much that it ends up better than the it would it it was created " normally" by a programmer. For example, a pistol, a programmer could try to create a great model of it that the game will then display, AI could maybe pick on the concept ot the pistol and improve the image even further than normal. I don`t know how much sense this makes, and even if it makes sense i really doubt that GPUs will be able to do something similar in the future, specially because they have to operate inside the contraints of the game engine. Quite frankly, this smells of marketing BS.

H83:

One remote possibility is that the AI is so advanced that it manages to understand what the game wants to show before the image is generated, improving the image so much that it ends up better than the it would it it was created " normally" by a programmer. For example, a pistol, a programmer could try to create a great model of it that the game will then display, AI could maybe pick on the concept ot the pistol and improve the image even further than normal. I don`t know how much sense this makes, and even if it makes sense i really doubt that GPUs will be able to do something similar in the future, specially because they have to operate inside the contraints of the game engine.

I think what you said makes sense but I don't think that's realistically possible for a game. It'd be fine for still images but that kind of prediction frame-by-frame is bound to have distracting artifacts.

Why does it only have to be trained on raster images? Why couldn't it be trained on datasets that contain pathtraced images at extremely high sample counts? (fwiw the lead dev of DLSS said it's trained on raytraced datasets)

H83:

So, you simply believe what some guy from Nvidia has said? Becuase i take what employes of companies say with a gigantic bucket of salt, for obvious reasons...

In that video it was implied that they are talking about a rastered native image vs a AI Reconstructed RT image. The point being that AI reconstruction allows more physically accurate techniques to be possible and thus exceeding the visual level of a natively rastered image.

Denial:

Why does it only have to be trained on raster images? Why couldn't it be trained on datasets that contain pathtraced images at extremely high sample counts? (fwiw the lead dev of DLSS said it's trained on raytraced datasets) In that video it was implied that they are talking about a rastered, native image vs a AI Reconstructed RT image.

I didn't watch the video but in the original article from Tom's Hardware, they are comparing native rendering against DLSS rendering, saying that the extra performance enabled by the later can be used on RT or PT, creating a better result native, due to the extra effects. He's not wrong on that aspect but it's a little misleading, at least for me.

Denial:

Why does it only have to be trained on raster images? Why couldn't it be trained on datasets that contain pathtraced images at extremely high sample counts? (fwiw the lead dev of DLSS said it's trained on raytraced datasets)

Well, to do that implies those pathtraced images are still artificially created. After all, I don't know how you would go about getting real-world path tracing datasets. I guess you could argue that maybe they could train the AI using significantly more rays, or perhaps rays that bounce more times.

schmidtbag:

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling? Sure, some details might look better, but the whole thing about AI upscaling is that it's giving a best effort to fill in missing data. It can (and does) do an incredible job, but when going across frames, I don't get how an AI could do better. But let's say for a moment you're enabling DLSS 3.5 without upscaling, so, the input resolution is the same as the output: while I get how an AI could help with things like denoising or smoothing edges, I don't understand how it could otherwise make a more realistic image than with it off. The AI has to be trained on what "more realistic" is supposed to look like, so how is it supposed to do that in a fictional universe? What trained it to know what a better render is supposed to look like, if all it can be trained on is rastered renderings? That's like giving a definition to a word by using the word in the definition. Or, that's like trying to describe what the color red looks like to a blind person. So, how does it make sense to generate a more realistic image when the AI doesn't have real-world images?

https://forums.guru3d.com/threads/info-zone-gengines-ray-tracing-dlss-dlaa-dldsr-tsr-fsr-xess-and-mods-etc.439761/page-112#post-6169139 They do compare not native resolution RT image versus upscaled RT image, but native resolution with old rasterizing tricks (the case before DLSS 3.5) versus upscaling with ray reconstruction (DLSS 3.5).

H83:

I didn't watch the video but in the original article from Tom's Hardware, they are comparing native rendering against DLSS rendering, saying that the extra performance enabled by the later can be used on RT or PT, creating a better result native, due to the extra effects. He's not wrong on that aspect but it's a little misleading, at least for me.

I don't see that in the original article? But if it is then the article isn't summarizing the video correctly imo. It's less of a comparison of the image quality of DLSS to native and more a comparison of pathtracing to rasteriziation with the argument that reconstruction enables pathtracing. Basically the PCMR guy says the community is concerned about "fake frames" and the CD Project Red guy follows that up by saying its' really no different than mipmaps, culling, etc - that DLSS is just a tool and some developers will use that as a crutch, other developers will use it to enable things like pathtracing which will yield image quality results better than a natively rendered raster image. I think the argument is kind of like, what makes an image "real" and Nvidia's argument is that pahtracing - simulated light - is more real than raster hacks and DLSS enables that. And while some pixels are inferred from training data, the final image that DLSS generates is more "accurate" to how light would actually behave in a scene than a native raster image would ever generate.

mbk1969:

https://forums.guru3d.com/threads/info-zone-gengines-ray-tracing-dlss-dlaa-dldsr-tsr-fsr-xess-and-mods-etc.439761/page-112#post-6169139 They do compare not native resolution with RT image versus upscaled RT image, but native resolution with old rasterizing tricks (the case before DLSS 3.5) versus upscaling with ray reconstruction (DLSS 3.5).

[Quote]NVIDIA's Bryan Catanzaro stated that not only is DLSS 3.5 more beautiful than native rendering, but in a way, its frames are more real when coupled with path tracing than native rendering with the traditional rasterized approach. This is heavily emphasized in the video and I feel like the articles are just ignoring it. It's very obvious in the video that the context is about a native raster image vs DLSS enabling technologies like pathtracing. The previous context is the one guy bringing up the idea that dlss "fake" and his argument is that dlss+pathtracing is more "real" than a native rasterized image would ever be able to achieve essentially. I think everyone should just go watch the video, it's super interesting and worth watching.

schmidtbag:

Well, to do that implies those pathtraced images are still artificially created. After all, I don't know how you would go about getting real-world path tracing datasets. I guess you could argue that maybe they could train the AI using significantly more rays, or perhaps rays that bounce more times.

They don't go into that much detail but the Nvidia chief DLSS guy in the video basically says they train the AI on a dataset that includes high quality path traced images from omniverse. So yeah, its definitely getting data that's way higher quality than anything in any natively rendered videogame.

schmidtbag:

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling? Sure, some details might look better, but the whole thing about AI upscaling is that it's giving a best effort to fill in missing data. It can (and does) do an incredible job, but when going across frames, I don't get how an AI could do better. But let's say for a moment you're enabling DLSS 3.5 without upscaling, so, the input resolution is the same as the output: while I get how an AI could help with things like denoising or smoothing edges, I don't understand how it could otherwise make a more realistic image than with it off. The AI has to be trained on what "more realistic" is supposed to look like, so how is it supposed to do that in a fictional universe? What trained it to know what a better render is supposed to look like, if all it can be trained on is rastered renderings? That's like giving a definition to a word by using the word in the definition. Or, that's like trying to describe what the color red looks like to a blind person. So, how does it make sense to generate a more realistic image when the AI doesn't have real-world images?

The way AI is being used is all about more efficiently using and retaining information. The more information you can keep the better the output can be. So AI in particluar: 1) can use temoral information in a way traditional rastering can't. This is touched on but essentially a traditionally rasterised frame throws away the previous frame and generates the new one from scratch. As the two frames will look very similar by throwing away a lot of useful information is lost. AI is much more capable of using that information so can generate a better output. 2) can combine stages. There are several denoising stages, and then and upscaling pass. Each stage going into the next loses information. In 3.5 it combines all of these into one operation using AI, so because that one operation has a much better input before any information was lost it can produce a much better output. In addtion it's just flat out faster then splitting it up.

All of these AI(software) enhancements are nice. But its a slippery slope as NVidia has proved with the 40 series over the 30 series. No generational improvements for some cards, but instead rely on AI (software) +upscaling to generate more frames. which equals more $$$$ since they can save money by skimping out on the hardware side. Hopefully next generation will be more of a happy balance, between the two, instead of going strait to software for performance improvements.

schmidtbag:

Something about this doesn't really make sense to me: First and foremost, how do you produce a better-than-native image when upscaling?

Nothing really to do with up-scaling or frame generation alone - its to do with using real world like path traced lighting and then the new AI de-noising as part of the DLSS 3.5 process... this is not part of existing games DLSS 2.x... only talking about latest ones like Cyberpunk.

H83:

So, you simply believe what some guy from Nvidia has said? Becuase i take what employes of companies say with a gigantic bucket of salt, for obvious reasons...

No... because I understand what he is talking about (and I have a degree in computer science from a major university and did specialized classes in computer graphics & 3D graphics over 20 years ago) and what he said makes sense to me... I also have some AI coding knowledge now... and I have seen it with my own eyes in current Cyberpunk Overdrive without even the latest de-noising in 3.5... there is simply no comparison to crappy old raster techniques for lighting (faked baked lighting in most games can fool a lot of people though since it comes close).

H83:

One remote possibility is that the AI is so advanced that it manages to understand what the game wants to show before the image is generated, improving the image so much that it ends up better than the it would it it was created " normally" by a programmer. For example, a pistol, a programmer could try to create a great model of it that the game will then display, AI could maybe pick on the concept ot the pistol and improve the image even further than normal. Quite frankly, this smells of marketing BS.

It appears you don't know much about the technical side of this stuff at all so your opinion that this is marketing BS I can tell you is completely wrong.

It depends a lot on the game content. I recently played the Dead Space remake at 1440p. Even with the LOD mods for DLSS, DLSS Quality wasn't as good as native TAA. But for the most part, I find that DLSS Quality is more than acceptable - great, even, for clearing up temporal artifacts. At 1440p in Cyberpunk, DLSS Balanced is no bueno.

geogan:

Nothing really to do with up-scaling or frame generation alone - its to do with using real world like path traced lighting and then the new AI de-noising as part of the DLSS 3.5 process... this is not part of existing games DLSS 2.x... only talking about latest ones like Cyberpunk. No... because I understand what he is talking about (and I have a degree in computer science from a major university and did specialized classes in computer graphics & 3D graphics over 20 years ago) and what he said makes sense to me... I also have some AI coding knowledge now... and I have seen it with my own eyes in current Cyberpunk Overdrive without even the latest de-noising in 3.5... there is simply no comparison to crappy old raster techniques for lighting (faked baked lighting in most games can fool a lot of people though since it comes close). It appears you don't know much about the technical side of this stuff at all so your opinion that this is marketing BS I can tell you is completely wrong.

Fair enough. You clearly know much more about this than i do, so i´ll take your word, but i continue to believe that they are exaggerating their claims. Anyway, for those interested TPU already has an article about this: https://www.techpowerup.com/review/nvidia-dlss-35-ray-reconstruction/

mattm4:

All of these AI(software) enhancements are nice. But its a slippery slope as NVidia has proved with the 40 series over the 30 series. No generational improvements for some cards, but instead rely on AI (software) +upscaling to generate more frames. which equals more $$$$ since they can save money by skimping out on the hardware side. Hopefully next generation will be more of a happy balance, between the two, instead of going strait to software for performance improvements.

I think you miss-understand what AI really means. It's essentially using a super computer to write code instead of humans. So in the past smart humans write all the code for the game and your computer runs it. With AI you take a super computer, tell it what you want it to do and it writes the code, which your computer runs. The AI super computer writes code differently and not surprisingly in some ways it's better at it then even the smartest humans. It's the start of a coding revolution. Nvidia have the AI super computers, know how to train them and have put hardware in their cards to run the code it generates. This allows them to do some things better. It's not a slippery slope or skimping, it's just smarter use of hardware. In 10-20 years time a lot more will be written by AI because as it develops it'll get better and better at writing code. Obviously there is a different slippery slope where we hand over the keys to everything to our AI overlords...

Gamer Nexus's last video was quite informative on the effectiveness of the new approach for me. Lots of clips and comparisons vs stills which served better as a reference point. [youtube=zZVv6WoUl4Y] As for the technical side, definitely interesting ideas, some of it is definitely marketing speak though in how wonderful this improvement is when it is still a developing and evolving concept, but it seems to deliver decent visual fidelity improvements upon what was previously available. I would like to see their implementation vs UE5 implementation for denoising comparisons, as UE5 report their algorithm is comparable to offline ray tracing rendering, which is an impressive claim.

H83:

Fair enough. You clearly know much more about this than i do, so i´ll take your word, but i continue to believe that they are exaggerating their claims. Anyway, for those interested TPU already has an article about this: https://www.techpowerup.com/review/nvidia-dlss-35-ray-reconstruction/

This was interesting, from the TPU article: "An interesting discovery is that Ray Reconstruction actually lowers VRAM usage. For example at 4K we measured 11.8 GB with RR disabled, and only 10.9 GB with RR enabled. One possible theory is that the default denoiser (which gets replaced by RR) has a higher memory usage, possibly because it keeps more history frames in its buffer. What might also help is that RR is integrated with the DLSS Super Resolution pass, which means some buffers can be shared and don't get duplicated."