Enhancing photorealism enhancement with new machine learning in GTA V
Enhancing photorealism enhancement with new machine learning in GTA V
Yeah the Machine Learning from this is great. I have an Nvidia Shield Pro and sometimes I like to watch videos from YouTube on my 4k OLED. However, many times the quality is locked at 1080p, but the Shield actually bumps up the quality to 4k using AI (rather than the usual expand each pixel to more pixels approach).
I think people are missing the point, what you put in is what you get out-
if you train the network using high quality footage then the output will look a lot better.
This could have incredible applications in the future when doing this sort of thing on the fly is part of the rendering pipeline. - I'd particularly be interested in a Network trained to learn how Ray traced lighting works so it could guesstimate lighting conditions in areas of a game where it would be unnoticeable to save on performance.
People shat on DLSS until it boosted performance and image quality vs native.
For those interested Neural Networks are capable of guessing physics interactions to a high degree of accuracy when trained with examples of physics interactions.
Probably more research based. Building out real world datasets requires paying a driver, or scouring the web, much of it is conditional and it can be hard to train for specific conditions. This way you can mod the game to have a camera in a number of different vehicles, camera positions, walking/biking setups, weather conditions, lighting/tod etc. You can use the buffer to isolate specific assets. Then you mix in the rendered visuals with real world datasets, to help train some software.
I think nvidia was working on something similar using the unreal engine.
I'm not sure what the application would be though
Assuming you can run this real time, a softer touch of it could work to touch up some current post processing effects. The biggest thing I see is how much better ambient occlusion looked in that last stretch, though I suppose that would only really matter if it was more performant than ray tracing. You could also use it to color grade based on weather effects or something if you had a few different models set to different in game times and weathers
I mean, look forward another console generation, where AI processing is decided more trivial for the GPU, and you could see studios using a successor to this kind of technology for effects that are less trivial to achieve by hand.
I guess oddly, the best potential for this would be film adaptations, in that you could use the original film as a ground truth set to make the game look dramatically more authentic. Or switch between various styles of photorealism.
But honestly I expect this kinda stuff to be more useful for effects houses in compositing their work to existing footage for a TV show or film.
I think it’s not realism in that the end result looks like real life. It’s realism in that the end result looks nearly identical to a video taken by a real life dash cam. So, in that was it’s “realism”. I assume you could get a much better image by using a higher quality video feed for the network.
Members
Online