Microsoft's DirectML has the potential to become a game-changer for the next generation Xbox console. It should allow developers to create next generation..
lordsofgaming.net
Xbox Series X DirectML: A Next-Generation Game Changer
June 1st, 2020
Marcin Featured,
News,
Tech,
Xbox 1 Comment
Microsoft’s DirectML has the potential to become a game-changer for the next-generation Xbox console. It should allow developers to create next-generation games that run at 4K resolution but also run at a higher frame rate, offering ray tracing at the same time.
Xbox Series X supports Machine Learning for games with DirectML, a component of DirectX. DirectML leverages unprecedented hardware performance in a console. Benefiting from over 24 TFLOPS of 16-bit float performance, and over 97 TOPS (trillion operations per second) of 4-bit integer performance on Xbox Series X. Machine Learning can improve a wide range of areas, such as making NPCs much smarter, providing vastly more lifelike animation, and greatly improving visual quality. Today we will focus on improving visuals.
AI Upscale of Low-Resolution Textures In Real-time
First let’s look at the work of Playfab, a company that Microsoft acquired back in early 2018. They have been working on making tools for the back-end of games supported in the cloud. Playfab is using the power of Microsoft’s Azure servers & AI to upscale low-resolution textures in real-time. Playfab’s GM, James Gwertzman talked with
VentureBeat on some of the things he and his team are working on.
“One of the studios inside Microsoft has been experimenting with using ML models for asset generation. It’s working scarily well. To the point where we’re looking at shipping really low-res textures and having ML models uprez the textures in real-time. You can’t tell the difference between the hand-authored high-res texture and the machine-scaled-up low-res texture, to the point that you may as well ship the low-res texture and let the machine do it”
Xbox Series X Machine Learning – SDR to HDR Conversion
Back in March
Eurogamer’s Digital Foundry was presented with the Xbox Series X using machine learning to convert SDR to HDR in a few games including Halo 5 in real-time.
“We got to see the Xbox One X enhanced version of Halo 5 operating with a very convincing HDR implementation, even though 343 Industries
never shipped the game with HDR support. Microsoft ATG principal software engineer Claude Marais showed us how a machine learning algorithm using Gears 5’s state-of-the-art HDR implementation is able to infer a full HDR implementation from SDR content on any back-compat title.”
Machine Learning Upscaling
Machine learning upscaling could potentially be the most impactful of the features as it improves both performance and visuals. We already have a great implementation of this tech on PC. It’s called DLSS 2.0.
“Deep Learning Super Sampling (DLSS) uses artificial intelligence and machine learning to produce an image that looks like a higher-resolution image, without the rendering overhead. AI algorithm learns from tens of thousands of rendered sequences of images that were created using a supercomputer. That trains the algorithm to be able to produce similarly beautiful images, but without requiring the graphics card to work as hard to do it.”
This means that developers will have an option to render a game at 1080p or 1440p. Then they can upsample it to 4K with little loss to image quality. Rendering a game in 1080p will also allow devs to utilize spare GPU resources for better graphics. Resources such as graphical effects, Ray Tracing, and/or better frame rate. How developers use it is up to them, and I’m sure it will depend on the type of game they make. DLSS on PC is usually used in games that use Ray tracing, allowing games to run at a high frame rate and with high image quality- upscaled 1440p or 4K.
According to developers, there may be even more uses for Machine Learning in gaming. Developers are still figuring out how to properly use DirectML, so I’m excited to see what the future will bring. Improved AI and physics would be a huge win for me.
www.forbes.com
