Magic Leap, in collaboration with NVIDIA, has made a breakthrough by developing immersive and photorealistic ray-traced digital twins. This strategic partnership aims to provide efficiency and value in various sectors, such as engineering, architecture, retail and more.
Enabling autonomous computing in Magic Leap 2
Magic Leap has partnered with NVIDIA to enable autonomous computing and streaming. 3D models at 1:1 scale in their augmented reality headset, Magic Leap 2. Thanks to this collaboration, business users can render and stream large-scale digital twins immersively from the platform NVIDIA Omniverse to Magic Leap 2 without compromising picture quality. In addition, an extension has been developed that integrates business simulation programs in the user interface.
Technical advances and outstanding features
The partnership between Magic Leap and NVIDIA has achieved significant technical advances, including:
- Ray tracing and Deep Learning Super Sampling (DLSS): These technologies allow you to reproduce content with exceptional detail and photorealistic quality.
- Magic Leap Dynamic Dimming Technology: Removes 99,7% of ambient light from the screen, allowing augmented reality to be used in various lighting conditions. lighting.
- Accurate Ambient Lighting and Digital Content Viewing: Provides an immersive experience by ensuring realistic lighting and accurate reflections.
- Depth-Based Reengineerability: Eliminates latency and enables permanent embedding of digital content into the user's environment.
Efficiency in 3D design and modeling
To improve the efficiency in the design and 3d modeling, as well as to allow the the most disruptive and professionals create accurate, photorealistic digital duplicates in Magic Leap 2, the Magic Leap Omniverse extension has been created. This extension plays content in tiempo real from the most common business modeling tools, such as Autodesk Maya, without the need for conversion or optimization. Content is stored in Omniverse Nucleus as USD files, where users can edit and collaborate in real time.
Streaming and remote rendering
Clicking "View in Magic Leap", the image is rendered on the server and instantly transmitted to Magic Leap 2, where it is displayed to the user in the correct position, orientation, and scale. Omniverse's flagship XR extension automatically generates high-quality stereo images and sends them over the API Industry standard OpenXR.
The OpenXR remote rendering server uses the acceleration of NVIDIA GPUs to combine and encode the stereo images, which are then transmitted over the network, received, decoded and displayed on Magic Leap 2. In addition, Magic Leap 2 continuously sends information about head position, eye tracking, controller status and other inputs to the server, allowing collaboration in real time and instant display of changes made to USD content.
Advantages and disadvantages
✅ Advantages:
- Rendering and streaming immersive of large-scale digital twins.
- Exceptional photorealistic quality and detail thanks to ray tracing and DLSS.
- Versatile use in various lighting conditions with dynamic dimming technology.
❌ Disadvantages:
- Early Access: The integration Magic Leap with Omniverse is currently in early access, limiting its general availability.
- Technical requirements: Implementing digital twins in enterprise environments may require infrastructure and Results suitable technicians.
FAQs
How does the partnership between Magic Leap and NVIDIA benefit enterprise users?
The partnership between Magic Leap and NVIDIA enables business users to render and broadcast large-scale digital twins in an immersive way, speeding development, fostering collaboration, and transforming design reviews and other experiences.
What featured technologies are used in the collaboration?
Featured technologies include ray tracing, Deep Learning Super Sampling (DLSS), Magic Leap's dynamic dimming technology, and depth-based reengineering capability.
What are the benefits of the Magic Leap Omniverse extension?
Magic Leap's Omniverse extension enables real-time playback of content from common enterprise modeling tools, such as Autodesk Maya, without the need for conversion or optimization.
When will the integration between Magic Leap and Omniverse be available?
The integration is currently in early access, with a general release expected in the first half of this year.
How do you achieve remote streaming and rendering in Magic Leap 2?
Remote streaming and rendering is achieved through the use of video acceleration. GPU from NVIDIA, which combines and encodes stereo images for transmission over the network.
This exciting advancement in immersive, photorealistic digital twins powered by Magic Leap and NVIDIA opens up new possibilities for businesses across multiple industries. As technology evolves and refines, digital twins are expected to play an increasingly relevant role in design, collaboration and business transformation.
April 12, 2023 - magic jump announced that coopera with NVIDIA pioneered immersive, photorealistic ray-traced digital twins at NVIDIA omniverse platformto create new efficiencies and value in engineering, architecture, retail and other industries.
Magic Leap partnered with NVIDIA to find a way to enable autonomous computing and streaming of 3:1 scale 1D models to its Magic Leap 2 augmented reality (AR) headset. According to Magic Leap, the collaboration enables enterprise users immersively render and stream large-scale digital twins from the NVIDIA Omniverse platform to Magic Leap 2 with no image quality degradation and includes an extension that integrates common business simulation programs into the user interface.
Technical advances that companies have worked on include:
- Ray tracing and Deep Learning Super Sampling (DLSS), which allows content to be reproduced with great detail and photorealistic quality;
- Magic Leap's dynamic dimming technology, which removes 99,7% of ambient light from the screen so AR can be used in a variety of bright lighting conditions;
- Precise ambient lighting and display of digital content;
- Depth-based re-engineering capability that eliminates latency and permanently embeds digital content into the user environment.
To help make 3D modeling and design more efficient, and to enable businesses and professionals who create, design, and collaborate to recreate physically accurate, photorealistic digital duplicates on the Magic Leap 2 device, Magic Leap created the Omniverse Extension, which plays content on the headset of common enterprise modeling tools, such as Autodesk Maya, without the need for conversion or optimization. Content is stored in Omniverse Nucleus as USD files. In Nucleus, users can edit and collaborate on live content.
By simply clicking 'View in Magic Leap', the image is rendered on the server and transmitted to Magic Leap 2, where it is instantly displayed to the user in the correct position, orientation and scale. Omniverse's core XR extension automatically generates high-quality stereo images and sends them to the industry-standard OpenXR API.
The OpenXR remote rendering server environment then uses NVIDIA GPU acceleration to combine and encode the stereo images. These images are then transmitted over the network where they are received, decoded, and displayed on Magic Leap 2.
Magic Leap 2 also continuously sends head positions, eye tracking, controller status, and other input to the server, from where it's streamed to Omniverse via OpenXR. On the backend, other users can continue to edit live USD content, with changes instantly replayed and streamed to Magic Leap 2.
Magic Leap said that by leveraging a number of features from Magic Leap 2 and NVIDIA RTX GPUs, the fix helps mitigate common issues, including:
- Reprojection reduces latency and locks content in place: Magic Leap 2 can support over 100 milliseconds of end-to-end latency while maintaining a user experience immersive;
- NVIDIA RTX ray tracing and DLSS provide photorealistic quality;
- Dynamic dimming technology makes content opaque and adds behaviorally realistic shadows;
- Precise ambient lighting and reflections ensure immersion. The company added that it is actively exploring ways to make the feature even more realistic with dynamic lighting evaluation.
Magic Leap said it will continue to work with NVIDIA and leverage its advances in digital twin technology to deliver greater value to enterprise users. With more immersive and accessible digital twins, equipment They can accelerate development, deepen collaboration, and transform design reviews and other experiences.
Magic Leap's integration with Omniverse is currently in Early Access, with a general release expected in the first half of this year. To learn more about working with Magic Leap to implement digital twins in the company and business, click here.
Image Credit: Magic Leap
READ MORE ARTICLES ABOUT: VR News.
READ THE PREVIOUS POST: Bandai Namco Entertainment: Investments in AI and the Metaverse of the Future.