Geospatial data provides rich real-world environmental and contextual information, spatial relationships, and real-time monitoring capabilities for applications…
Geospatial data provides rich real-world environmental and contextual information, spatial relationships, and real-time monitoring capabilities for applications in the industrial metaverse.
Recent years have seen an explosion in 3D geospatial data. The rapid increase is driven by technological advancements such as high-resolution aerial and satellite imagery, lidar scanners on autonomous cars and machines, improvements in 3D reconstruction algorithms and AI, and the proliferation of scanning technology to handheld devices and smartphones that enable everyday people to capture their environment.
To process and disperse massive heterogenous 3D geospatial data to geospatial applications and runtime engines across industries, Cesium has created 3D Tiles, an open standard for efficient streaming and rendering of massive, heterogeneous datasets. 3D Tiles are a streamable, optimized format designed to support the most demanding analytics and large-scale simulations.
Cesium for Omniverse is Cesium’s open-source extension for NVIDIA Omniverse. It delivers 3D Tiles and real-world digital twins at global scale with remarkable speed and quality. The extension enables users to create real-world-ready models from any source of 3D geospatial content—at rapid speed and with high accuracy—using Universal Scene Description (OpenUSD).
With Cesium for Omniverse, you can jump-start 3D geospatial app development with tiling pipelines for streaming your own content. You can also enhance your 3D content by incorporating real-world context from popular 3D and photogrammetry applications such as Autodesk, Bentley Systems, and Matterport.
For example, you can integrate Bentley’s iTwin model of an iron ore mining facility with Cesium for project planners to visualize and analyze the facility in its precise geospatial context. With Cesium for Omniverse, project planners can use a digital twin of the facility to share plans and potential impacts with local utilities, engineers, and residents, accounting for location-specific details such as weather and lighting.
One of the most intriguing features of the extension is an accurate, full-scale WGS84 virtual globe with real-time ray tracing and AI-powered analytics for 3D geospatial workflows. Developers can create interactive applications with the globe for sharing dynamic geospatial data.
New opportunities for 3D Tiles with OpenUSD
Just as Cesium is building the 3D geospatial ecosystem through openness and interoperability with 3D Tiles, NVIDIA is enabling an open and collaborative industrial metaverse built on OpenUSD. Originally developed by Pixar, OpenUSD is an open and extensible ecosystem for describing, composing, simulating, and collaborating within 3D worlds.
By connecting 3D Tiles to the OpenUSD ecosystem, Cesium is opening new possibilities for customization and integration of 3D Tiles into metaverse applications built by developers across global industries. For example, popular AECO tools can leverage OpenUSD to add 3D geospatial context streamed by Cesium to enable powerful workflows.
To further interoperate with USD, developers at Cesium created a custom schema in USD to support their full-scale virtual globe (Figure 2).
Cesium’s virtual globe is a digital representation of the earth’s surface based on the World Geodetic System 1984 (WGS84) coordinate system. It encompasses the earth’s terrain, oceans, and atmosphere, enabling users to explore and visualize geospatial data and models with high accuracy and realism.
Creating a full-scale virtual globe
“Leveraging the interoperability of USD with 3D Tiles and glTF, we create additional workflows, like importing content from Bentley’s LumenRT for Omniverse, Trimble Sketchup, Autodesk Revit, Autodesk 3ds Max, and Esri ArcGIS CityEngine into NVIDIA Omniverse in precise 3D geospatial context,” said Shehzan Mohammed, director of 3D Engineering and Ecosystems at Cesium.
In Omniverse, all the information for the globe such as tilesets, imagery layers, and georeferencing data is stored in USD. USD is a highly extensible and powerful interchange for virtual worlds. A key USD feature is custom schemas, which you can use to extend data for complex and sophisticated virtual world use cases.
Cesium’s team developed a custom schema, with specific classes defined for key elements of the virtual globe. The C++ layer of the schema actively monitors state changes using the OpenUSD TfNotice system, ensuring that tilesets are updated promptly whenever necessary. Cesium Native is used for efficient tile streaming. The lower-level Fabric API from Omniverse is employed for tile rendering, ensuring optimal performance and high-quality visual representation of the globe.
The result is a robust and precise WGS84 virtual globe created and seamlessly integrated within the USD framework.
Developing the extension
To develop the extension for Omniverse, Cesium’s developers leveraged Omniverse Kit, a low-code toolkit to help developers get started building tools. Omniverse Kit provides sample applications, templates, and popular components in Omniverse that serve as the building blocks for powerful applications.
Omniverse Kit supports both Python and C++. The extension’s code was predominantly written in Python, while the tile streaming code was implemented in C++. Communication between the Python code and C++ code uses a combination of PyBind11 bindings and Carbonite plug-ins where possible.
During the initial stages of the project, the team heavily relied on the kit-extension-template-cpp as a reference. After becoming familiar with the platform, they began to take advantage of Omniverse Kit’s highly modular design, and developed their own Kit application to facilitate the development process. This application served as a common development environment across Cesium’s team where they could establish their own default settings and easily enable often-used extensions.
Cesium used many existing Omniverse Kit extensions, like omni.example.ui and omni.kit.debug.vscode, and created their own to streamline task execution. For instance, their extension Cesium Power Tools has more advanced developer tools, like geospatial coordinate conversions and syncing Sun Study with the scene’s georeferencing information. They plan on developing more of these extensions in the future as they scale with Omniverse.
High-performance streaming
Maintaining high-performance streaming for 3D Tiles and global content can be a challenge for Cesium’s street-level to global scale workloads. To address this, their team relied on the Omniverse Fabric API, which enables high-performance creation, modification, and access of scene data. Fabric plays a vital role in achieving optimal performance levels for Cesium, improving load speed, runtime performance, simulation performance, and availability of data on GPUs.
Building on Fabric, Cesium incorporated an object pool mechanism that enables recycling geometry and materials as tiles unload, optimizing resource utilization. Tile streaming occurs either over HTTP or through the local filesystem, providing efficient data transmission.
Getting started with Cesium for Omniverse
Cesium for Omniverse is free and open source under the Apache 2.0 License and is integrated with Cesium ion. This provides instant access to cloud-based global high-resolution 3D content including photogrammetry, terrain, imagery, and buildings. Additionally, industry-leading 3D tiling pipelines and global curated datasets are available as part of an optional commercial subscription to Cesium ion, enabling you to transform content into optimized, spatially indexed 3D Tiles ready for streaming to Omniverse. Learn more about Cesium for Omniverse.
Explore Cesium learning content and sample projects for Omniverse. To get started building your own extension like Cesium for Omniverse, visit Omniverse Developer Resources.
Attending SIGGRAPH? Add this session to your schedule: Digital Twins Go Geospatial With OpenUSD, 3D Tiles, and Cesium on August 9 at 10:30 a.m. PT.
Get started with NVIDIA Omniverse by downloading the standard license free, or learn how Omniverse Enterprise can connect your team. If you are a developer, get started with Omniverse resources to build extensions and apps for your customers. Stay up to date on the platform by subscribing to the newsletter, and following NVIDIA Omniverse on Instagram, Medium, and Twitter. For resources, check out our forums, Discord server, Twitch, and YouTube channels.