Advancing AI with NVIDIA Omniverse at CVPR 2024: AI City Challenge Highlights
Published:
As the lead organizer of the AI City Challenge at the Computer Vision and Pattern Recognition (CVPR) 2024 conference, I’m excited to share our progress, particularly with the creation of the largest indoor synthetic dataset using NVIDIA Omniverse.
The AI City Challenge draws over 700 teams from nearly 50 countries to develop AI models for improving operational efficiency in various physical settings. This year, NVIDIA Omniverse played a pivotal role, providing datasets for tasks like retail, warehouse management, and intelligent traffic systems.
In large indoor spaces like factories and warehouses, AI models require extensive data for training, which is often time-consuming and costly to collect manually. To overcome this, we used physically based simulations and digital twins created with NVIDIA Omniverse. These virtual environments generate synthetic data essential for training AI models to operate effectively and safely.
For this year’s AI City Challenge, we focused on the Multi-Camera Person Tracking track, the most popular with over 400 teams. NVIDIA provided a dataset with 212 hours of 1080p video at 30 frames per second, covering 90 scenes across six virtual environments, including warehouses, retail stores, and hospitals. These scenes, created in Omniverse, simulated nearly 1,000 cameras and featured around 2,500 digital human characters, allowing teams to test and refine their AI models accurately.
Our challenge saw global collaboration with ten prestigious institutions, including the Australian National University, Johns Hopkins University, and the Emirates Center for Mobility Research. These partnerships highlight the worldwide effort to advance AI technologies for smart cities and industrial automation.
Generative physical AI, combining reinforcement learning in simulated environments with high-fidelity physics-based simulation, is set to transform infrastructure automation and robotics. NVIDIA’s latest innovations, like the Omniverse Cloud Sensor RTX microservices, will speed up the development of fully autonomous systems by providing accurate sensor simulation in realistic virtual environments.
At CVPR 2024, NVIDIA Research will present over 50 papers on generative physical AI with applications in autonomous vehicle development and robotics. Highlights include unified 6D pose estimation and tracking, digital twin creation of unknown articulated objects, and customizable dataset generation via simulation.
For those interested in these advancements, NVIDIA offers a free standard license for Omniverse and extensive resources to get started. Join the Omniverse community on platforms like Instagram, Medium, LinkedIn, and Discord to stay updated and connect with other developers and researchers.
Together, we are driving AI innovation, creating smarter, safer, and more efficient environments for all.
