One of the most meaningful parts for me was seeing two benchmarks I supported, VANTAGE-Bench and Traffic Anomaly Reasoning (TAR), highlighted as part of the Cosmos 3 release story. Both benchmarks focus on the kind of operational video understanding that matters for physical AI: fixed cameras, long videos, small but important events, spatial-temporal reasoning, and explanations that go beyond object detection.

VANTAGE-Bench evaluates vision-language models on real-world fixed-camera footage across warehouse/logistics, transportation, and smart-space domains. The benchmark contains 3,346 image and video assets with 35,027 expert-curated annotations, organized around semantic, spatial, temporal, and spatio-temporal understanding. I helped prepare VANTAGE-Bench as a NeurIPS 2026 competition effort, supporting the benchmark and leaderboard framing around the “Infrastructure AI Gap”: how well VLMs can produce physically grounded insights from fixed infrastructure cameras under realistic deployment constraints.

TAR is the foundation of AI City Challenge 2026 Track 3: Anomalous Events in Transportation, part of the 10th AI City Challenge at ECCV 2026. TAR moves traffic anomaly evaluation from binary detection toward multi-task reasoning, with 44,040 training annotations across 10 task types covering 3,670 CCTV transportation videos. The track asks models to detect, reason about, and explain traffic anomalies through question answering, temporal reasoning, causal linkage, scene description, and video summarization. I supported the launch alignment for TAR as the Track 3 benchmark, helping keep the evaluation and leaderboard direction focused and ready for the Cosmos 3 public launch.

Together, these efforts show a broader shift in video AI evaluation: from recognizing visible objects to understanding what happened, why it happened, and what might happen next. They also connect to my broader MetroAI direction on multimodal AI for cities, transportation, and infrastructure, where video summarization, event understanding, and embedding-based video search are becoming central tools for operational intelligence. It was exciting to see that work connected to the Cosmos 3 launch and to the larger push toward physical AI systems that can reason over real-world environments.

This milestone continues a decade of AI City Challenge workshops and competitions, bringing together researchers and practitioners working on computer vision, multimodal AI, intelligent transportation, smart cities, and large-scale video analytics.

The 2026 challenge focuses on Sim2Real transfer, unified reasoning, and smart-city video AI across six tracks:

1. Multi-Camera 3D Perception, for Sim2Real multi-object tracking across camera networks.
2. Transportation Safety Understanding and Captioning, for captioning and VQA around pedestrian-centric risk.
3. Anomalous Events in Transportation, for detecting, localizing, and explaining traffic anomalies.
4. Text-Based Person Re-Identification, for natural-language retrieval by appearance and actions.
5. Generative Traffic Video Forecasting, for future-frame generation from video history and text descriptions.
6. Cross-City Object Detection, a Milestone Systems Hafnia track for geographic generalization.

I am grateful to continue helping organize this community as it moves from detection-centric perception toward richer reasoning, prediction, and deployment-aware evaluation for real-world infrastructure.

The demo showed how VSS can help developers build visual AI agents that review events, understand physical context, and streamline decisions across large volumes of live and recorded video. The workflow brought together search, summarization, Q&A, active alerts, VLM/CV/model implementation, and event review, with examples spanning smart cities, warehouses, and factories.

My main contribution focused on the video embedding search stack behind the demo. I helped prepare search models and datasets, finalized embedding models and person attribute search checkpoints, generated example queries and captions for warehouse and other video scenarios, and supported evaluation data preparation for demo clips and KPI validation. I also helped debug and validate key pieces of the real-time search path, including text embedding behavior, model packaging, and search-profile integration.

This work connected directly to the broader VSS search roadmap. In the VSS 3.1.0 release, the Search Agent Workflow added attribute search, multi-embedding fusion search, and a critic agent for reviewing search results. The release also updated the RT-CV microservice to support embedding generation for detected objects, including RADIO-CLIP and SigLIP2 embedding models.

For me, the most rewarding part was seeing several threads come together in one visible experience: video embeddings for event and activity search, object/person attribute search through vision-language embeddings, model packaging for deployment, and practical demo workflows that make large video archives feel searchable and useful.

We highlighted the nine-year evolution of the AI City Challenge, which now spans multi-camera 3D perception, traffic safety reasoning, warehouse spatial intelligence, and fisheye object detection—reflecting the growing needs of smart cities and intelligent infrastructure.

Our talk also introduced NVIDIA’s cloud-native, streaming multi-camera tracking workflow, combining DeepStream-based perception, MTMC fusion, real-time location tracking (RTLS), and a full Sim2Deploy pipeline powered by Omniverse, TAO Toolkit, and Metropolis microservices.

We shared progress on MCBLT, a BEV-based multi-camera 3D detection and hierarchical GNN tracking framework that achieves state-of-the-art results on AI City Challenge and WildTrack benchmarks.

Finally, we presented Sparse4D, an end-to-end multi-camera 3D perception model that jointly predicts 3D bounding boxes, tracking, velocity, and ReID embeddings—ranking #1 on the AICity’25 vision-only leaderboard.

It was great connecting with researchers from Google Mobility AI, DeepMind, Columbia University, and many others working to build the next generation of intelligent urban systems. Looking forward to continued collaboration across the UrbanAI and AI City Challenge communities.

This demo illustrates the future of Physical AI: unifying simulation, multimodal perception, and large-scale intelligence to power safer, more efficient, and more autonomous industrial environments. Grateful to our teammates across Metropolis, Robotics, and Omniverse who made this possible.

In parallel, I was also honored to represent NVIDIA Metropolis as host of the 9th AI City Challenge Workshop at ICCV 2025 in Honolulu. This year, the challenge brought together 245 teams from 15 countries, pushing forward research in:

• Multi-camera 3D perception
• Traffic safety reasoning
• Warehouse spatial intelligence
• Edge-optimized fisheye detection

The challenge datasets—including PhysicalAI-SmartSpaces and WTS Traffic Safety—were downloaded nearly 400,000 times on Hugging Face, reflecting the global momentum behind large-scale spatial AI benchmarks.

Additionally, I was invited as a keynote speaker at the 8th Workshop on Benchmarking Multi-Target Tracking (BMTT), where I presented NVIDIA’s recent advances in:

• MCBLT (BEV-SUSHI): hierarchical GNN-based multi-camera 3D tracking
• Sparse4D: end-to-end multi-camera 3D perception
• Integration with SUSHI GNN for scalable, long-range association

Special thanks to collaborators Yizhou Wang, Sameer Pusegaonkar, and our partners in Laura Leal-Taixé’s group, whose contributions continue to push the boundaries of multi-camera 3D understanding.

Congratulations to all participants, winners, reviewers, and organizers who made this year’s workshops a success. Looking forward to driving the next wave of Spatial AI, digital twins, and large-space perception together with the community.

As a lead contributor, I helped drive the creation and integration of this large-scale synthetic dataset—designed specifically for multi-target multi-camera (MTMC) tracking, 3D occupancy prediction, and smart infrastructure simulation. It is one of the most comprehensive synthetic datasets ever released for spatial AI, capturing warehouse, factory, and public space scenarios with fine-grained annotations and synchronized multi-view video.

Built with NVIDIA Omniverse and aligned with our Metropolis AI workflows, the dataset enables the development of robust computer vision systems for real-world deployment. From training deep learning models to validating digital twin applications, Physical AI Smart Spaces serves as a benchmark-ready resource for cutting-edge research and development.

We’re proud to empower the global research community with open tools and high-quality data. Whether you’re developing advanced trackers, occupancy estimators, or large-scale analytics systems, this dataset is built to accelerate your innovation.

Explore the dataset here:
👉 huggingface.co/datasets/nvidia/PhysicalAI-SmartSpaces

The Mega Omniverse Blueprint is NVIDIA’s comprehensive framework for simulating, deploying, and scaling robot fleets in highly realistic, physics-informed digital twins. These virtual environments, powered by the NVIDIA Omniverse platform, allow organizations to design, test, and optimize robotics and AI systems before deploying them in the real world.

Our contribution through Metropolis Spatial AI agents includes real-time location systems (RTLS) powered by advanced multi-camera tracking, 3D perception using our BEV-SUSHI framework, and GNN-based tracking modules. These components enable accurate and scalable monitoring of autonomous agents—such as robots, vehicles, and humans—within massive industrial environments like warehouses and factories.

In collaboration with teams across NVIDIA, we integrated Metropolis Spatial AI into the Omniverse ecosystem alongside Isaac Sim™ and other robotics tools. The result is a tightly integrated digital twin solution for spatial intelligence—demonstrated in factory and warehouse scenarios during the CES keynote.

This marks a major step forward for real-time vision AI in digital twins, unlocking new levels of automation, safety, and operational efficiency. I’m beyond excited to see where this journey leads as we continue to push the boundaries of AI-powered infrastructure.

The Best Presentation Award recognized “BEV-SUSHI: Multi-Target Multi-Camera 3D Object Detection and Tracking in Bird’s-Eye View”, a collaboration with NVResearch-DVL. The work was authored by Yizhou Wang, Tim Meinhardt, Orcun Cetintas, Chris Yang, Sameer Satish Pusegaonkar, Benjamin Missaoui, Sujit Biswas, Zheng Tang, and Laura Leal-Taixe.

The Best Poster Award recognized “From MTMC to RTLS: Towards Real-Time, Scalable Multi-Camera Object Tracking and Localization”, authored by Zheng Tang, Ganapathy Seshadri Cadungude Aiyer, Akshay Agrawal, Shuo Wang, Sameer Satish Pusegaonkar, Kartikay Thakkar, and Sujit Biswas.

Both projects reflect our broader effort to move multi-target multi-camera tracking from offline perception pipelines toward real-time location systems for large spaces, industrial automation, and spatial AI.

Our AI-powered multi-camera tracking system is designed to accelerate the development of vision AI applications, providing a comprehensive solution for measuring and managing infrastructure and operations across large spaces.

Imagine a world where factories operate automatically with maximum safety and efficiency, retail spaces are optimized for a superior shopper experience, and public areas like hospitals, airports, and highways are safer and more streamlined. Multi-camera tracking allows for accurate object tracking and activity measurement across multiple cameras and spaces, enabling effective monitoring and management.

NVIDIA’s customizable multi-camera tracking workflow offers a validated path to production, eliminating months of development time. The solution includes state-of-the-art AI models pretrained on real and synthetic datasets, customizable for various use cases. The workflow spans from simulation to analytics and integrates NVIDIA’s cutting-edge tools, including Isaac SIM™, Omniverse™, TAO, and DeepStream. It features real-time video streaming modules and is built on a scalable, cloud-native microservices architecture. With expert support and the latest product updates through NVIDIA AI Enterprise, this workflow accelerates vision AI projects without additional costs, apart from infrastructure and tool licenses.

Applications of multi-camera tracking include enhancing manufacturing and warehouse operations by optimizing routes for autonomous robots, equipment, and workers. It also improves retail store layouts by analyzing customer navigation to maximize sales and revenue, and enhances in-hospital patient care by providing continuous monitoring for safety and security.

Our workflow includes the entire development pipeline—from data generation to model training to application development—helping developers build complex vision AI applications for large spaces. Key components include creating 3D digital twins of real-world environments using NVIDIA Omniverse, simplifying agent simulation with NVIDIA Isaac Sim, streamlining model training with NVIDIA TAO Toolkit, and using NVIDIA Metropolis microservices for modular, cloud-native application building blocks.

Jumpstart the development of multi-camera vision AI applications with NVIDIA’s end-to-end workflow, from Omniverse simulation for synthetic data generation to TAO for streamlined model development to Metropolis microservices for modular, cloud-native application building blocks.

Join us in this exciting journey towards transforming how we monitor and manage large spaces with cutting-edge AI technology.

The AI City Challenge draws over 700 teams from nearly 50 countries to develop AI models for improving operational efficiency in various physical settings. This year, NVIDIA Omniverse played a pivotal role, providing datasets for tasks like retail, warehouse management, and intelligent traffic systems.

In large indoor spaces like factories and warehouses, AI models require extensive data for training, which is often time-consuming and costly to collect manually. To overcome this, we used physically based simulations and digital twins created with NVIDIA Omniverse. These virtual environments generate synthetic data essential for training AI models to operate effectively and safely.

For this year’s AI City Challenge, we focused on the Multi-Camera Person Tracking track, the most popular with over 400 teams. NVIDIA provided a dataset with 212 hours of 1080p video at 30 frames per second, covering 90 scenes across six virtual environments, including warehouses, retail stores, and hospitals. These scenes, created in Omniverse, simulated nearly 1,000 cameras and featured around 2,500 digital human characters, allowing teams to test and refine their AI models accurately.

Our challenge saw global collaboration with ten prestigious institutions, including the Australian National University, Johns Hopkins University, and the Emirates Center for Mobility Research. These partnerships highlight the worldwide effort to advance AI technologies for smart cities and industrial automation.

Generative physical AI, combining reinforcement learning in simulated environments with high-fidelity physics-based simulation, is set to transform infrastructure automation and robotics. NVIDIA’s latest innovations, like the Omniverse Cloud Sensor RTX microservices, will speed up the development of fully autonomous systems by providing accurate sensor simulation in realistic virtual environments.

At CVPR 2024, NVIDIA Research will present over 50 papers on generative physical AI with applications in autonomous vehicle development and robotics. Highlights include unified 6D pose estimation and tracking, digital twin creation of unknown articulated objects, and customizable dataset generation via simulation.

For those interested in these advancements, NVIDIA offers a free standard license for Omniverse and extensive resources to get started. Join the Omniverse community on platforms like Instagram, Medium, LinkedIn, and Discord to stay updated and connect with other developers and researchers.

Together, we are driving AI innovation, creating smarter, safer, and more efficient environments for all.