PSI-VQA: Evaluating Multimodal LLM Social Reasoning under Ambiguous Pedestrian Intent in Automated Driving
Published in 2026 European Conference on Computer Vision - 10th AI City Challenge Workshop, 2026
Recommended citation: Shaozhi Wang, Yin-Chun Lu, Zheng Tang, Han Zhang, David C. Anastasiu and Renran Tian. "PSI-VQA: Evaluating Multimodal LLM Social Reasoning under Ambiguous Pedestrian Intent in Automated Driving". Proceedings of European Conference on Computer Vision Workshops (ECCVW 2026). 2026. https://openreview.net/forum?id=icgcFuZ88R
[Paper]
Abstract
PSI-VQA is a video question-answering benchmark for evaluating multimodal LLM social reasoning under ambiguous pedestrian intent in automated driving. The benchmark reuses and extends human annotations from the PSI dataset and asks models to predict whether a pedestrian will cross, list visual cues for and against crossing, select the correct cue from multiple options, and locate the moment that drives the decision. It contains 1,475 questions over 244 dashcam videos and reveals that current multimodal LLMs remain far from solved on fine-grained pedestrian-intent reasoning.
BibTeX
@InProceedings{Wang26PSIVQA,
author = {Shaozhi Wang and Yin-Chun Lu and Zheng Tang and Han Zhang and David C. Anastasiu and Renran Tian},
title = {{PSI-VQA}: {E}valuating multimodal {LLM} social reasoning under ambiguous pedestrian intent in automated driving},
booktitle = {Proc. ECCV Workshops},
address = {Malm{"o}, Sweden},
year = {2026}
}