Translating road signs in real time onto heads up displays

Abstract

We propose a heads-up display (HUD) system that augments driver navigation and safety by translating road signs in real time. This is particularly relevant in regions where travelers frequently cross boundaries - both territorial and and linguistic - such as in Europe or Asia. This system enables drivers to quickly identify the critical information without having to divert their attention from the road.

This system utilizes a vision-language model capable of recognizing and translating road signs across a wide range of source languages into a user-selected target language. These translations are then overlaid onto the screen in front of the driver. For the development and demonstration purposes of this project, we are using a Meta Quest VR Headset