VLM-Guided Semantic Safety for Onboard Aerial Autonomy

Geometric collision avoidance alone is not enough for aerial robots that operate around people and valuable assets. A drone should also understand what it is flying over and near, and adjust its behavior based on the semantics of the scene. Vision-language models open a path to this kind of semantic safety reasoning directly onboard the robot.

Our first work in this direction, “VLM-Guided Semantic Safety for Onboard Aerial Autonomy” by Tarunkumar Palanivelan, Anirudh Nallawar, and Prof. Guanrui Li, was accepted at two IROS 2026 workshops, the 3rd AI Meets Autonomy workshop and the Workshop on Embodied Neuro-Symbolic AI for Reliable and Safe Robotics (ReS AI). This student project builds on that work. Students will help extend the framework, improve its onboard performance, and validate it in real flight experiments.

Desired Skills

Students are not expected to have experience in every area, but familiarity with one or more of the following is desirable.

• Deep learning with PyTorch, ideally with vision-language or other foundation models

• ROS 2 or other robotics middleware

• An interest in onboard computing, perception, and safe autonomy