VLM-Guided Semantic Safety for Onboard Aerial Autonomy

Fri, 02 Oct 2026 12:30:00 +0000

Geometric collision avoidance alone is not enough for aerial robots that operate around people and valuable assets. A drone should also understand what it is flying over and near, and adjust its behavior based on the semantics of the scene. Vision-language models open a path to this kind of semantic safety reasoning directly onboard the robot.

Our first work in this direction, “VLM-Guided Semantic Safety for Onboard Aerial Autonomy” by Tarunkumar Palanivelan, Anirudh Nallawar, and Prof. Guanrui Li, was accepted at two IROS 2026 workshops, the 3rd AI Meets Autonomy workshop and the Workshop on Embodied Neuro-Symbolic AI for Reliable and Safe Robotics (ReS AI). This student project builds on that work. Students will help extend the framework, improve its onboard performance, and validate it in real flight experiments.

Desired Skills

Students are not expected to have experience in every area, but familiarity with one or more of the following is desirable.

• Deep learning with PyTorch, ideally with vision-language or other foundation models

• ROS 2 or other robotics middleware

• An interest in onboard computing, perception, and safe autonomy

Safety-Critical Control of Quadrotors with High-Order Control Barrier Functions

Sat, 25 Jul 2026 20:07:33 +0000

Modern autonomous drones must operate safely in complex and cluttered environments. While many existing collision avoidance methods assume that thrust commands can be executed instantaneously, real quadrotors are subject to actuator dynamics that introduce delays between the commanded and actual motor responses. Ignoring these dynamics can degrade safety and performance during aggressive maneuvers.

This project aims to develop a High-Order Control Barrier Function (HOCBF) framework that explicitly accounts for rotor-speed dynamics in the flight control loop. Students will design, implement, and experimentally validate safety-critical controllers on a real quadrotor platform. The project includes controller development, simulation in ROS 2 and MuJoCo, and deployment on embedded flight-control hardware.

Desired Skills

Students are not expected to have experience in every area, but familiarity with one or more of the following is desirable:

• Basic control systems (state-space, control barrier function)

• Experience with flight controllers (e.g., PX4, ArduPilot, Pixhawk) or embedded systems

• Hardware integration or electronics debugging

• Python and/or C++ programming, ROS 2 and Linux

• Experience with simulation tools (e.g., MuJoCo) is a plus

Students with strong motivation to learn are encouraged to apply even if they do not meet all of the above qualifications. Paper publication is expected for this project.

Application Window: now - 16th Sep, 2026

Safety | ACP Lab

VLM-Guided Semantic Safety for Onboard Aerial Autonomy

Desired Skills

Safety-Critical Control of Quadrotors with High-Order Control Barrier Functions

Desired Skills