Prof Angela Schoellig Utilizes ChatGPT to Choreograph Drone Swarms Safely

Prof Angela Schoellig Utilizes ChatGPT to Choreograph Drone Swarms Safely

(IN BRIEF) Professor Angela Schoellig from the Technical University of Munich (TUM) has harnessed the power of ChatGPT to develop choreographies for swarms of drones synchronized to music, while implementing an additional safety filter to prevent mid-air collisions. This innovative approach, termed “SwarmGPT,” combines ChatGPT’s text generation capabilities with a safety algorithm, ensuring precise flight paths and collision avoidance. Schoellig’s research team has successfully demonstrated the system’s effectiveness, enabling rapid development of choreographies for drone swarms with minimal human intervention. Moreover, this advancement marks a significant leap forward in drone choreography design, drastically reducing development time and complexity. Schoellig envisions broader applications for ChatGPT in robotics, potentially revolutionizing interface design for various robotic tasks beyond drone choreography.

(PRESS RELEASE) MUNICH, 29-Apr-2024 — /EuropaWire/ — Prof. Angela Schoellig from the Technical University of Munich (TUM) uses ChatGPT to develop choreographies for swarms of drones to perform along to music. An additional safety filter prevents mid-air collisions. The researchers’ results demonstrate for the first time that large language models (LLMs) such as ChatGPT can be used in robotics.

The web interface is easy to use: Doctoral student Martin Schuck selects a music track and enters a text requesting a suggested choreography. Additional instructions can be given to the drone swarm via another prompt in the ChatGPT tool before an algorithm checks whether the suggested flight paths are feasible. The screen in the Learning Systems and Robotics Lab now shows a simulated airfield with six drones flying in circles to the music. If the scientist from the Chair of Safety, Performance and Reliability for Learning Systems at TUM likes this choreography, he logs it in. Soon afterwards, six palm-sized drones take off from the floor of the robotics lab.

Choreography with three flying robots is completed within five minutes

In Prof. Angela Schoellig’s laboratory, the research team has installed six ceiling-mounted cameras in a room measuring around 40 square meters and three meters high. Crosses are marked on the floor with insulating tape. These indicate the starting positions of the drones. Once the computer has verified a possible choreography, they can take off. The cameras detect the position of the quadrocopters, which are equipped with four propellers and motors, 200 times a second. The system compares them with the desired position. The “airshows” in the Learning Systems and Robotics Lab, which the research team realizes with up to nine drones, are 100 percent safe today. Without the special safety filter, only one in four demonstrations is accident-free.

SwarmGPT: additional algorithm ensures safety of flying robots

Prof. Angela Schoellig combined ChatGPT with the safety filter for the “Dance of the Flying Robots”. “The ChatGPT AI tool was primarily created to generate texts, but it can also suggest choreographies,” says the professor. “However, it initially knows nothing about the properties of drones and physical limits for the flight paths. So it is clear that ChatGPT makes mistakes.”

The additional safety algorithm closes this gap by mapping out flight paths for the proposed choreography so precisely that mid-air collisions are completely avoided. Drones can even approach each other diagonally. Prof. Schoellig calls the overall concept of ChatGPT and security filter designed to use several flying robots “SwarmGPT”. The tool generates the processes in the air and at the same time serves as an interface between the robot and the human, who does not require any expert knowledge.

A quantum leap through ChatGPT

When Angela Schoellig began her drone research almost 15 years ago, choreographies were hand-crafted. It took more than three years to develop the first six choreographies for six drones and get them up and running. “ChatGPT has brought about a quantum leap,” says Prof. Schoellig. Over the past three months, the researchers have experimented with over 30 choreographies for up to nine drones. Today, it takes the researchers only around five minutes to develop a safe choreography for 30-second music clips with three drones. The more drones are added, the more time ChatGPT needs for its calculations and the longer it takes to propose a choreography. But Schoellig is certain: “The concept is scalable.”

Robotics: ChatGPT becomes an interface for non-experts

Can other robots be used with a similar interface via ChatGPT? For robots that use voice control to pick up objects, lay cables or open doors, the success rate of those actions is currently only 63, 56, and 80 percent. So far, applications in other robotics scenarios have been somewhat unreliable. For Prof. Angela Schoellig, this is an incentive: “I assume that our approach will keep getting better in other scenarios, too.” It may soon be possible to reprogram suction-based and industrial robots simply by voice command without the need for expert knowledge or programming skills.

Publications

Vivek K. Adajania, Siqi Zhou, Arun Kumar Singh, Angela P. Schoellig
AMSwarm: An Alternating Minimization Approach for Safe Motion Planning of Quadrotor Swarms in Cluttered Environments;  https://ieeexplore.ieee.org/document/10161063

Further information and links

• Prof Angela Schoellig is Head of the Chair of Safety, Performance and Reliability for Learning Systems at TUM and Director of the Munich Institute of Robotics and Machine Intelligence (MIRMI). At TUM’s Robotics and AI Institute, more than 70 professors are  focusing on developing robotic and intelligent solutions in the fields of health, the environment, mobility, and work.
• Swarm-GPT: Combining Large Language Models with Safe Motion Planning for Robot Choreography Design, NeurIPS 2023 workshop paper

Media Contact:

Corporate Communications Center
Andreas Schmitz
presse@tum.de

Contacts to this article:

Prof. Angela Schoellig
Chair of Safety, Performance and Reliability for Learning Systems
Technical University of Munich (TUM)
Angela.schoellig@tum.de

SOURCE: Technical University of Munich