Using drone swarms to monitor and combat future wildfires
Wildfires are a major environmental threat posed by climate change. The FireMan consortium is developing AI-based technology for fast detection of wildfires and for creating situational awareness during fires by using unmanned aerial systems.
Image: Lauri Markelin
Fabrice Saffre, Research Professor at VTT Technical Research Centre of Finland, opens a simulation screen with green and grey dots marking flammable material and rocky areas in a forest. Nothing happens until he ignites a virtual fire with a click of the mouse: the fire quickly spreads, and three drones start tracking it. “This is a simplified model of a forest to simulate a fire event and what happens when we have drones monitoring the event,” he explains. “The drones follow the event as best as they can by keeping a safe distance to the propagating wildfire.”
This simulation has been created in the FireMan project by Saffre to showcase how drone swarms could be used in real-life wildfire situations in Finland or anywhere in the world in the future. The semi-autonomous system under development in the FireMan project will be demonstrated in Finland in 2024. Climate change has caused huge wildfires around the world and the need for efficient monitoring systems is increasing.
By using thermal infrared cameras, the drones are able to measure the source of the heat. By exchanging position data with each other, the drones are able to form a monitoring perimeter around the fire that responds to changes in its geometry as it propagates. When downwind, drones stay further away from the blaze, resulting in the swarm organizing itself into an ellipse rather than a circle to ensure that all units remain a safe distance from the flames. Depending on how many drones you have at your disposal, the system will adapt the best it can. In addition, the detection of multiple fire events is being developed in the FireMan project.
The key point is that the swarm operates without detailed control by a human operator. There are different optical sensor technologies used to detect fires: visual and thermal infrared cameras are typically used, but multispectral and hyperspectral cameras could also be possible. Miniaturized cameras are also possible in the future.
“Basically, in the FireMan project, we are looking for answers to how novel disruptive Unmanned Aerial Systems (UASs) can be used for fast, cost-efficient and low-emission detection of wildfires. The UAS solution also works very well in creating situational awareness and as decision support tools for the commanders of fire brigades,” says Senior Scientist Hannu Karvonen from VTT.
“We hope that the weather will be suitable for us to arrange drone test flights in prescribed forest-burning events in the coming summers in various locations around Finland. Eventually, we will see how a fully or partly autonomous swarm system could be used for monitoring a wildfire and help firefighting crews in their critical decision-making. We will also evaluate and define the roles of human operators and their interaction with a drone swarm system and other stakeholders,” says Karvonen.
Do AI-based drones change the work of the firefighters?
“We aim to provide fire brigades with better situational awareness for less effort thanks to the automation of routine tasks,” says Saffre. “We don’t yet know the details of our final solution, but it will definitely involve drone swarms and automation. A swarm monitored by a human operator can help the fire brigade coordinate their activities based on the data received. It’s possible that in a few years, the drones will have the capability to put out fires all by themselves with nature-friendly fire retardants as well,” says Saffre.
Saffre’s role in the project is related to the development of collective intelligence to improve situational awareness of the drones. “One of the main advantages in using swarms is that by simply clicking on a map, you are able to give them high-level instructions like ‘go to this area and give me as much information as possible about the propagation of the fire’. After that, the drones would set a mission plan and calculate their own flight paths.”
“Specifically, there would be less manual and routine work needed by human operators to plan and operate everything thanks to the high automation level. A dedicated data analyst may also be needed to help the fire brigade leader make correct and fast decisions in different dynamic situations,” says Karvonen, alluding to how the current working roles may change.
In the future, a fully integrated system that includes satellite images, quadcopters and fixed-wing drones could result in an improved response time, particularly in wilderness areas where other early-warning systems may be lacking. Currently, fire monitoring flights in Finland are made with small aeroplanes. Could that be done more efficiently and in a more environmentally friendly manner with drones that are able to intelligently detect a fire and recruit other drones to that area?
“This is where the integration of autonomous platforms will play a big role. Especially in wilderness areas—such as the Amazon, where you don’t necessarily have an early detection system—you could have a satellite image indicating a fire somewhere in a very remote place. In that case, you might want to send an aircraft to monitor the situation and confirm if it’s a real fire or not. If there is a fire, you go to next level where you involve quadcopters to monitor it closer. You would have a detection advantage if you used a highly automated system,” says Saffre.
Remote-controlled drones are already used in fighting forest fires, especially in Southern Europe and California, but not ‘intelligent’ swarms of partially autonomous UAS, according to the scientists’ knowledge.
“A part of our objective is to demonstrate how feasible it would be to enable one pilot to operate several drones with the support of swarm intelligence. The utilization of remote platforms, drones or even ground stations in the monitoring of hazardous sites requires lot of work. One of the origins of the project is the idea to have a fleet of drones where one pilot is piloting a single drone while the other drones are organized around it. This flying fleet formation would give more diverse angles to the same event and provide more information,” says Saffre.
The first test flights with quadcopter drones were conducted in Karkkila in May 2022 with the National Land Survey of Finland. Additional demonstrations with different types of drones, cameras and optical sensors will follow in 2022 and 2023, providing that weather conditions enable the controlled forest burning exercises. This will allow the research teams to study how a semi-autonomous swarm system could be used for monitoring a wildfire and help firefighting crews in decision-making. In addition, the roles of the human operators will be evaluated and defined.
Image: Lauri Markelin
“The objective is to design distributed decision-making algorithms that will scale with the swarm size. If they’ve got only two drones to play with, they’ll do the best they can. With 10 drones, the performance will increase accordingly. The size of the monitored area depends on the drone model that is used,” describes Saffre.
In the FireMan project, the following AI technologies are used: deep learning for data analytics (mostly offline), computer vision for object recognition and navigation (real-time), and collective intelligence for swarm operations. The objective is for the swarm to be autonomous but capable of executing instructions if required. One important aspect is to make the swarm monitoring and operation as easy and intuitive as possible for the human operator. For instance, the operator may designate an area of interest and leave the swarm to ‘decide’ how to proceed with the reconnaissance mission execution, freeing the operator from detailed planning.
When will we have AI-based swarm of drones monitoring wildfires?
“It depends on the skill we are talking about. We could see some of these monitoring techniques deployed in 5 years, but a fully autonomous swarm capable of detecting, monitoring and suppressing a wildfire without human intervention is probably 20 years away. We will be seeing a progression from the very first prototype to a fully automated integrated solution,” says Saffre.
The FireMan results will be showcased to the target users at several events. Saffre emphasizes the need for feedback. “One example has been the self-positioning of drones around a wildfire. Before discussing with professionals, we assumed they would distribute themselves isometrically around the blaze. We have now incorporated the notion that downwind is intrinsically less safe than upwind, and we factored that into the optimal observation distance calculation. So now the simulated drones essentially organize themselves into an ellipse instead of a circle,” he says.
Plenty of jobs for swarms of drones
Saffre foresees plenty of other AI applications for the future. “Any kind of propagating disaster that requires monitoring is potentially amenable to similar techniques. One example is oil spills or floating debris in open water. Given the right sensors, it should also be possible to monitor and find the edge of a plume, to ‘sniff’, for example, an airborne chemical,” he says.
Drone swarms could be used in a wide variety of public safety applications, for example, in exploring the ruins of a collapsed building following an earthquake or to locate missing persons in the wilderness.
More about The FireMan project
The FireMan project is funded by the Academy of Finland, and the total budget of the project is 2.8 million euros for the years 2022–2024. The research consortium comprises the Finnish Geospatial Research Institute, the University of Oulu, VTT, and the University of Jyväskylä. Over 50 authorities and companies are involved as collaborators. More information: https://www.maanmittauslaitos.fi/en/research/fireman