FireBot – An Autonomous Surveillance Robot for Fire Prevention, Early Detection and Extinguishing

FireBot presents an innovative concept for fire prevention, fire detection, and extinguishing. It utilizes state-of-the-art technologies that enable autonomous navigation, including avoiding all obstacles, video surveillance, fire prevention and detection, and fire extinguishing. It has a LiDAR and a depth camera, as well as the IR and ultrasonic sensors used in RTAB, a cutting-edge SLAM (simultaneous localization and mapping) algorithm for autonomous mapping and navigation, and a modern convolutional neural network (CNN) paired with infrared thermal (IRT) and RGB cameras for fire and temperature anomaly detection. Furthermore, it has various other sensors for monitoring the surroundings and detection of potential anomalies (various gas sensors, microphone for detecting water and gas leaks, intrusion detection). FireBot is also equipped with a fire extinguisher paired with the rotating mechanical hand on top of which is an electronic nozzle that can be precisely directed to the source of fire for fast and efficient extinguishing.

Each year, fire causes a significant number of fatalities as well as significant material losses. As a result, society now places a high importance on fire prevention and early detection, and thus, it is also the main area of study and development for many scientists and different sectors. Rapid technological advancement, particularly in the areas of robotics, embedded systems, and machine learning, is having an impact on how the field of firefighting is developing. This field is becoming more effective and safe and consequently, minimizing the danger for the people and the property, as well as the firefighters. There are some existing robotic solutions for firefighting in the area of forest firefighting as well as some robotic systems for outdoor firefighting. Over time, there have been several attempts to develop an autonomous robot for firefighting that uses advanced methods for navigation and mapping as well as different image processing techniques for fire prevention and detection. Due to the many challenges involved, such as real-time prevention and detection, complex mapping and navigation inside narrow and often dark areas, changes of environment, obstacle avoidance, and balancing the processing power with energy consumption, none of those attempts resulted in a commercially available solution. Our work is focused on developing an autonomous robot for fire prevention, fire detection, and extinguishing. It utilizes state-of-the-art technologies that enable autonomous navigation, including avoiding all obstacles, video surveillance, fire prevention and detection, and fire extinguishing. It has a LiDAR and a depth camera, as well as the IR and ultrasonic sensors used in RTAB, a cutting-edge SLAM (simultaneous localization and mapping) algorithm for autonomous mapping and navigation, and a modern convolutional neural network (CNN) paired with infrared thermal (IRT) and RGB cameras for fire and temperature anomaly detection. Furthermore, it has various other sensors for monitoring the surroundings and detecting potential anomalies (various gas sensors, a microphone for detecting water and gas leaks, intrusion detection). FireBot is also equipped with a fire extinguisher paired with the rotating mechanical hand, on top of which is an electronic nozzle that can be precisely directed to the source of fire for fast and efficient extinguishing. FireBot system architecture consists of the three main logical components for indoor fire prevention, early detection, and fire extinguishing: an embedded system module for sensor and actuator management (ESMSAM), a system module for SLAM and navigation (SMSN), and a system module for fire prevention and detection (SMFPD). Another important module is a system module for fire extinguishing (SMFE), which consists of three different fire extinguishing devices (powder, foam, and CO2) along with the movable arm and electronic nozzle. Furthermore, a charging station is provided separately from the robot.

When it comes to fire detection using a visual camera, our main goal was to create a dataset that can be used to train, validate, and test custom or existing convolutional neural network (CNN) architectures that can detect fire in input images. To enhance the precision of fire detection even further and to localize the fire in input images, two approaches are used, semantic segmentation and object detection. The goal of semantic segmentation is to cluster the parts of an image together that belong to the same object class. The main task of object detection is to detect instances of objects of a certain class within an image. After the image segmentation and object detection, in addition to knowing whether the fire is present in an image, we also know the exact location of the fire on that image. This is critical for directing the fire extinguishing nozzle to the proper location for fast and effective fire extinguishment.

When it comes to fire prevention by determining potential temperature anomalies, an infrared thermal camera is used. Infrared thermal imaging is the best way to capture the temperature range characteristic of the Earth’s electromagnetic spectrum. The idea for anomaly detection is as follows, FireBot will have a predefined route for patrolling during which it will actively search for a fire using a visual camera. On that route, it will have predefined points of interest that present a potential danger or contain some expensive equipment that needs to be monitored regularly, for example, computers, electric machines, electrical cabinets, server racks, etc. Every time FireBot comes to a certain point of interest, it will take a thermal image of that scene, detect the hotspots, calculate the area of each hotspot, average temperature, and maximum temperature of every hotspot, and compare that data to the data gathered from all previous passes. If the detected temperatures are increased in comparison to the previous states, the number of hotspots is increased, or the area of a hotspot is increased beyond the predetermined threshold, it will be a trigger for an alarm, and an automatic warning message will be sent to the supervisor along with the location of that point of interest.

In addition to the visual and IRT cameras, various other sensors for monitoring the surroundings and detection of potential anomalies will be implemented. Various gas sensors will detect the gas products of burning different materials, so an anomaly can be detected even if there is no direct line of sight with the anomaly. A microphone will also be implemented along with a sound processing algorithm to detect sound anomalies such as water and gas leaks, explosions, breaking sounds, etc.

When combining all subsystems together, FireBot will present a complete and innovative solution for autonomous navigation, fire prevention, fire detection and extinguishing. It will be capable of replacing the night guard / firefighter on duty by its capabilities for early awareness and anomaly detection.