Comparative Analysis of Fire Detection Performance Using Deep Learning-Based Flame Segmentation Models
Article information
Abstract
The development of computational models designed to minimize fire damage by detecting fires early from image data using deep-learning models has attracted considerable research attention. However, these models have not yet been widely refined for practical applications. Therefore, in this study, the performance of flame segmentation models based on deep learning was comparatively evaluated in terms of detection accuracy, inference speed, and size; these models were the U-Net, DeepLabV3+, and YOLOv8-seg. When 1,000 images with and without fire were quantitatively analyzed, DeepLabV3+ exhibited the best segmentation accuracy, with a mean intersection over union (mIoU) of 72.9% and an F1-score of 83.6%, whereas the U-Net and YOLOv8-seg achieved mIoUs of 70.5% and 63.1% and F1-scores of 81.9% and 76.4%, respectively. Furthermore, YOLOv8-seg demonstrated the best performance in real-time fire detection owing to its lightweight architecture, achieving an inference speed of 145.2 fps with approximately 3.26 million parameters. These findings indicate that models for designing image-based real-time fire detection systems should be selected by comprehensively examining their accuracy, prediction speed, and size while considering the hardware performance of the intended detection device.
1. Introduction
The combustion pattern of a fire is determined on the basis of several factors, including the fuel supply after ignition, ventilation conditions, and distribution of nearby combustible materials. As these conditions interact in a complex manner, fire can spread to surrounding areas, which can potentially lead to large-scale fires [1-3]. Early fire detection and suppression are emphasized as key technologies in fire safety management because casualties and property damage can escalate rapidly. In particular, fires can occur in diverse spatial and background conditions as urban areas become denser and more complex with an increasing number of high-rise buildings. Accordingly, the demand for a technology that can accurately identify a fire’s point of origin and where the fire is spreading is growing to enable early detection of and appropriate response to fire incidents. Image-based fire detection technology is capable of meeting these demands because it can directly utilize visual information such as the shape, color, and movement of flame. This approach overcomes the limitations of existing detection methods based on specialized sensors, which often suffer from narrow detection ranges and malfunctions due to changes in the surrounding environment [4-5]. However, actual fire images often include notable variations in lighting, reflections, complex backgrounds, and objects with similar colors, which pose some notable challenges in accurately separating flame from the background in an image because the visual properties of flames can easily blend with those of their surroundings. Although previous fire detection methods based on color spaces are effective under relatively simple conditions, their performance is easily degraded by environmental changes, and non-flame regions of an image are often misidentified as flames, which would imply a false alarm in a real application [6-7].
Deep learning has been applied in image-based fire detection to overcome these limitations. In particular, object detection and segmentation technologies have been widely adopted as core methods to accurately identify flames and smoke against complex backgrounds. Yan et al. [8] proposed a YOLOv5-based model that combined coordinate attention with a Swin Transformer, which significantly improved detection performance for small flames and complex backgrounds while effectively reducing local feature bias present in existing YOLO-based architectures. Li et al. [9] significantly improved the average precision by stably extracting multi-scale features through a convolutional neural network (CNN)-transformer hybrid architecture by developing a detection-transformer-based flame and smoke detection model. In addition, Song et al. [10] proved that a high level of intersection over union (IoU) can be obtained from a limited computational environment by developing a lightweight segmentation model that integrates depthwise separable convolutions and a confusion block based on FusionNet. Choi et al. [11] considerably enhanced the accuracy of flame segmentation in outdoor images through an improved encoder-decoder architecture considering the characteristics of outdoor environment. Pan et al. [12] proposed the YOLO-FireAD model, which incorporates an attention-guided inverted residual module and a dual-pooling feature retention architecture to enable stable detection of small flames even under varied brightness and complex background conditions. Ali et al. [13] developed a fire classification, detection, and segmentation model using multi-sensor satellite images and compared different deep-learning architectures. They found that CNN-based models achieved the highest detection accuracy for images of wildfires. Furthermore, flame image analysis has been extended to high-dimensional applications based on the characteristics of fire, such as predicting heat release rates (HRRs), and research has been conducted to supplement existing experimental measurements with an analysis of the characteristics of a given flame from image data to estimate the HRR [14-16].
Although the performance of segmentation methods based on deep-learning models in detecting fires has been evaluated in numerous studies, critical factors, such as real-time processing performance, lightweight models, and robustness across different environments, for practical applications have not been comprehensively examined. In particular, the performance of different models has not been compared using datasets reflecting diverse conditions, such as indoor and outdoor environments, daytime and nighttime scenes, and variations in lighting, reflections, and obstacles, while considering real-world constraints such as the architecture of the model, the number of parameters, and inference speed. Accordingly, accuracy, real-time processing capability, computational efficiency, and size must be examined together for the development of image-based fire detection systems that can be implemented at real sites. Therefore, in this study, the performance of representative object segmentation models (the U-Net, DeepLabV3+, and YOLOv8-seg) based on deep learning was evaluated across diverse images with and without fire and the structural characteristics and real-time applicability of each model were systematically analyzed. A total of 1,000 images were used in the testing dataset to quantitatively compare flame segmentation accuracy using mean intersection over union (mIoU), precision, recall, and F1-score. The suitability of the models for practical applications was assessed based on indices of complexity such as inference speed, the number of parameters, and size.
2. Research Method
2.1. Deep-learning models for object segmentation
Object segmentation is a pixel-level labeling process that classifies each pixel in an image according to their semantic categories. This approach is widely applied as a core technology in computer vision for precise scene interpretation. Ulku & Akagunduz [17] defined object segmentation as “pixel-wise labeling of an image” and emphasized that it plays a key role in various applications where precise visual information is important. Object segmentation methods can be largely classified into three types according to their objective and level of detail. Semantic segmentation treats all regions belonging to the same class as a single semantic unit, whereas instance segmentation independently separates individual objects within the same class. Recently, panoptic segmentation has been proposed as an approach that integrates these two methods by allowing all pixels in a scene to be labeled consistently. In this study, the flame region was defined as a single class, and the objective was to identify the presence of flames in an image and their spatial distribution. Flames shown in photographs of fires lack clear boundaries and are constantly changing, with their regions expanding or shrinking depending on surrounding environmental factors such as smoke, lighting, and reflections. Therefore, semantic segmentation was applied to ensure that the entire flame is processed as a single semantic region. Furthermore, the spatial information required for fire detection was obtained reliably without complex boundary-tracking processes because flame regions are interpreted as sets of continuous pixels.
2.1.1 U-Net
In this study, the U-Net is adopted as a representative encoder-decoder-based segmentation model. The U-Net is a U-shaped network in which contracting and expanding paths are arranged symmetrically. These models are capable of precise pixel-level segmentation even with a small amount of data based on a fully convolutional network architecture. The contracting path typically adopts a CNN-based encoder architecture. After 3 × 3 convolutions followed by rectified linear unit activations are repeatedly applied, 2 × 2 max-pooling is used to reduce the spatial resolution by half while increasing the number of feature channels. Global features and context information of the image are extracted accordingly. The expanding path applies a 2 × 2 up-convolution to the upsampled feature map, which is then concatenated with the corresponding feature map from the contracting path at the same level to restore the spatial information lost in the encoder. The expanding path applies a 2 × 2 up-convolution to the upsampled feature map, which is then concatenated with the corresponding feature map from the contracting path at the same level to restore the spatial information lost in the encoder. A key structural feature of the U-Net is the direct transfer of high-resolution encoder features to the decoder through symmetrical skip connections to minimize the loss of boundary information and improve segmentation accuracy. Furthermore, all computations are based solely on convolutional operations without the use of fully connected layers to enable seamless segmentation of images with various resolutions regardless of input size. The overlap-tile strategy is applied to compensate for insufficient contextual information at image boundaries by using reflection-based image expansion. The overall architecture of the U-Net is shown in Figure 1 [18].
Architecture of U-Net
2.1.2 DeepLabV3+
In this study, DeepLabV3+ was utilized for more precise flame segmentation because it can simultaneously capture multi-scale contextual information and preserve boundaries. DeepLabV3+ is an encoder-decoder network that retains the atrous spatial pyramid pooling (ASPP)-based encoder of DeepLabV3 and adds a decoder module for more precise boundary restoration. The encoder extracts contextual information at multiple scales by adjusting the output stride using atrous convolutions while expanding the network’s field of view. In particular, the ASPP module at the top of the encoder captures features at multiple spatial scales by combining parallel atrous convolutions with image-level pooling, each using different dilation rates. This architecture provides high robustness to objects with irregularly changing shapes and regions of varying sizes such as flames. The decoder restores spatial resolution reduced during the encoder step. The encoder’s output feature map is upsampled by a factor of four and then concatenated with low-level features extracted from the backbone to restore the detailed shapes near boundaries. Subsequently, successive 3 × 3 convolutions are applied to refine the concatenated features, which are then upsampled by another factor of four to produce the final segmentation results. In addition, depthwise separable convolutions are applied to the ASPP and decoder modules to reduce computational load while maintaining segmentation accuracy. The overall architecture of DeepLabV3+ is shown in Figure 2 [19].
Architecture of deepLabV3+
2.1.3 YOLOv8-seg
A YOLOv8-seg model was also trained as a lightweight architecture capable of segmenting flame regions in real time. YOLOv8-seg is a fast and efficient single-stage model that integrates anchor-free detection, an improved backbone-neck structure, and a segmentation head based on advanced structural features developed after YOLOv5. The backbone was designed by replacing the previous C3 module with a C2F module to reduce computational volume while maintaining the feature representation. Furthermore, 6 × 6 convolutions were applied instead of 3 × 3 convolutions to reduce the number of parameters to lighten the network. The neck comprises a structure that combines a feature pyramid network and a path aggregation network and is designed to capture the characteristics of object at multiple scales by integrating feature maps of different resolutions. The head improves prediction stability and speed by separately processing object localization and classification using a decoupled head design; the overall architecture of the YOLOv8-seg model is presented in Figure 3 [20,21]. In addition, objects of different sizes and shapes can be effectively detected using anchor-free, center-point-based detection without the need for specific anchors. Segmentation is performed in a mask branch added to the existing YOLO architecture that combines bounding-box-based detection results with feature maps to generate pixel-level object masks.
Architecture of YOLOv8-seg
2.2. Training process and input dataset
In this study, 15,400 images with and without fire from diverse backgrounds were collected to compile a dataset to train the deep-learning models. Images showing fire were largely categorized as outdoor and indoor fires, with a wide variety of objects and background brightness of fire scenes. Images that did not show fire included various backgrounds and objects with colors that resembled those of flames. Figure 4 presents typical images in the dataset used in this study. To train and validate the deep-learning models, 14,400 images were divided into training and validation sets at a ratio of 8:2. As a preprocessing step, flame regions were annotated as polygons in each collected image using Roboflow [22]. Then, the images were then output as 640 x 640 RGB files with text labels to be used to train the models. The performance of the deep-learning models optimized through hyperparameter tuning using the training and validation datasets was evaluated on a set of 1,000 images, which comprised 900 images showing fire and 100 that did not include fire. The training and evaluation process was performed on a system with a 13th Generation Intel® Core™ i9-13900KS CPU with 24 cores at a clock frequency of 3.20 GHz, an NVIDIA GeForce RTX 4090 GPU with 24 GB of VRAM, 128 GB of DDR5 memory, and Windows 11.
Representative images from the image dataset.
2.3 Data analysis
To assess the segmentation performance of the models, the mIoU values were comparatively analyzed among the U-Net, DeepLabV3+, and YOLOv8-seg. The mIoU was calculated as the ratio of the intersection to the union between the predicted segmentation region and the ground truth averaged over all classes as shown in Eq. (1).
where Pi and Gi respectively represent the predicted and actual flame region in the i-th image.
Additionally, a pixel-level confusion matrix was used to analyze the detailed segmentation performance of each model, as shown in Table 1. Accordingly, the ground-truth and predicted masks were compared for all evaluation images, which were binarized as flame (1) and non-flame (0). First, pixels for which both the ground truth and the prediction were labeled as flame are defined as true positive (TP) detections, whereas background pixels predicted as showing flame are defined as false positive (FP) detections. Similarly, background pixels correctly detected as such are defined as true negative (TN) detections, and pixels that did show flame but that the model failed to predict as such are defined as false negatives (FN). Based on the TP, FP, TN, and FN values calculated as described above, we computed precision, recall, and F1-score as shown in Eqs. (2)-(4) to quantitatively assess the models’ performance in detecting fire. The F1-score is defined as the harmonic mean of precision and recall.
Confusion Matrix
3. Research Result and Discussion
3.1 Flame detection performance of deep-learning object segmentation models
The segmentation results of the U-Net, DeepLabV3+, and YOLOv8-seg were compared using evaluation images representing daytime and nighttime outdoor fires and bright and dim indoor fires in the testing dataset; representative examples are depicted in Figure 5. The results of the three models were analyzed in terms of the reproducibility of the shape of the flame, boundary delineation, and misdetection of non-flame regions with reference to the same ground truth.
Flame segmentation results.
Figure 5(a) shows daytime outdoor fires. The U-Net demonstrated the highest accuracy with an IoU of 80.2%. The U-Net detected the outline of the actual flame relatively stably even in areas where the boundary between the building’s outer wall and the flames was unclear. By contrast, DeepLabV3+ and YOLOv8-seg had an IoU of 68.1% and 61.4%, respectively, but they tended to overestimate certain regions due to failing to clearly distinguish the boundary between flames and the background. Figure 5(b) depicts a nighttime outdoor fire scene in which reflections from flames and strong shadows are present simultaneously. Under these conditions, the U-Net and YOLOv8-seg had a low IoU of 52.9% and 44.5% respectively due to incorrectly classifying flame reflections and actual flames. In contrast, DeepLabV3+ achieved an IoU of 82.7% due to stably segmenting the entire flame region, which indicates that the multi-scale feature extraction architecture was able to handle complex lighting changes effectively. Figure 5(c) shows an indoor flame image that is likely to be misdetected due to the background color being similar to that of the flames. The U-Net exhibited strong performance with an IoU of 86.5% but tended to misclassify parts of the surrounding area as flames. DeepLabV3+ demonstrated the best performance with an IoU of 92.4% and was able to detect the actual appearance of the flame with the highest accuracy. YOLOv8-seg also exhibited satisfactory performance with an IoU of 87.1%, but it tended to predict the outer boundary of flames as being larger than the actual size. Overall, all three models achieved stable detection performance on images with simple indoor backgrounds. Figure 5(d) displays a strong flame with high brightness occurring in a dim indoor environment. Although the U-Net achieved an IoU of 80.1% and stably detected flame centers, false detections occurred in which certain parts of the flame reflections were misclassified as flames. DeepLabV3+ and YOLOv8-seg recorded a relatively low IoU of 67.1% and 71.9%, respectively, which can be attributed to flame boundaries being irregularly predicted in regions with high brightness. The overall analysis showed that DeepLabV3+ exhibited the most robust segmentation performance, even under nighttime or complex background conditions, whereas the U-Net produced stable results primarily in relatively simple backgrounds. Although the lightweight architecture of the YOLOv8-seg model is advantageous for real-time processing, its performance degrades in complex or high-brightness environments due to overprediction.
To quantitatively compare the flame segmentation performance of the deep-learning models shown in Figure 5, the mIoU values for all 1,000 images in the test dataset are presented in Figure 6, representing the average segmentation performance of each model. Comparative analysis showed that the mIoU values were 70.5%, 72.9%, and 63.1% for the U-Net, DeepLabV3+, and YOLOv8-seg, respectively, with DeepLabV3+ exhibiting the best flame segmentation performance.
Comparison of mIoU results for flame segmentation models.
3.2 Performance evaluation
A testing dataset composed of 1,000 images of fire and non-fire scenes was used to comparatively evaluate the performance of the U-Net, DeepLabV3+, and YOLOv8-seg models on the fire detection task; the results are presented in Figure 7. The testing dataset consisted of 900 images showing fire and 100 images without fire, with the fire images evenly distributed among indoor and outdoor and daytime and nighttime fires. The U-Net achieved a precision of 72.9%, DeepLabV3+ reached 75.6%, and YOLOv8-seg only reached 63.6%, with all three models exhibiting relatively low precision. The U-Net, DeepLabV3+, and YOLOv8-seg achieved recalls of 95.9%, 95.8%, and 98.8%, respectively, indicating that all three models exhibited a tendency to over-predict flame regions to minimize the likelihood of omission. Owing to this tendency, certain parts of the background were regularly misclassified as flames, which lowered the precision. Notably, this phenomenon was more pronounced in images with low color contrast between the flame and the surrounding background, as well as in the presence of reflections, variations in lighting, and objects with similar colors. The F1-score, which combines precision and recall, was 81.9%, 83.6%, and 76.4% for the U-Net, DeepLabV3+, and YOLOv8-seg, respectively. The overall performance was the highest for DeepLabV3+, followed by the U-Net and YOLOv8-seg; this is consistent with the analysis of flame detection performance. Notably, all three models tended to overestimate flame regions under conditions of low visual contrast between the flame and the background, which was a major factor that contributed to increased recall and reduced precision.
Comparison of fire prediction performance of the flame segmentation models.
To evaluate the performance, we focused on accuracy, real-time processing performance, computational complexity, and computational efficiency. First, the inference speeds of the models were compared, as shown in Figure 8. The real-time processing performances of the U-Net, DeepLabV3+, and YOLOv8-seg models were evaluated based on these values. The inference speeds of all models were measured under the same input resolution (640 × 640) and batch size. The number of parameters and the size of the models are presented in Table 2, and the computational complexity of each model was analyzed along with the degree to which the architecture comprised a lightweight design. The inference speed of YOLOv8-seg was the highest (145.2 fps). Because of its lightweight network architecture and optimized decoder design, the model is well-suited for real-time fire detection systems that require rapid decision-making from continuous images, particularly during the early stages of a fire when flame spreads rapidly. The U-Net achieved a relatively faster processing efficiency with an inference speed of 102.5 fps, whereas DeepLabV3+ exhibited the lowest speed (53.0 fps). DeepLabV3+ displayed strong segmentation performance; however, its inference speed was reduced owing to the high computational load associated with its multi-scale feature extraction structure. A comparison of the number of parameters and model size revealed significant differences in model capacity. YOLOv8-seg has the lightest architecture among the three models, with approximately 3,260,000 parameters and a size of 6.5 MB. The model’s lightweight design makes it suitable for deployment in resource-constrained environments such as edge devices and drones. By contrast, DeepLabV3+ boasts nearly 60.99 million parameters and a model size of 233.3 MB and, thus, typically has limited real-time processing speed unless executed in a high-performance GPU-based environment. The U-Net has approximately 31.04 million parameters and a size of 118.5 MB, exhibiting an intermediate level of complexity among the three models. Overall, DeepLabV3+ performs the best in terms of segmentation precision, whereas YOLOv8-seg is considered the most advantageous for practical applications because of its real-time processing speed and compact size. The U-Net achieves a trade-off between the two elements and, thus, is be highly useful in a general environment requiring high performance and computational efficiency. Furthermore, the flame size and region predictions in this study are significant because they can be further developed for precise fire detection technology beyond merely judging the presence of fire. Information on the spatial distribution of flames can be utilized to estimate the initial size of the fire and analyze its growth and propagation over time. The present work also contributes to applied research in various areas, including the analysis of the behavior of fire and the establishment of fire suppression strategies.
Comparison of inference time of the flame segmentation models.
Comparison of Deep-Learning Models in Terms of the Number of Parameters and Model Size
4. Conclusion
In this study, we comprehensively examined the performance of deep-learning-based models trained to detect fire in various relevant situations; the models were the U-Net, DeepLabV3+, and YOLOv8-seg. Fire detection performance was evaluated using a testing dataset representing various environments that included daytime and nighttime outdoor fires as well as bright and dim indoor fires. Additionally, we analyzed the inference speed and computational complexity of the models. Our key conclusions are summarized as follows.
- In terms of flame detection performance, DeepLabV3+ achieved the best results with a mIoU of 72.9% and an F1-score of 83.6%, whereas the U-Net and YOLOv8-seg achieved mIoUs of 70.5% and 63.1%, respectively. All three models demonstrated a high level of recall but low precision, which can be attributed to overpredicting flames to avoid missing actual flames, which included some pixels that were part of the background and led to false detections.
- YOLOv8-seg was the most suitable for real-time fire detection owing to its lightweight architecture, with a high inference speed of 145.2 fps and 3.26 million parameters. By contrast, DeepLabV3+ was limited in real-time applications despite its high accuracy owing to its high computational load, whereas the U-Net exhibited a balanced level of accuracy and size.
In future research, the accuracy and realism of fire detection methods should be further enhanced by designing lightweight models to learn features to improve the models’ robustness to smoke and variations in lighting. Multi-sensor convergence techniques should also be developed.
Notes
Author Contributions
Conceptualization and methodology, J.R. and M.K.; formal analysis and investigation, J.R.; writing—original draft preparation, J.R.; writing—review and editing, M.K. and S.S.; supervision, M.K.; project administration, M.K.; All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
Acknowledgments
This work was supported by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant, funded by the Ministry of Land, Infrastructure, and Transport (Grant RS-2022-00156237).