Deep Convolutional Neural Network for Plume Rise Measurements in Industrial Environments

1. Introduction

Smokestack Plume Cloud (PC) rises due to momentum and buoyancy. Finally, the PC dissipates and is carried downwind at a constant height. This height is called plume rise height or Plume Rise (PR). PR calculation is not straightforward, and it is a substantial problem in predicting the dispersion of different harmful effluents into the air [1]. PR contributes to 1) the distance pollutants carried downwind, 2) their concentration at the surface, where they are deposited in the green environment or inhaled by people, and 3) the amounts of greenhouse gases mixed into the upper troposphere. Therefore, accurate measurement of the PR is of concern for research and operational applications such as air-quality transport models, local environment assessment cases and global climate models [2].

The parameterizations of PR prediction were developed in the 1960s by Briggs [3,4]. Dimensional analysis was used to estimate the PR based on smokestack parameters and meteorological measurements in different atmospheric conditions. Early observations of PR were used to test and rectify the parameterizations developed using dimensional analysis [5]. Wind tunnel studies and field observations using technologies including film photography, theodolites, and cloud-height searchlights [6] were several calibration techniques utilized in this domain. There are also three-dimensional air-quality models using parameterization equations, including GEM-MACH [7], CAMx [8], and CMAQ [9].

Some studies tested the parameterizations of PR prediction in the 1970s and 1980s by comparing them to actual observations and demonstrated that the Briggs equations overestimate the PR [10,11,12,13]. In 1993, an aircraft-based measurement was done to measure ${\mathrm{SO}}_2$ emissions of a power plant which indicated an overestimation of about 400 m [14]. Although these earlier studies showed some degree of overestimation, in 2002, Webster et al. [15] performed surface measurements and concluded that the Briggs parameterizations tend to underestimate PR. In 2013, as part of the Canada-Alberta Joint Oil Sands Monitoring (JOSM) Plan, an aerial measurement study was done in northern Alberta’s Athabasca oil sands region to study the dispersion and chemical processing of emitted pollutants [16,17,18]. The project consisted of 84 flight hours of an instrumented Convair aircraft over 21 flights designed to measure pollutants emissions, study the transformation of chemicals downwind of the industry, and verify satellite measurements of pollutants and greenhouse gases in the region. Using aircraft-based measurements and reported smokestack parameters and meteorological data, it was demonstrated that the Briggs equations significantly underestimate PR at this location.

Given the results of [16,17,18] and the gap of more than 30 years since the Briggs equations were developed and previously tested, there is a need for further testing and possible modification of the Briggs equations based on modern observation techniques. In recent decades, there have been many significant advancements in environmental monitoring activities over industrial regions for safety and pollution prevention [19,20,21,22]. Moreover, several smoke border detection and recognition models have been introduced recently using digital image analysis, such as wavelet and support vector machines [23], LBP and LBPV pyramids [24], multi-scale partitions with AdaBoost [25], and high-order local ternary patterns [26] which are well-performed and impressive. These improvements have led to the development of instrumentation which can be deployed near any smokestack to give information on pollutant dispersion and potential exposure to people downwind. This information will be based on actual real-time observation, i.e. digital images, as opposed to potentially erroneous and decades-old parameterizations. Due to the similarity of our work to smoke recognition on the one hand, and on the other, the unavailability of PC recognition research, smoke recognition studies will be reviewed in the following.

To find the smoke within an image or a video frame, either a rough location of smoke is identified using bounding boxes called smoke border detection [27], or pixels are identified and classified in detail, named smoke recognition [29]. Due to the translucent edges of smoke clouds, the recognition task needs far more accuracy than border detection. Traditional smoke recognition methods utilize manual features, which lead to low accuracy recognition results due to a large variety of smoke appearances. These low-level features consist of motion characteristic analysis of the smoke [30], smoke colour [31], and smoke shape [32]. In another research, [33] took advantage of the Gaussian Mixture Model (GMM) to detect the motion region of the smoke and [34] combined rough set and region growing methods as a smoke recognition algorithm which seems to be a time-consuming algorithm due to the computational burden of the region growing process. Since using colour information is less effective due to the similarity of smoke colour to its surrounding environment, the combination of motion and colour characteristics is considered for smoke recognition [35]. Some algorithms utilize infrared images and video frames in their experiments [36], which are not easily accessible and can increase the project’s costs. Moreover, using digital images makes the algorithm more flexible as it can be used with more hardware. On the other hand, some smokes are too close to the background temperature to be captured by the near red-channel wavelength. A higher-order dynamical system introduced in 2017 used particle swarm optimization for smoke pattern analysis [37]. However, this approach had a low border detection rate and high computational complexity.

In recent years, deep learning-based methods, especially Convolutional Neural Network (CNN) based methods, have led to significant results in semantic segmentation [38] and object recognition [39]. Similarly, these methods are widely used in smoke border detection and recognition [40] with different architectures such as three-layer CNN [41], generative adversarial network (GAN) [42], and two-path Fully Convolutional Network (FCN) [28]. Recently, a count prior embedding method was proposed for smoke recognition to extract information about the counts of different pixels (smoke and non-smoke) [43]. Experimental results showed an improvement in the recognition performance of these studies. However, the high computational complexities of these huge models are an obstacle to their use in PR real-time observations.

We have proposed a novel framework using Deep Convolutional Neural Network (DCNN) algorithms to measure PR. Our approach comprises three stages: 1) recognizing the PC region using an improved Mask R-CNN, 2) extracting the PC’s Neutral Buoyancy Point (NBP) from the centerline of the recognized PC, and 3) transforming the PC’s geometric measurement from an image-scale to real-world scale.

This strategy accurately recognizes the PC and measures PR in real-time. Here, we reinforce the bounding box loss function in Region Proposal Network (RPN) [46,47] through engaging a new regularization to the loss function. This regularizer restricts the search domain of RPN to the smokestack exit. In other words, it minimizes the distance between the proposed bounding boxes and the desired smokestack exit, called smokestack exit loss ($L_S$). The proposed method is also computationally economical because it generates only a limited number of anchor boxes swarmed across the desired smokestack exit. Consequently, the main contributions of this paper can be summarized as follows:

• Proposing Deep Plume Rise Network (DPRNet), a deep learning method for PR measurements, by incorporating PC recognition and image processing-based measurements. We have provided a reproducible algorithm to recognize PCs from RGB images accurately.
• To the best of our knowledge, this paper estimates the PCs’ neutral buoyancy coordinates for the first time, which is of the essence in environmental studies. This online information can help update related criteria, such as the live air-quality health index (AQHI).
• A pixel-level recognition dataset, Deep Plume Rise Dataset (DPRD), containing: 1) 2500 fine segments of PCs, 2) The upper and lower boundaries of PCs, 3) The image coordinates of smokestack exit, 4) The centerlines and NBP image coordinates of PCs, is presented. As is expected, the DPRD dataset includes one class, namely PC. Widely-used DCNN-based smoke recognition methods are employed to evaluate our dataset. Furthermore, this newly generated dataset was used for PR measurements.

This paper is organized as follows—Section 2 briefly explains the theoretical information used in our proposed framework. Section 3 describes our proposed framework for measuring the PR of a desired smokestack. Then, Section 4 present our dataset collection procedure, under-study site, experimental results of the proposed method and evaluation results using different metrics, and PR and PR distance calculations. Finally, this research’s conclusions, findings, and future studies are described in Section 5.

2. Theoretical background

2.1. Briggs PR prediction

The PR calculation is an ill-posed problem to predict the dispersion of harmful effluents in atmospheric science [1]. PR is affected by two phenomena, buoyancy and momentum. Typically, the PCs are buoyant, which means they are hotter than the ambient air. Therefore, they rise since they are less dense than the surrounding air. Also, the PCs have a vertical velocity and momentum when they exit the smokestack, which causes them to rise again. PCs can also fall due to the gravitational force when cold and dense and when some surrounding obstacles cause them to move downwind [48]. In 1975 Briggs proposed an equation for the maximum distance of the PR, which was practically suitable for calculating PR. Considering both momentum and buoyancy in calculations, PR ($\Delta z$) in horizontal distance x from the smokestack exit can be obtained as [4],

$$\Delta z=(\frac{3F_{m}x}{0.6^2{\overline{u}}^2}+\frac{3F_b{x}^2}{2\times {0.6}^2{\overline{u}}^3}{)}^{1/3}$$

(1)

where $0.6$ is an entrainment rate (the mean rate of increase of PC in the wind direction) and $\overline{u}$ is the mean horizontal wind speed. Also, the momentum flux parameter ($F_m$) and buoyancy flux parameter ($F_b$) are defined below,

$$F_m=\left[\frac{\overline{{\rho }_s}}{\overline{\rho }}\right]r_s^2{\overline{w_s}}^2$$

(2)

$$F_b=[1-\frac{\overline{{\rho }_s}}{\overline{\rho }}]g{r}_s^2{\overline{w_s}}^2$$

(3)

where $\overline{{\rho }_s}$ is the smokestack gas density, $\overline{\rho }$ is the atmospheric air density, $r_s$ is the smokestack radius, $\overline{w_s}$ is the vertical velocity of the smokestack gas, and g is the acceleration due to gravity.

It should be noted that if we evaluate the wind speed at the local PC height and not the source height, the calculations should be operated iteratively [49,50]. The wind strongly affects PC buoyancy, horizontal momentum movements, and PR [1]. Moreover, PR is unaffected by wind speed fluctuations in stable conditions with low turbulence, making the measurements difficult. However, significant PR variations in unstable conditions have been witnessed at a fixed distance downwind.

2.2. CNN and convolutional layer

CNNs are particular types of neural networks suitable for processing grid data, such as image data with a two-dimensional or three-dimensional mesh structure of pixels. The name given to a CNN is derived from the convolutional layers used in it. Each convolutional layer contains several kernels and biases that are locally applied to the input and produces a feature map or an activation map according to the number of filters. Suppose this convolutional layer is applied to the input image in a two-dimensional manner. In that case, its $j^{th}$ feature map $O_j$, which is obtained by using the $j^{th}$ kernel, is calculated at the position $(x,y)$ as [51],

$$O_j^{xy}=B_j+\sum _k\sum _{m=0}^{M-1}\sum _{n=0}^{N-1}{w}_j^{mn}{z}_k^{(x+m)(y+n)}$$

(4)

where k moves along the depth dimension of input $z\in {\mathbb{R}}^{p\times q\times r}$ and $w_j^{mn}$ is the two-dimensional kernel weight $W\in {\mathbb{R}}^{M\times N}$ at position $(m,n)$. $B_j$ is the bias matrix.

2.3. Mask R-CNN

Mask R-CNN is a region-based CNN family member proposed in [44] and is used widely in different identification tasks, including COCO dataset recognition. Firstly, RCNN was presented in [52], where computer vision techniques generated region proposals. Then in [47], Fast RCNN was introduced with a CNN before region proposals to reduce running time. In 2017, Faster RCNN continued this evolution and offered the RPN to propose regions of interest (ROIs) [53]. Finally, Mask R-CNN, as an extension of Faster RCNN, added a CNN for pixel-level recognition of the border-detected objects. Mask R-CNN is a relatively simple model and easy to generalize to other similar tasks [44]. Therefore, Mask R-CNN can create pixel-level masks for the objects besides object localization and classification tasks.

2.3.1. RPN

An RPN is a deep FCN that proposes regions and is crucial to the Mask R-CNN. RPN helps selectively focus on valuable aspects within the input images. This network takes an image and gives a set of region proposals beside their scores for being an object. It slides a window over the convolutional feature map (backbone output) and maps it to a lower-dimensional feature. The generated feature is then fed into two fully-connected layers to obtain the proposed regions class (object vs. non-object) and four corresponding coordinates [53]. A maximum of k possible proposals are parameterized relative to k reference boxes or anchors for each sliding window location. RPN is trained end-to-end by back-propagation and stochastic gradient descent. Figure 1 depicts a scheme of RPN in which the proposed regions are generated as the module outputs.

2.3.2. Loss function

Mask R-CNN loss function is a weighted summation of other losses related to different sections of this comprehensive model. As a definition based on [44], a multi-task loss function is proposed on each sampled ROI as,

$$L=L_{cls}+L_{reg}+L_{msk}$$

(5)

where $L_{cls}$ recognizes the class type of each object while $L_{reg}$ attempts to find the optimum anchor box for each object. Note that, in this study, we have one class, PC. $L_{msk}$ tries to recognize the optimum object’s segment in each bounding box.

3. Methodology

The proposed framework for PR measurement is represented in Figure 2. The images containing PC(s) are fed to DPRNet for PC border detection and recognition. PR and PR distance are then measured on DPRNet’s output based on extracting the NBP. The measured image coordinates of NBP are combined with the wind direction information to be processed by geometric transformation calculations. The main output of the system will reveal the PR as a physical height and the PR distance downwind at which the PR occurs. Also, a schematic diagram in Figure 6 describes the definitions of PC centerline, PR and PR distance.

3.1. DPRNet

This research aims to precisely recognize the PC of the desired smokestack from a wide range of image datasets captured from the study area. DPRNet is an adapted Mask R-CNN version with two novel smokestack PR measurement modules. These modules are 1) the physical module and 2) the loss regularizer module. These modules can improve RPN performance in locating the most probable proposal PCs. Mask R-CNN is the base of our proposed method, one of the widespread border detection and recognition methods. This robust framework can consider the irregular shapes of the PC, its translucent edges, and similar pixel values of the PC to its background [44]. As seen in Figure 3, DPRNet is an application-oriented version of Mask R-CNN to which two new modules have been added.

In this architecture, ResNet50 [54] is used as the backbone network to extract feature maps of the input images. Feature Pyramid Network (FPN) uses all these feature maps to generate multi-scale feature maps, which carry more helpful information than the regular feature pyramid. Then, RPN detects the PC by sliding a window over these feature maps to predict whether there is a PC and locate the existing PC by creating bounding boxes. Therefore, we have a set of PC proposals from RPN and the generated feature map by the backbone network. The ROI Align module works to scale the proposals to the feature map level and prevent misalignment by standardizing the aspect ratios of the proposals. Finally, these refined feature maps are sent to three different outputs. The first is a classification block that decides whether the ROI corresponds to the foreground (PC). The second one is a regression block which predicts the bounding boxes based on the provided ground truth. And the last block indicates a recognition mask for the detected PC using an FCN [55].

Two modules are added to Mask R-CNN to improve its efficiency and reduce the computational burden. The first module, a simple image processing one, approximates smokestack. The second module attempts to improve the loss related to $L_{reg}$ by adding a regularizer loss, elaborated in Section 3.1.2. These modules are explained in detail in the following subsections.

3.1.1. Physical module

The physical module was implemented to detect an exit point where the PC of interest rises from a chimney stack in the middle of the imagery. Please note that this module is only used during the training phase, and the exit point is automatically predicted in the network’s inference process without the physical module. In this physical module, the smokestack exit is detected from ground truth binary image by an image processing technique detecting extrema points of foreground boundaries [57]. It stands to reason that the smokestack exit is the feasible region of the plume rise. As a result, proposed regions can be considered around this point (Figure 3). Thanks to this module, the method does not detect small PC pieces, sometimes seen in different parts of images other than the smokestack exit.

To extract the smokestack exit, eight extrema points on each PC’s boundary are extracted based on image processing analysis [57], illustrated in Figure 4. This algorithm first smoothes noises of foreground regions (i.e., PC region labelled in the ground truth imagery). After that, it tracks boundary pixels of the foreground region and extracts eight extrema points showing the locally highest geometric curvature. Each PC segment’s bottom-left corner is considered a smokestack exit.

3.1.2. Loss regularizer module

Based on Section 2.3.2, it is a crucial problem to set an efficient loss function, which can make the model stable as it can get. In this regard, a new regularizer is added to the loss function, which regresses the coordinates of the most attainable PC regions. Indeed, we try to minimize the distance of proposed bounding boxes by RPN and the smokestack exit. Suppose a box with coordinates of ($x,y,w,h$) is defined here. In that case, the regression loss related to the smokestack exit can be defined as,

$$L_S=R(u-u^{*}),$$

(6)

in which,

$$\begin{array}{c}\hfill u_x=\frac{x-x_a}{w_a},u_y=\frac{y-y_a}{h_a},u_w=log\left(\frac{w}{w_a}\right),u_h=log\left(\frac{h}{h_a}\right),\\ \hfill u_x^{*}=\frac{x^{*}-x_a}{w_a},u_y^{*}=\frac{y^{*}-y_a}{h_a},u_w^{*}=log\left(\frac{w^{*}}{w_a}\right),u_h^{*}=log\left(\frac{h^{*}}{h_a}\right)\end{array}$$

(7)

where u and $u^{*}$ represent the coordinates of our predicted and ground truth smokestack exit and R is the robust loss function. Note that subscript a and superscript * variables represent the anchor coordinate and the ground truth coordinates, respectively, while the rest are defined as the predicted coordinates. The point $(x,y)$ indicates the position of the bounding box’s top-left corner and the parameters w and h are, respectively, the width and height of the bounding box.

Unlike the Mask R-CNN model, in DPRNet, the loss regularizer module ($L_S$) minimizes the recognition task errors of a specific PC, which copes with the main problems of this model, such as missing the desired smokestack exit and multi-box proposal for a single PC. $L_S$ helps us avoid spanning the whole image pixels, causing high training time and computational complexities.

3.2. NBP extraction

As seen in Figure 2, the PR measurements are performed based on the extraction of NBP for every PC image. The NBP extraction phase can be summarized in three steps. An overall processing chain is shown in Figure 5. Firstly, the centerline is extracted as the PC’s skeletonized curve., which consists of some points demonstrating the meandering of the PC. To accomplish this task, we classify the PC’s boundary pixels into upper and lower boundary pixels, represented with cyan and yellow coloured lines, respectively, in Figure 5. This binary classification of boundary lines is conducted by tracking the upper pixel, found at the highest row index, while the lowest one for the lower boundary pixel from each fixed column index with the coordinate origin at the lower-left corner in the image space. The center line pixel per each column is found by simply averaging the two-row indices. Measuring the NBP for plume rise in the context of atmospheric science using imagery requires identifying the point at which the buoyant force from the plume is equal to the gravitational force acting on it. At this point, the plume ceases to rise further and begins to spread laterally. The central lines detected from the imagery show salient visual cues caused by many factors involved in atmospheric dynamics. However, they cannot explicitly estimate the NBP and require a physical interpretation to determine the NBP from the centerlines. Thus, we fit an asymptotic function [59] to the center line pixels using a conventional least-squared method to drive this function’s horizontal asymptote to determine the NBP from the imagery. Equation 8 shows the asymptotic function employed in this study.

$$y=-a{e}^{-bx};a,b>0$$

(8)

3.3. Geometric transformation

To convert the image measurement results to real-world ones, we need to perform some calculations to transform the PR ($\Delta z$) and the PR distance ($X_{max}$) on an image to real-life measurements using wind direction.

The PR distance can be defined as the horizontal distance between the smokestack exit and NBP, discussed in Section 3. Figure 6 shows PR and PR distance definitions on a sample PC image.

Figure 6. PR, PR distance, and $NBP$ on a sample image. $\theta$ represents the PC deviation due to the wind, and $\phi$ denotes the wind direction. Also, S indicates the smokestack exit, and the blue line shows the PC centerline.

Figure 6. PR, PR distance, and $NBP$ on a sample image. $\theta$ represents the PC deviation due to the wind, and $\phi$ denotes the wind direction. Also, S indicates the smokestack exit, and the blue line shows the PC centerline.

As observed in Figure 7 and Figure 8, the PC is affected by wind direction and is out of the image plane. Wind direction is always reported as degrees from the north, represented by $\phi$. For instance, $\phi ={90}^{\circ }$ is wind from the east, and $\phi ={180}^{\circ }$ is wind from the south. Based on the camera’s position, capturing desired smokestack images, and geographical directions, wind direction relative to the image plane can be obtained as $\theta =|\phi -252|$ in this study (This information is prior knowledge). Accordingly, $X_{NBP}$, shown in Figure 7, can be calculated by the following equation,

$$X_{NBP}=\begin{array}{cc}\hfill \frac{D}{tan\theta +\frac{1}{tan\gamma }}& \mathrm{if}\theta \ge 0\\ \hfill \frac{D}{\frac{1}{tan\gamma }-tan\theta }& \mathrm{otherwise}\end{array}$$

(9)

where $\gamma$ and $\theta$ are additional parameters that help us define equations as concisely as we can get. D indicates the distance between the camera and the smokestack.

We define G and $G_{NBP}$ as the value of the ground sample distance, respectively, at the image plane and the location of NBP. Consequently, based on [45],

$$\begin{array}{cc}\hfill & G_{NBP}=\frac{X_{NBP}}{x_{NBP}}\hfill \\ \hfill & Z_{NBP}=G_{NBP}\times z_{NBP}\hfill \\ \hfill & Z_S=G\times z_S\hfill \end{array}$$

(10)

where $Z_S$ and $z_S$ are the distance between the smokestack exit and the image center, respectively, in real life and on the image.

Thus, the PR and the PR distance for each PC can be calculated as,

$$\Delta z=\begin{array}{cc}\hfill |Z_{NBP}|-|Z_S|,& \mathrm{if}|Z_{NBP}|\ge |Z_S|\\ \hfill |Z_S|-|Z_{NBP}|,& \mathrm{otherwise}\end{array}$$

(11)

$$X_{max}=\sqrt{X_{NBP}^2+Y_{NBP}^2}$$

(12)

4. Experimental results and discussion

In this section, we describe our image datasets and the industrial area in which these image datasets have been collected and shared. Also, we will explain the validation metrics used to compare our proposed method with the other competitive methods in smoke border detection and recognition. Then, our discussion falls into two last sections, named comparison with existing smoke recognition methods and plume rise measurement, in which the performance of the proposed method is evaluated, and the PR is calculated based on our "DPRNet," respectively. To validate the performance of our proposed method, we used a computer equipped with Core i9, 3.70 GHz/4.90 GHz, 20 MB cache CPU, 64GB RAM and NVIDIA GeForce RTX 3080,10 GB graphic card. The total training time of the network was about one hour using Python 3.8 with PyTorch Deep Learning framework. Finally, for the geometric transformation and image processing analysis, we used MATLAB R2022b software.

4.1. Site description

The imaging system was deployed on a meteorological tower with a clear sightline to the desired smokestack operated by the Wood Buffalo Environment Association (WBEA). It is located outside the Syncrude oil sands processing facility north of Fort McMurray, Alberta, Canada. Figure 9 represents the satellite images, the location of the camera, and the desired smokestack.

WBEA operates a 10-meter-tall meteorological tower with a clear sightline to the smokestack at Syncrude (https://wbea.org/stations/buffalo-viewpoint). The camera system is mounted on this tower above the tree canopy because they are on a hill sloped downward from the tower location, and the biggest smokestack and its PC are always visible. The system consists of a digital camera with shutter control and a camera housing for weather protection with interior heating for window defrost and de-icing.

The Syncrude processing facility has six main smokestacks. The tallest one is about 183 m, and the heights of the other five are between 31 m to 76 m. To isolate a single smoke plume rise, we have concentrated on the area’s tallest one, which can help find the PR for one plume source. All six smokestacks are listed in Table 1. Wind directions during the capturing period were determined from the Mildred Lake Air Monitoring Station (https://wbea.org/stations/mildred-lake), located at the Mildred Lake airstrip (AMS02: Latitude: $57.{05}^{\circ }$, Longitude: $-111.{56}^{\circ }$), approximately 5 km from the Syncrude facility.

4.2. Deep Plume Rise Dataset (DPRD)

The greatest challenge in using deep learning for PC recognition is inadequate annotated images for training. Hence, creating image datasets for PC recognition for research and industry purposes is invaluable. For this study, 96 images were captured daily; for the first part of the project, 35K images were collected from January 2019 to December 2019. The collected images demonstrated various types of plume shapes in different atmospheric conditions. The dataset has been classified into day, night, and cloudy/foggy conditions. The collected dataset revealed that among 96 images captured daily, we have 48-day and 48-night images. There were some outlier images for different reasons, such as camera handle shaking, auto-focus problems, disturbing smoke and severe snow and hail. Furthermore, some PCs could not be recognized from their background, even by visual image inspection. As a consequence, among 35K collected images, 10684 were valid. Note that about 8110 images were captured when the facility was not working.

This paper introduces a new benchmark, DPRD, including a 2500 annotated dataset. DPRD contains PC upper and lower borders, smokestack exit image coordinates, PC centerline, and NBP image coordinates. 60% of DPRD is considered training data and 40% is used for validation and testing. Rows (a) and (b) in Figure 10 show sample images from the region and their corresponding ground truth, which are generated by the "Labelme" graphical image annotation tool at https://github.com/wkentaro/labelme. We tried to select images of different atmospheric conditions, such as clear daytime, nighttime, cloudy, and foggy, to represent the results of different situations.

4.3. Model validation metrics

The performance of the methods in question is evaluated using the metrics of accuracy, recall, precision and F1 score. These metrics are defined using four values of True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) obtained from the confusion matrix of each introduced method [58]. The accuracy validation metric is the ratio of observations predicted correctly to the total observations. In our application, the model’s accuracy represents how accurately our model can recognize the PC pixels. This criterion is valid as long as the values of FP and FN are almost the same [58]. Otherwise, other validation metrics should be considered. The foreground pixel coverage of the sample images is revealed in Figure 10, confirming that accuracy is not suitable for this study. Recall or sensitivity is the ratio of positive observations predicted correctly to all actual observations. Recall shows how many PC pixels are labelled among all the actual PC pixels. The recall is obtained as follows,

$$Recall=\frac{TP}{TP+FN}$$

(13)

Precision is the ratio of positive observations which are predicted correctly to all observations which are predicted as positive. This metric represents how many PC pixels exist among all the pixels labelled as PC. Therefore, a low rate of FP can achieve high precision. This validation metric is obtained as follows,

$$Precision=\frac{TP}{TP+FP}$$

(14)

As it is implied from Equations 13 and 14, precision and recall take either FP or FN into account. The last validation measure in this paper, the F1 score, considers both FP and FN as a weighted average of recall and precision metrics. Unlike accuracy, this metric is more useful when FP and FN are not the same as in our study. Our FP is less than FN, or the amount of non-actual PC pixels predicted as PC pixels is less than that of actual PC pixels predicted as non-PC pixels. Therefore, the F1 score helps us look at both recall and precision validation metrics as follows,

$$F1score=\frac{2\times Recall\times Precision}{Recall+Precision}$$

(15)

4.4. Comparison with existing smoke recognition methods

In this section, we evaluate the performance of DPRNet and compare it with several competitors. To choose suitable smoke recognition methods for comparison, we considered both the identification accuracy and computational complexity of the reviewed approaches, which led to the selection of DeepLabv3+ [56], FCN [55], and regular Mask R-CNN. Our proposed DPRNet is evaluated using three metrics introduced in Section 4.3.

As is clear from Table 2, DPRNet performs much better than competitive methods for 90 test images selected from various day, night, foggy, and cloudy conditions. In detail, the recall and precision metrics express the reasonable difference between the models, which shows the effectiveness of the proposed model in recognizing the actual PC pixels. Compared to the rivals, the more considerable value of the F1 score guarantees that DPRNet outperforms the other three methods and shows the efficacy of this method. Among our competitive methods, DeepLabv3 performed better regarding all validation metrics, and Mask R-CNN had the worst performance.

Besides these average values, the detailed statistics for each model are given in Figure 11 in terms of each used validation metric. At a glance, our proposed method shows the most robustness in all circumstances. Of competitors, Mask R-CNN and FCN have the worst performance, whereas, DeepLabv3 has the best efficiency slightly.

To further validate our DPRNet performance, we compared the models over the day, night and foggy & cloudy data sets in terms of different validation metrics, which are given in Figure 12. It can be observed that all methods, except Mask R-CNN, have acceptable performance using day and night datasets. Even with night precision, FCN is better than our proposed method. However, as discussed in Section 4.3, this metric can only partially convey the merit of a model individually, and it needs to be analyzed with the F1 score. Our proposed DPRNet outperforms the other rival methods by recognizing roughly all of the PC pixels correctly. Most datasets are related to cloudy and foggy conditions and are frequently seen within image batches. The strength of our DPRNet is its robust performance in this case, which is of paramount importance in our application. The DPRNet could improve the recall metric by 66%, 58%, and 87% on average in cloudy and foggy conditions relative to FCN, DeepLabv3, and Mask R-CNN frameworks, respectively, which means that the proposed method is able to find the PC regions appropriately, using $L_S$. This capability produces high-quality image recognition with a more complicated mixture of PCs and the sky behind. These high recall values help us meet our research application requirement, in which we should identify the entire PC stream for PR distance measurement.

To demonstrate the qualitative results of the proposed method, we show some visual results to compare competitive methods. Figure 13 depicts these recognition results. The first two rows represent the input images and their corresponding ground truths, respectively, and the other rows give the output of different models. We tried to visualize samples from all classes such that the first two images are related to cloudy/foggy conditions, the second two are from the nighttime dataset, and the last two are obtained from our daytime dataset. It is observed that DPRNet outperformed the other methods by attaining high accuracy of PC localization and, consequently, correctly recognizing the desired smokestack PC.

4.5. Plume rise measurement

As discussed in Section 3, DPRNet gives PC border detection and recognition. Then, we take advantage of the NBP image coordinates and the wind direction information from the meteorological tower to obtain PR real-life measurements through geometric transformations. Figure 14 illustrates the asymptotic curve for four PC images and the automatically chosen point NBP, where the PC reaches neutral buoyancy. Apart from the PR and PR distance values of each sample PC, estimated by the proposed framework (tabulated in Table 3), the averaged hourly measured wind directions at the image sampling times are given as prior information of this study. These realistic PR and PR distance values within an extended period are required for future studies.

5. Conclusion

To measure the PR through remote sensing images, PC recognition is of the essence as the first step. In this regard, a novel deep learning-based method, inspired by the nature of the problem, is proposed in this paper to detect and recognize the PC accurately. In the next stage, image processing analysis is leveraged to extract the PC centerline. Afterward, the critical point of this curve is estimated, the y-component coordinate of which is equivalent to PR. Lastly, this image measurement is transformed into a real-life world under the geometric transformation stage. Experimental results indicate that the proposed method, DPRNet, significantly outperformed its rivals. This work also demonstrated that PR could be determined with a single-camera system and wind direction measurements, allowing further investigation and understanding of the physics of how buoyant PCs interact with their environment under different meteorological conditions. The proposed strategy can be extended to a more comprehensive method through several advancements in future research:

• Generalizing DPRNet to predict the PC and PC centerline simultaneously.
• Reinforcing DPRNet to recognize multi-source PCs occurring in industrial environments.
• Conducting comparative studies using meteorological and smokestack measurements between the estimated PR and PR distance from the proposed framework and the Briggs parameterizations equations.
• Briggs parameterization modification via estimated PR and PR distance from the proposed framework.