Multi-Object Tracking with Confidence-Based Trajectory Prediction Scheme

1. Introduction

MOT has been highly spotlighted recently, which deals with the detection of multiple objects across video frames and identifier assignment for each trajectory. It plays a vital role in intelligent video surveillance, military surveillance and autonomous driving etc. Over the past few years, MOT has benefited greatly from the rapid development of advanced object detection schemes. Especially, the tracking-by-detection (TBD) paradigm has achieved tremendous success in MOT. Most existing works divided MOT into detection and association tasks and solve them independently under TBD framework [1,2]. Typically, they firstly apply object detectors to detect known targets and infer the trajectories via different matching strategies for cross-frame data association. For instance, MOTRv2 [3] integrates YOLOX with MOTR tracker, where YOLOX creates high-quality object proposals to facilitate the association process by MOTR. However, TBD has a notable problem: the tracking performance is highly relied on the detection results (i.e. the core impact on the tracking performance is the threshold of the detector and subsequent matching mechanisms). In particular, the fixed threshold of the detector is difficult to cope with complex situations where multiple objects interact. Classical TBD based methods include SORT [4], DeepSORT [1], and ByteTrack [2] etc. Among them, SORT [4] simply uses intersection over union (IoU) and motion information to measure matching similarity and creates new trajectories and reject lost ones. DeepSORT [1] employs a matching cascade and calculates Mahalanobis distance between predicted track boxes and detection boxes (instead of IoU), since the accuracy of the inactive tracks decreases over time. However, the Mahalanobis distance is only used to avoid irrational assignments, which is not suitable for matching. ByteTrack [2] associates the low-confident detections that are unmatched in the first stage, which indeed pushes the record of MOT to a new level and thereby has been adopted by many following works. However, the high-confident detections are usually preferred over low-confident ones, while the low-confident detections will no longer be assigned to inactive trajectories. Fortunately, with the continuous development of object detection, the detection results are becoming increasingly reliable.

Apart from detection, frequent object occlusions under crowded scenes are still the major challenge for MOT[5], causing IDS. Various appearance models with effective feature learning methods are employed to calculate feature similarity so as to correct object ID when occlusions occur [6]. TransCenter [7] advocates the use of image-related dense detection queries and efficient sparse tracking queries under their query learning networks (QLN). AMtrack[8] is built upon an encoder-decoder Transformer architecture, which realizes data association across frames via evolving a set of track predictions. However, it has slower inference speed, and its predictive ability on linear motion is not as good as KF. DeNoising-MOT [9] learns denoising process in an encoder-decoder structure and develops a cascaded mask strategy to prevent the mutual suppression between neighboring trajectories. However, as the scenes become more complex, the tracking performances of the above methods will still be severely affected by occlusion. Although recent improvements in detector performance are conducive to correct object associations., even the best detectors make false predictions. There remains a room for improvement in existing methods using detections.

The linear KF algorithm has long been widely used in visual object tracking [2,10]. During the update process, the measurement noise is set to a constant value. For a reliable detection result, the measurement noise should be low accordingly. To obtain more accurate update results, we adjust the measurement noise based on CS (output by the detector) so as to ensure the quality of the tracking results. During the matching process between the prediction and detection boxes, IoU and feature similarities are usually used to form CM (to measure the discrepancy between the predicted target position and the detection box) of the Hungarian algorithm. When computing CM, the reliability of the current bounding box is often ignored. We therefore fuse CS into CM to improve its reliability. In some cases, some trajectories exist for only a period of time but discontinue in the following frames, which disturb the subsequent trajectory matching and cause incorrect association. We regard them lost and hence delete them in due course. Through analyzing the connection between the detection and associations, we believe the detection results have not been fully utilized. Our results show that the performance of the current TBD based methods can be further improved by combining CS of the detection results. Therefore, an appropriate detection processing scheme from tracking perspective is required to bridge the gap between detection and tracking applications.

In a nutshell, we have made contributions in 3 key stages of MOT, including data association, trajectory prediction and trajectory management:

1. In data association stage, confidence scores are integrated into the cost matrix to improve matching accuracy and achieve more reliable assignment results.

2. In trajectory prediction stage, a confidence based trajectory prediction scheme has been proposed based on KF, which achieves more accurate prediction performance by controlling the measurement noise in KF.

3. In trajectory management stage, a trajectory deletion scheme has been proposed to determine the duration of trajectories and delete less reliable trajectories to avoid possible incorrect matches.

The experimental results shows that our tracking scheme further improves the performances of the current TBD based methods. On the MOT Challenge benchmarks, ConfMOT ranks among the top on MOT17 [12] and MOT20 [13]. ConfMOT achieves 64.5% HOTA and 80.5% MOTA on the MOT17 dataset, and 62.9% HOTA and 78.3% of MOTA on the MOT20 respectively.

The rest of the paper is organized as follows: Section 2 briefly reviews related work, Section 3 describes our proposed method in details. Experimental results are provided in Section 4 with ablation studies. Finally, a conclusion is drawn in Section 5.

2. Related works

With the rapid development of object detection [11], the current research hotspots of visual tracking [2,14] has been shifted from designing complex and individual trackers to subsequent data association schemes that are built upon advanced detectors. The main goal is to maintain the correct trajectories of the objects.

2.1. Motion Models

Most of the recent TBD based methods are based on motion models (under the assumption of constant velocity). As one of the classical models, KF is a Bayes filter which predicts and updates in a recursive way. The ground-truth state is an unobserved Markov process, while the measurement is observed from a hidden Markov model. The measurement noise is the noise exists in signal observation process. OC-SORT [14] uses virtual trajectories to smooth parameters for solving the cumulative error of KF. ByteTrack [2], on the other hand, uses a normal linear KF to associate every detection box to recover true objects and filter out the background elements. However, the above 2 methods apply the same KF to all targets, achieving trajectory prediction quickly and efficiently, but ignoring the differences between various detections. You et al. [15] proposes a novel Spatial-Temporal Topology-based Detector (STTD) algorithm and introduces a topology structure to dynamics of moving targets. It indeed reduces false positives, but since it only considers group motion between targets to build topology, it hence overlooks camera motion. Li et al. [6] presents a TBD based framework for MOT, where a deep association network is developed (followed by detection) to infer the association degree of objects. Despite some advantages in correlation operation, the object detection and tracking tasks are somehow independent of each other, where 2 deep feature extractions are needed in each stage. GIAOTracker [16] proposes a modified KF with confidence scores. Khurana et al. [17] adopts depth estimates from a monocular depth estimator to forecast the trajectories of occluded people. However, they rely heavily on appearance features, which introduces high computational cost. Wang et al. [18] publishes a novel approach to tackle long-term vehicle tracking without appearance information. But it cannot be extended to other types of objects. Tracktor++ [19] employs camera motion compensation for frame alignment. The former method performs global association by developing an appearance-free link model to address missing detection and missing association problems. While the latter employs global information and optimization strategies to associate object trajectories. But it is not applicable to online real-time tracking.

2.2. Cost Matrix

In order to update the objects’ trajectories, the prediction boxes obtained from the KF prediction phase need to be matched with the detection boxes obtained from the detector. OC-SORT [14] claims that using IoU alone is sufficient to generate good results if the detector is accurate enough. But it ignores the impact of detection differences on the cost matrix. MOTDT [20] constructs the cost matrix of the tracking processes with appearance features and IoU respectively. FairMOT [21], on the other hand, fuses IoU and appearance feature, but it does not consider the negative impact of unreliable cost matrix. The above 2 methods combine multiple indicators to form a cost matrix, which can solve the confused assignment problem to some extent. However, extraction of appearance information also slows down the tracking speed in return.

2.3. Trajectories Management

Most trackers set different durations for lost trajectories for different datasets. For example, JDE [10] and FairMOT [21] save the unmatched trajectories for 30 frames, while ByteTrack [2] keeps lost trajectories for 14 frames for MOT17-06 and 25 frames for MOT17-14 according to the length of the videos. However, these parameter settings require pre-processing of the video, sine the time of disappearing trajectory varies due to different occlusion situations. It would be impractical to make different settings manually for different datasets. From a practical perspective, a unified trajectory processing scheme should be developed that is applicable to different datasets, while different time intervals should be assigned to different kinds of lost trajectories.

3. Method

The block diagram of our tracker is drawn in Figure 1. Basically, CS is incorporated into the tracking process, where the detection boxes with either high or low scores are taken care of by the first and the second association processes respectively to ensure correct associations of trajectories.

3.1. Confidence-Based Adaptive KF

KF (with a constant velocity) is commonly adopted in object tracking tasks in building a motion model. It follows the Markov assumption that the current system state is only related to the state of its previous moment, which is irrelevant to any previous states. Then the previous state vector and the state observed at the current frame are needed to estimate the current state.

KF consists of a prediction step and an update step. In the first step, KF generates the estimated state variables (along with their uncertainties), which will then be updated with a weighted average of the estimated state and measurement. Since there is no active control in multi-object tracking, the state vector ${\widehat{x}}_{t-1|t-1}$ at frame $t-1$ is represented by:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\widehat{x}}_{t-1|t-1}={[{\widehat{u}}_{t-1|t-1},{\widehat{v}}_{t-1|t-1},a,{\widehat{h}}_{t-1|t-1},\dot{u},\dot{v},\dot{a},\dot{h}]}^T,\end{array}\end{array}$$

(1)

The update process of KF is expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & K_t=P_{t|t-1}{H}^T{(H{P}_{t|t-1}{H}^T+(1-c_t)R_t)}^{-1},\hfill \\ \hfill & {\widehat{x}}_{t|t}={\widehat{x}}_{t|t-1}+K_t(z_t-H{\widehat{x}}_{t|t-1}),\hfill \\ \hfill & P_{t|t}=P_{t|t-1}-K_{t}H{P}_{t|t-1},\hfill \end{array}\end{array}$$

(2)

$$\begin{array}{c}\hfill H=\left[\begin{array}{cc}{I}_{4\times 4}& O_{4\times 4}\end{array}\right],\end{array}$$

(3)

$$\begin{array}{c}\hfill z_t={\left[\begin{array}{c}{z}_x,z_y,z_w,z_h\end{array}\right]}^T,\end{array}$$

(4)

where $K_t$ is the Kalman gain and H is the observation matrix. $c_t$ is the detection CS and $R_t$ is the covariance of the measurement noise, which represents the level of measurement noise (of the detections in the current frame). A higher noise level indicates a lower weight of the measurement during the state update (i.e., higher uncertainty). $z_t$ represents the observation state vector.

As can be seen, KF estimates the current state vector of the trajectory and updates it through $R_t$. In many previous works [14,21], R is a constant matrix (i.e. the value of the measurement noise remains constant for different detection results). Intuitively, different measurements should contain different levels of noise, and the measurement noise should also be adaptive to detection confidence. In this paper, we use CS to balance the measurement noise of different scales [16]. Specifically, we rectify the constant $R_t$ into $(1-c_t)R_t$ in Eqn. (2), so that a higher CS will result in lower noise (i.e., more reliable detection results). Conversely, a lower $(1-c_t)R_t$ means that the detection will be assigned higher weight in the state update step.

3.2. Data Association

The trajectory of frame t is predicted by the trajectory of frame $t-1$, which is used to calculate IoU with the detection of frame t. Then the IoU is used to construct CM. Then the trajectories will be associated with the detection boxes via Hungarian algorithm based on the CM and be updated via KF. The detection boxes with low confidence scores are regarded as unreliable, which should have higher cost for successful matching.

For ConfMOT, we use CS to rectify CM: the lower the confidence level of the detection, the higher the value of the corresponding element in the matrix will be. Formally:

$$\begin{array}{c}\hfill C_{i,j}=1-{\displaystyle \frac{1+S_j}{2}}\times (1-c_{i,j}^{iou}),\end{array}$$

(5)

where $C_{i,j}$ represents the confidence score of the $j^{th}$ detection box and $c_{i,j}^{iou}$ represents the IoU cost between the $i^{th}$ prediction box and the $j^{th}$ detection box.

3.3. Deletion Strategy

In previous methods, the lost trajectories were usually kept for a certain number of frames, which may be recovered if matches were found, or otherwise deleted. However, for objects with lower CS, their trajectories (or recovered trajectories) are less reliable. We use the track score to define the reliability of a trajectory by summing up the CS of an object on its last tracklet. For a tracklet that lasts for n frames, its track score is expressed as:

$$\begin{array}{c}\hfill S_{track}=\sum _{i=1}^n{S}_i,\end{array}$$

(6)

where $S_i$ represents the CS of an object in the $i^{th}$ frame of the traklet. The lower the track score, the more unstable and less reliable the trajectory will be, which will be endowed shorter time of duration before it is labeled as lost. For the lost trajectories, we use the following equation to determine the time to delete them:

$$S_{\mathrm{del}}=min\left\{\alpha S_{\mathrm{track}}+S_{\mathrm{last}},1\right\}-log\left(1+\beta T_{\mathrm{umatch}}\right),$$

(7)

where $S_{last}$ is the confidence score of the last frame of the tracklet and $T_{umatch}$ is the number of frames when the trajectory does not find a match in the following frames. $\alpha$ and $\beta$ are two weights. A trajectory will be deleted if its confidence score $S_{del}$ is lower than a certain threshold. To avoid a reliable trajectory with high confidence score but short duration from being deleted in an early stage (e.g. a newly generated but soon occluded trajectory), we sum the confidence score of the last frame with the track score to ensure that the track score is not too low. Meanwhile, different trajectories should have different time durations before deletion, so as to reduce the possibility of object drifting to other unreliable trajectories that last for long time.

3.4. Tracking Procedure

The pseudo-code of tracking is shown in Algorithm 1, and the corresponding diagram has been drawn in Figure 2 to better understand the process. Firstly, we adopt YOLOX as our detector to obtain the bounding boxes and CS, where the detection boxes are divided into 2 parts: ${\mathcal{D}}_{high}$ (yellow in Figure 2) and ${\mathcal{D}}_{low}$ (green in Figure 2) based on confidence thresholds ${\tau }_{high}$ and ${\tau }_{low}$(which are set to 0.6 and 0.1 respectively according to OC-SORT[14]). The bad detection boxes with CS lower than ${\tau }_{low}$ are removed. Then we use KF to predict the new locations of each trajectory in ${\mathcal{T}}_{all}$ (labeled 1, 2, 3 in Figure 2).

The first association is performed between the high score boxes ${\mathcal{D}}_{high}$ and all trajectories ${\mathcal{T}}_{all}$. We only use the IoU CM via Eqn. (5) and utilize Hungarian algorithm to assign the detection boxes to corresponding trajectories. In particular, if the IoU cost between the detection box and the prediction box is greater than a threshold, then the matching will be rejected. The unmatched detections in the first association are kept in ${\mathcal{D}}_{remain}$ and the unmatched trajectories (e.g. No. 3 in Figure 2) are kept in ${\mathcal{T}}_{remain}$ (line 14 to 18 in Algorithm 1), which will be matched with the low score boxes in the second stage association.

In the second association, ${\mathcal{T}}_{remain}$ will be associated with the low score boxes ${\mathcal{D}}_{low}$. The unmatched detection boxes in ${\mathcal{D}}_{low}$ are regarded as the background and unmatched trajectories are marked as lost and kept in ${\mathcal{T}}_{lost}$(line 19 to 21 in Algorithm 1), which will be deleted after a certain period of time.

The successfully matched trajectories are updated by KF through Eqn. (2). For trajectories in ${\mathcal{T}}_{lost}$, their CS (${\mathcal{S}}_{del}$) will be calculated via Eqn. (7). If they are lower than ${\tau }_{del}$, they will be moved from ${\mathcal{T}}_{lost}$ to ${\mathcal{T}}_{del}$. If a lost trajectory is recovered, it will be moved from ${\mathcal{T}}_{lost}$ to $\mathcal{T}$, and the track score will be recalculated.

Finally, for detection boxes in ${\mathcal{D}}_{remain}$, we initialize new trajectories if their CS are higher than ${\tau }_{new}$ and exist for 2 consecutive frames(line 29 to 31 in Algorithm 1).

4. Experiment

4.1. Datasets

The training process of MOT17 [12] is conducted jointly on CrowHuman [22], CityPersons [23], ETHZ [24] and the training set of MOT17. And we use CrowdHuman and the training set of MOT20 [13] to train MOT20. While testing is carried out on MOT17 and MOT20 under the “private detection" protocol. For ablation study, we combine CrowdHuman and half of the training set of MOT17 as the

Algorithm 1: Pseudo-code of tracking

Note: Trajectory recovery process is included in the matching process of lost trajectories.

training set and test our tracker on the second half of MOT17. The initial parameters of our model is pre-trained on COCO [25].

4.2. Metrics

We adopt Higher Order Tracking Accuracy (HOTA) Multi-Object Tracking Accuracy (MOTA), ID F1 Score (IDF1) (the ratio of correctly identified detections over the average number of ground-truth and computed detections) and Number of Identity Switches (IDS) as the main metrics to evaluate the performance of our model. Considering MOTA is mainly used to assess detection performance while IDF1 characterizes the ability to keep constant identities, we use HOTA as the final evaluation metric for ranking, which is a more comprehensive indicator to reflect the general performance of detection accuracy, association and localization. In addition, we adopt Association Accuracy (AssA), Association Precision (AssPr), Association Recall (AssRe), Detection Accuracy (DetA) and Localization Accuracy (LocA) to further compare the trackers with similar performances. AssA, AssPr and AssRe are used to measure association performance while DetA for detection quality.

4.3. Implementation Details

All the experiments are implemented in PyTorch and under NVIDIA Tesla V100 GPU. We use YOLOX as the detector with YOLOX as the backbone. The datasets and training strategy are described in Section 4.1. The optimizer is SGD with weight decay of $5\times {10}^{-4}$ and a momentum of 0.9 and the initial learning rate is set to ${10}^{-3}$. The parameters of the tracker are the same as the baseline. The high and low confidence threshold ${\tau }_{high}$ and ${\tau }_{low}$ are empirically set to 0.6 and 0.1 respectively. If the CS of unmatched detections are higher than 0.7[2], we initialize new trajectories starting from them. The lost trajectories will be deleted if their CS are lower than 0.1. GSI is adopted as post-processing method.

4.4. Experimental Results

Extensive experiments have been conducted on MOT17 [12] and MOT20 [13] to testify the effectiveness of our tracker.

4.4.1. Resutls on MOT17

The video sequences in MOT17 are filmed by both static and moving cameras. The comparative results on MOT17 are listed in Table 1. Generally, ByteTrack [2], OC-SORT [14],SCTrack [43] etc. and ConfMOT outperform other trackers with some margins. Among them, ConfMOT surpass ByteTrack by 1.4% and 2.0% in HOTA and MOTA respectively, with lower IDS as well. Meanwhile, we lead OC-SORT in HOTA, MOTA and IDF1. Our obvious advantage in HOTA is attributed to the proposed early trajectory deletion strategy which ensures the tracking accuracy.

To further investigate the subtle difference among ByteTrack [2], OC-SORT [14] and our tracker, we compare another 5 metrics: AssA, AssPr, AssRe, DetA and LocA. As mentioned in Section 4.2, the first 3 metrics reflect the data association ability of a tracker, while DetA and LocA demonstrate the accuracies of object detection and localization respectively. As shown in Table 2, except that we lag behind OC-SORT in AssPr, ConfMOT outperforms other 2 trackers in all other metrics. The reason is that OC-SORT is stronger than our trackers in association ability, but weaker in detection. It is worth noting that ConfMOT ranks first in AssRe, which reflects the accurate prediction of the target trajectories.

A group of visualization results of ConfMOT on the test set of MOT17 is shown in Figure 3. MOT17-01 scene is a normal outdoor scenario with pedestrians walking around. Apparently, the color of the bounding boxes do not either change or swap, indicating constant identities during tracking process. MOT17-03 is a crowded outdoor scene, in which our tracker still completes tracking task without obvious miss detections or ID switches. MOT17-06 is a normal street scene, it’s worth noting that the old man with number 27 dark blue box (in the middle of the scene) always remains unchanged after several overlap by other pedestrians, which indicates the ability of our tracker in dealing with occlusions. In MOT17-07, the video is taken by moving cameras, and there is a man sitting at the left corner in the first frame, who is moving closer in the following frames with a constant ID. The MOT17-8 shows a tracking scenario with varying lighting conditions. Still, our tracker demonstrates robustness in tracking multiple targets with fixed IDS. Lastly, MOT17-14 is a video sequence containing small targets which is captured on a moving box. Clearly, we can correctly detect and locate the small targets over moving viewpoint.

4.4.2. Results on MOT20

MOT20 is a more challenging benchmark with crowded scenes. As shown in Table 1, ConfMOT ranks first in HOTA, MOTA and IDF1 due to our proposed deletion strategies, but it also causes slightly higher IDS, since the lost tracks are difficult to be matched. Like MOT17, we also compare AssA, AssPr, AssRe, DetA and LocA among OC-SORT, ByteTrack and ConfMOT. As shown in Table 3, ConfMOT surpasses both OC-SORT and ByteTrack in all 5 metrics, which reflects stronger association accuracy and more precise localization. The deletion of unreliable lost trajectories in the early stage ensures the overall tracking performances, while the integration of the confidence-based cost matrix guarantees the robustness of the tracker during long period of tracking process.

A group of visualization results of ConfMOT on the test set of MOT20 is shown in Figure 4, including indoor and outdoor scenarios under different lighting conditions. As mentioned earlier, MOT20 include crowded scenes. MOT20-04 is an outdoor scene at night, while MOT20-06 and MOT20-08 are daytime scenarios. No matter under which scenes, our tracker maintains excellent tracking performances, which is less sensitive to lighting conditions, and the ID of each target remain largely unchanged.

4.4.3. Comprehensive Analysis

As mentioned earlier, ByteTrack [2], OC-SORT [14] and our tracker achieve better results than other competitors. Here, we further analyze the advantages / disadvantages of the 3 methods. ByteTrack [2] utilizes pure IoU to process all the detections. In contrast, we process the detection boxes using the cost matrix which is adjusted by the confidence score. Our scheme matches the high-quality detection boxes in a more effective way, resulting in more stable assignment results. As a result, we achieve the best overall secondary indexes (including AssA, AssPr, AssRe, DetA and LocA). Besides, we obtain more precise object location due to the proposed modified KF, which has been verified by the highest DetA (Detection Accuracy) we have reached. In summary, we have achieved the best overall results on MOT-17 and on MOT-20.

4.4.4. Visualization of Trajectories

To further demonstrate the superiority of our tracker, a comparison among ByteTrack [2], OC-SORT [14] and ConfMOT is made in Figure 5, in which the trajectory of each object of the last 50 frames is drawn with different colors. Obviously, OC-SORT (shown in Figure 5(a)) has many sporadic segments of trajectories, which reflect target loss under crowded scene, while it has almost no trajectories at the left hand side of the scene. ByteTrack (shown in Figure 5(b)) has more trajectories, but there are some zigzags and stagnation, indicating incorrect matching and ID switches. In comparison, as shown in Figure 5(c), the trajectories of ConfMOT appear more stable, with almost identical trajectory lengths and no broken segments. In addition, the colors of the trajectories do not change within the dense areas, indicating the robustness of ConfMOT under crowded scenes.

4.5. Ablation Studies

In ablation experiment, we use YOLOX as the backbone and train our model jointly on CrowdHuman and the first half of the training set of MOT17. While the second half of the training set of MOT17 is used for validation. In this section, we will firstly verify the effectiveness of the core modules of our tracker, including KF, CM+CS and deletion strategy (DS). The comparative results on MOT17 validation set is shown in Table 4.

Figure 5. Comparison of trajectories generated by three tracking methods.

Figure 5. Comparison of trajectories generated by three tracking methods.

4.5.1. KF Based on the Confidence Score

As shown in Table 4, when KF (with CS) is integrated into the tracking network, we enjoy a slight increase in both HOTA and IDF1. By incorporating confidence, KF assigns weights to different detections based on their CS, making it more robust to noise and uncertain detections. During the update of KF, the high quality detection boxes will be assigned higher weights than the prediction boxes, making the detection boxes more dependent on the detection results. Reversely, for low quality detection results, the detection boxes will depend more on the prediction boxes, which will be assigned higher weights.

4.5.2. Deletion Strategy based on the confidence score

As shown in Table 4, when the proposed lost trajectory deletion strategy has been added into the tracking architecture, the HOTA and IDF1 value increase by 0.2% and 0.4%, respectively. This results illustrate the fact that when a lost trajectory has been deleted, the subsequent detected target cannot find a match and therefore a new trajectory will be created. Moreover, the increased IDF1 suggests that the deleted trajectories are indeed unreliable, and our deletion strategy reduces the likelihood of incorrect matches between a detected target and the wrong trajectory. Intuitively, if a trajectory remains unmatched for a certain period of time and is re-matched again, it’s quite possible that it matches with a different target (instead of the previous one). Therefore, our deletion strategy becomes very necessary to avoid target drifting caused by re-matching.

4.5.3. Cost matrix based on the confidence score

When calculating the matching cost between the prediction and detection boxes, such cost will be higher for low-quality detection boxes. As shown in Table 4, after the CS incorporated into CM, both HOTA and MOTA increase by 0.4%. This result indicates that the integration of confidence score indeed reduces the probability of matching low-quality detection boxes and continuation of incorrect trajectory.

5. Conclusion

In this paper, we thoroughly analyze the limitations of current TBD based trackers, and present ConfMOT to ensure stable continuation of trajectories among different objects. To realize the goal, we adopt YOLOX as our detector to obtain the detection results. We adopt two-stage association strategy and predict the trajectories based on the confidence-score and Kalman filtering technique. Then a cost matrix has been constructed to measure the cost of unreliable matching. Finally, a deletion strategy has been proposed to determine the duration of a trajectory and delete the less reliable trajectories. Extensive experiments have been conducted on MOT17 and MOT20 datasets along with ablation studies to testify the efficacy of our tracker. The results prove that our trackers have obvious advantages in the HOTA metric compared to other methods, due to the cost matrix and trajectory deletion strategy we designed. However, the performance of our tracker in IDS is mediocre (ID switch caused by occlusion), since we delete unreliable trajectories in an early stage.

In future work, we will be engaged in developing light-weight appearance feature extraction module to improve the robustness of the model in dealing with fast appearance changes. We will also employ specific motion modelling schemes to ensure the stability and continuation of target trajectory.