Towards the Development of A Low-Cost Solution for Wildlife Tracking using An Unmanned Aerial Platform

1. Introduction

Zoological surveys have always been a great challenge for conservation authorities due to various factors. The prime factor is the operational cost and complexity in conducting such surveys that are normally carried out using helicopters. Firstly, a skilled crew is required in order to carry out these surveys. Obviously, a skilled photographer is required; moreover, the need of an experienced pilot is indispensable.

As reported in a manual [1] produced by World Wide Fund (WWF), aerial surveys in Zimbabwe are conducted jointly by trained staff of Zimbabwe Parks and Wildlife Management Authority (ZMPWMA) and WWF. It is reported that these surveys are effective in areas usually greater than 1,000 km2. In addition to this, these surveys are only useful for large dark bodied animals. Therefore, it is difficult to track small bodied animals. Normally, a dry season is preferred for surveys as animals more visible in the dry season [1]. For animal counting, a skilled observer is required to sit on either side of the helicopter to count the animals manually; this poses safety threats to the observer as well as to the helicopter. Further, possibility of human error cannot be neglected. On the other hand, charter costs are too high ranging 1800 £ per hour to 7550 £ per hour depending upon the terrain and altitude of the flight.

Unmanned Air Vehicles (UAV) such as Quad copters are used for various purposes around the globe. A quad rotor provides a low cost solution with ease of maneuverability with vertical take-off and landing. Moreover, these UAVs can be operated with minimal training. The most significant advantages of these quadrotors over conventional fixed wing aircrafts are that they can take-off and land in limited space and easily hover above the target. The main purpose of this study is to develop an easily accessible solution using a quadrotor which can be used to monitor wildlife. This will not only reduce the survey cost significantly, but also it will provide a portable solution to wildlife conservation specialist.

1.1. Aerial Imaging

The idea of extraction of objects from aerial imagery was in market since the 1990s as proposed by Helmut Mayer. This approach focuses on extraction of buildings from aerial imaging using two criteria; content and density. Objects are found on the basis of basic 2D & 3D models. Movable objects such as people and vehicles were not considered in this research. [9]

In order to collect information for town planning and environment related investigations, this same strategy with focus on coupling of 2D & 3D images for estimation of shape and locations of buildings was utilized. They start off with image descriptions using points, lines and regions ultimately leading to integration of these 2D and 3D images and graphs. [10]

1.2. Unmanned Aerial Imaging

The use of unmanned aerial vehicles was popularly sought after in the same decade. UAVs were rendered as the human assistants that could monitor and record unusual events using intelligent visual surveillance.

Monitoring of wildlife using the airplanes has been in the game for years although the use of unmanned aerial vehicles also known as drones has been in-demand in the last two or three decades.

In order to conduct survey of wildlife using UAVs, various approaches have been proposed. Thermal imaging technique was used by M. Israel et al. [2] for the detection of fawns in the meadows. Images were captured using a thermal infrared camera and video transmission was also used for manual animal detection. In order to accelerate the flow work for high detection, a waypoint planning software was also developed.

Another thermal imaging technique was used by P. Christiansen et al. [3] for wildlife recognition. Top-view images are taken from a telescopic boom above the target area. Detection of hot objects and feature extraction is performed on the basis of dynamic threshold. A thermal feature extraction algorithm is proposed for classification of animals.

An acoustic biotelemetry approach for wildlife tracking using an autonomous UAV was proposed by Y. Q. Chen et al. [4]. A low-cost RF telemetry system was developed for triangulation and geo-location of the target. A Fish Tracking Payload System (FTPS) was designed and developed on which several flights were conducted and successful detection was made. [20]

The technique applied by A. M. Jensen et al [5] focused on the multi-UAV navigation and transmitter localization. Swarm-like navigation methods and Kalman Filters were applied on a real world simulator. The results were then compared using Monte Carlo Analysis.

A vision based technique is used by W. Selby et al[6] for tracking of marine animals. Algorithms of image segmentation and target identification were developed for monitoring whales at sea. This helped in identifying and following targets at constant speed.Various other approaches can be seen in work done by J.Linchant et al [7]. The recent advances in the visual tracking methods and techniques [14]

1.3. Preserving Wildlife

One of the emerging technologies to preserve the diversity of individual animals is being improved by devices and methods that are able to track animals using the drones. Owing to restraints on size and resources, use of airplanes is a lot of bill but drones or UAVs provide a better, timely and cheap alternative to the mentioned way as well as satellites. [11]

Also, a small device using radio technology on a near-earth orbit satellite to track small animals around the globe was proposed in the late 2006. [12]

Yungfei Fang et al. presented a method for wildlife monitoring and surveying of animals in wilderness. They presented a method utilizing global patterns of pixel by applying optical flow methods for surveying and monitoring animals on the move. Unwanted parts of the background were removed by applying a pixel velocity threshold. The segmented regions from the above were reinforced further by filtering out negative patches that could possibly belong to the background. [13]

This paper presents a technique to develop a low cost solution for the purpose. A simple blob detection algorithm is used and an experiment is designed in order to calculate the error of the developed algorithm.

The paper is organized as follows. Section II explains proposed technique while section III presents the background theory. Experimentation is discussed in section IV. Results are presented in section VI. Section VII presents the conclusion and future work.

2. Proposed Approach

Aerial images acquired through the aerial platform are imported in Matlab. Since the acquired images are of very high resolution i.e. 4160 x 3120, therefore, the proposed algorithm first resizes the images to 512 × 512 pixels. The algorithm then employs Gaussian filter for smoothing of images. In order to compensate the lighting effects, the image is converted from RGB color space to HSV color space. After application of HSV filter optical flow is applied on images. Application of optical flow calculates the flow vector, to get better estimation threshold the flow vectors. After thresholding we have flow vectors (u,v). Summary of the described algorithm is given in Figure 1.

2.1. Background theory

“An optical flow is the technique to calculates the displacement of brightness patterns from one image frame to another”[15] [20]. Several different approaches to optical flow estimation have been proposed, including correlation and energy-based methods. This paper concentrates on differential approach and gives comparison of the two differential techniques namely Lucas-Kanade and Horn-Schunck. These methods are widely used for flow estimation. Horn-Schunck algorithm and Lucas-Kanade algorithm are analysed and compared based on the results obtained.

2.1.1. Differential Techniques:

Differential techniques compute optical flow information from spatial and temporalvariations of the image brightness based on brightness constancy and temporal consistence assumptions[15],[16]. These assumptions result in a motion constraint estimated by the first order Taylor’s expansion of

$$C\left(X,t\right)=C(X-U_t,0)$$

(1)

where $U={(u,v)}^T$. The Taylor expansion of (1) or intensity conservation assumption

$$\frac{dC(X,t)}{dt}=0$$

(2)

implies that

$$\nabla C\left(X,t\right).U+C_t\left(X,t\right)=0$$

(3)

Where $\nabla C\left(X,t\right)={(C_x\left(X,t\right),C_y(X,t\left)\right)}^T$ and $\nabla C.U$ denotes the usual dot product. Equation (3) gives the normal component (U_n) of motion of constant intensity spatial contour as $U_n=s\mathit{n}$, where s is the normal speed and n is the normal direction given by.

$$s\left(X,t\right)=\frac{{-C}_t(X,t)}{‖\nabla C(X,t)‖}$$

$$n\left(X,t\right)=\frac{C_t(X,t)}{‖\nabla C(X,t)‖}$$

(4)

Second order differential methods bounds 2-D velocity using second order derivatives

$$\left[\begin{array}{cc}{C}_{xx}(X,t)& C_{yx}(X,t)\\ C_{xy}(X,t)& C_{yy}(X,t)\end{array}\right]\left[\begin{array}{c}{U}_1\\ U_2\end{array}\right]+\left[\begin{array}{c}{C}_{tx}(X,t)\\ C_{ty}(X,t)\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]$$

(5)

The conservation of equation (5) implies $\nabla C\left(X,t\right),d\nabla C(X,t)/dt=0$

The coefficients of these equations are combinations of spatial and temporal derivatives of the image brightness.

• A. Horn– Schunck method:

The Horn-Schunck method uses the assumption that flow is smooth over the whole image[17].Therefore solutions which show more smoothness are preferred. The minimum distortions in the flow are ensured by minimizing a global energy function. The energy function is given for two-dimensional imagery as[15],[18]:

$$\int {\left(C_{x}u+C_{y}v+C_t\right)}^2+{\phi }^2({‖\nabla u‖}^2+{‖\nabla v‖}^2)dx$$

(6)

where $C_x$,$C_y$ and $C_t$ are the derivatives of the pixel intensity values along the x, y and time t dimensions respectively$,U={(u\left(x,y\right),v(x,y\left)\right)}^T$ is the flow vector, and $\phi$ is a regularization constant. To get smoother flow the value $\phi$ is made larger[19]. Iterative equations (6) are used to minimize the global energy function and obtain image velocity in 2-D.

$$u^{J+1}=u^{-J}-\frac{C_x(C_x{u}^{-k}+C_y{v}^{-k}+C_t)}{{\phi }^2+{C_x}^2+{C_y}^2}$$

$$v^{J+1}=v^{-J}-\frac{C_y(C_x{u}^{-k}+C{v}^{-k}+C_t)}{{\phi }^2+{C_x}^2+{C_y}^2}$$

(7)

where J is the number of iteration.

The Horn and Schunck method use first-order differences to estimate intensity derivatives. The Horn-Schunck algorithm applied to Fig 3(a) and Fig 3(b) gives the images in Fig 3(c) and Fig 3(d). The velocities obtained through this method are (-0.16, -0.38).

• B. Lucas–Kanade method:

Although Horn-schunck algorithm is a complete algorithm for optical flow estimation but its iterative nature results in high computational cost[17]. The solution is provided by Lucas-Kanade methods which implements least square method to find the velocity minimizing the constraints errors, squared errors in this case[16],[18]. Major assumption here is constant flow in local neighborhood of the pixel under consideration. Velocity is obtained by solving basic optical flow equations for all the pixels in that neighborhood $\sigma$ satisfying the least square criterion

$${\sum }_{x\in \sigma }{D}^2\left(X\right){[\nabla C\left(X,t\right).U+C_t(X,t\left)\right]}^2$$

(8)

where $D\left(X\right)$ is an n×n diagonal matrix containing the weights. The diagonal matrix ensures that constraints have more influence on pixels at the center of the neighborhood than those at the periphery. The solution to (8) is given by

$$B^T{D}^{2}BU=B^T{D}^{2}b$$

(9)

where for n points $x_i\in \sigma$at a single time t,

$$\begin{gathered} B={[\nabla C(X_1,\dots ..,\nabla C\left(X_n\right)\left)\right]}^T \\ D=diag(D\left(X_1\right),\dots .,D(X_n\left)\right) \\ b=-{\left[C_t\right(X_1),\dots \dots ,C_t(X_n\left)\right]}^T \end{gathered}$$

The solution to (10) is $U={\left[B^T{D}^{2}B\right]}^{-1}{B}^T{D}^{2}b$ , the solution exists if$B^2{D}^{2}B$ is nonsingular matrix.

$$B^2{D}^{2}B=\left[\begin{array}{cc}\sum D^2\left(X\right){C_x}^2\left(X\right)& \sum D^2\left(X\right)C_x\left(X\right)C_y\left(X\right)\\ \sum D^2\left(X\right)C_y\left(X\right)C_x\left(X\right)& \sum D^2\left(X\right){C_y}^2\left(X\right)\end{array}\right]$$

(10)

$B^2{D}^2$ where all sums are taken over points in the neighborhood.

Equations (8) and (9) represent weighted least squares estimates of velocity components from estimates of normal velocities $U_n=sn$. Equation (10) is equivalent to

$$\sum _{x\in \sigma }{D}^2\left(X\right){d^2\left(X\right)[U.n\left(x\right)-s(X\left)\right]}^2$$

Where the coefficients $D^2\left(x\right)$ reflect that how good in the normal velocity estimates are; here $d\left(x\right)=‖\nabla C(X,t)‖$

To solve equation$B^{T}BU=B^T{D}^{2}b$, $B^T{B}^{\prime }s$ eigenvalues should satisfy the constraint ${\lambda }_1\ge {\lambda }_2>0$[17]. ${\lambda }_2$ should not be too small in order to avoid noise. Another constraint is, small value of $\raisebox{1ex}{${\lambda }_1$}\!\left/ \!\raisebox{-1ex}{${\lambda }_2$}\right.$ which if not satisfied leads to the aperture problems. This condition is also works for Corner detection[18].

Fig 4(c) and Fig 4(d) are the resultant images after application of Lucas-Kanade algorithm to Fig 4(a) and Fig 4(b). The velocities obtained through this method are (0.10, -0.63).

Experimentation

Experimentation is performed at around 4 pm in an evening at NUST College of E&ME Rawalpindi using a grid of 1m x 1m having four cells. The aerial platform that is used in experimentation is an X-configuration quad-rotor. It has an empty weight of 989grams and a payload carrying capacity of another 900 grams leading to a total takeoff and flying weight of approximately 2000 grams. The quadrotor uses 4 BLDC motors having thrust of 1280Nm each. The motors are controlled by a PD controller. The BLDCs has a maximum speed of 3200rpm. For imaging, the quad-rotor is equipped with LG G3 mobile phone having a 16 megapixels wide angle camera. Figure 2 shows the imaging platform.

A ball is placed first in the center of the grid that is taken as the starting point. The quad-rotor is made to hover over the grid at a constant altitude of 1.5 m above the ground. The ball is moved to different locations in the grid and the images are acquired. The images are then imported in MATLAB environment and are processed by the developed algorithm to calculate the change in position of the ball in different frames.

Figure 3 shows the experimental grid, whereas; Figure 4 shows the different positions of the ball in the grid. Figure 5 show the outcome of HSV filter and Figure 6 shows an inverted binarized image produce by the algorithm to calculate the change in position of the ball.

Figure 3. Lucas-Kanade.

Figure 3. Lucas-Kanade.

Figure 4. Horn-Schunck.

Figure 4. Horn-Schunck.

Figure 5. Lucas Knade(vector u).

Figure 5. Lucas Knade(vector u).

Figure 6. Lucas Knade(vector v).

Figure 6. Lucas Knade(vector v).

3. Experimental results

Experimental Table 1 shows the outcome of the algorithm versus the measured distance. The mean percentage error for change in X-direction is found to be 6.28%, whereas, for change in Y-direction, it is found to be 15.22%. The main reason for this error is uneven light distribution. This can also be seen in error graphs for X and Y as shown in Figure 7 and Figure 8. It is clear from Figure 6 that only sample no. 6 has a minimum error of 0.69% in the X-direction. This may be because of the reason that at such position the lighting is fairly better. Moreover, it can be seen that for the same reason, the overall error in X-direction is much lesser than the error in Y-direction.

4. Conclusion

The proposed technique shows considerable results with mean percentage error of 6.28% for change in X-direction and 15.22% for change in Y-direction. Future work involves the tracking of real animals in the field using the proposed technique. Figure 9 shows an aerial image of horses available at riding club of the university campus. Figure 10 shows a binarized image of the aerial view shown in

Figure 9. Difference of u.

Figure 9. Difference of u.

Figure 9. Difference of v.

Figure 9. Difference of v.

Figure 9. Future work also involves the implementation of a real-time tracking algorithm using an on-board processor.

Figure 9. Future work also involves the implementation of a real-time tracking algorithm using an on-board processor.