Recognition of the human interaction on the unconstrained videos taken from cameras and remote sensing platforms like a drone is a challenging problem. This study presents a method to resolve issues of motion blur, poor quality of videos, occlusions, the difference in body structure or size, and high computation or memory requirement. This study contributes to the improvement of recognition of human interaction during disasters such as an earthquake and flood utilizing drone videos for rescue and emergency management. We used Support Vector Machine (SVM) to classify the high-level and stationary features obtained from Convolutional Neural Network (CNN) in key-frames from videos. We extracted conceptual features by employing CNN to recognize objects from first and last images from a video. The proposed method demonstrated the context of a scene, which is significant in determining the behaviour of human in the videos. In this method, we do not require person detection, tracking, and many instances of images. The proposed method was tested for the University of Central Florida (UCF Sports Action), Olympic Sports videos. These videos were taken from the ground platform. Besides, camera drone video was captured from Southwest Jiaotong University (SWJTU) Sports Centre and incorporated to test the developed method in this study. This study accomplished an acceptable performance with an accuracy of 90.42%, which has indicated improvement of more than 4.92% as compared to the existing methods.

In this paper, we propose a novel approach of video skimming by exploiting the fusion of video temporal information and keyword information representation extracted from multi-model video information including audio, text and visual indices. In addition, we introduce the brand-safe filtering and sentiment analysis in order to only reserve the user-friendly content in the video skim. In the experiment by using the videos from YouTube-8M dataset, we have proved that the semantic conservation in the video skim from the proposed approach highly outperforms the approaches by only partial information of the video in conserving the semantic content of the video.

The first detection of gypsum (CaSO4.2H2O) by the Mars Science Laboratory (MSL) rover Curiosity in the Gale Crater, Mars created a profound impact on planetary science and exploration. The unique capability of plasma spectroscopy involving in situ elemental analysis in extraterrestrial environments, suggesting the presence of water in the red planet based on phase characterization and providing a clue to Martian paleoclimate. The key to gypsum as an ideal paleoclimate proxy lies in its textural variants, and in this study terrestrial gypsum samples from varied locations and textural types have been analyzed by Laser Induced Breakdown Spectroscopy (LIBS) technique. Petrographic, sub-microscopic and powder X-ray diffraction characterizations confirm the presence of gypsum (hydrated calcium sulphate; CaSO4.2H2O), bassanite (semi-hydrated calcium sulphate; CaSO4.1/2H2O) and anhydrite (anhydrous calcium sulphate; CaSO4) along with accessory phases (quartz and jarosite). The principal component analysis of LIBS spectra from texturally varied gypsums can be differentiated from one another because of the chemical variability in their elemental concentrations. The concentration of gypsum is determined from the partial least-square regressions model. Rapid characterization of gypsum samples with LIBS is expected to work well in extraterrestrial environments.

We introduce PyNTA, a modular instrumentation software for live particle tracking. By using the multiprocessing library of Python and the distributed messaging library pyZMQ, PyNTA allows users to acquire images from a camera at close to maximum readout bandwidth while simultaneously performing computations on each image on a separate processing unit. This publisher/subscriber pattern generates a small overhead and leverages the multi-core capabilities of modern computers. We demonstrate capabilities of the PyNTA package on the featured application of nanoparticle tracking analysis. Real-time particle tracking on megapixel images at a rate of 50 Hz is presented. Reliable live tracking reduces the required storage capacity for particle tracking measurements by a factor of approximately 10³, as compared with raw data storage, allowing for a virtually unlimited duration of measurements

Over the past few years, the importance of video surveillance in securing the national critical infrastructure has significantly increased, whose applications include detecting failures and anomalies. Accompanied by video proliferation is the increasing number of attacks against surveillance systems. Among the attacks, false frame injection (FFI) attacks that replay video frames from a previous recording to mask the live feed has the highest impact. While many attempts have been made to detect FFI frames using features from the video feeds, video analysis is computationally too intensive to be deployed on-site for real-time false frame detection. In this paper, we investigate the feasibility of FFI attacks on compromised surveillance systems at the edge and propose an effective technique to detect the injected false video and audio frames by monitoring the surveillance feed using the embedded Electrical Network Frequency (ENF) signals. An ENF operates at a nominal frequency of 60 Hz/50 Hz based on its geographical location and maintains a stable value across the entire power grid interconnection with minor fluctuations. For surveillance system video/audio recordings connected to the power grid, the ENF signals are embedded. The time-varying nature of the ENF component is used as a forensic application for authenticating the surveillance feed. The paper highlights the ENF signal collection from a power grid creating a reference database and ENF extraction from the recordings using conventional short-time Fourier Transform and spectrum detection for robust ENF signal analysis in the presence of noise and interference caused in different harmonics. The experimental results demonstrate the effectiveness of ENF signal detection and/or abnormalities for FFI attacks.

Detection of the region of interest (ROI) is a critical step in laryngeal image analysis for the delineation of glottis contour. The process can improve both computational efficiency and accuracy of the image segmentation task, which will facilitate subsequent analysis and characterization of the vocal fold vibration as it correlates with voice quality and pathology. This study aims to develop machine learning based approaches for automatic detection of ROI for glottis image sequences captured by high-speed video-endoscopy (HSV), a clinical laryngeal imaging modality. In particular, we first applied the supporting vector machine (SVM) method using histogram of oriented gradients (HOG) feature descriptor, and second, trained a convolutional neural network (CNN) model for this task. Comparisons are made for both approaches in terms of accuracy of recognition and computation time.

Background: Video assisted thoracoscopic surgery (VATS) has become the recommended approach for treatment of resectable lung cancer. However, no large randomized clinical trial has been conducted formally comparing surgical resections completed by VATS to those done by open thoracotomy (OT) in low volume centers. The current study sought to assess differences in recurrence-free survival (RFS), overall survival (OS), positive margins and postoperative length of stay (LOS) between VATS and OT lobectomies in our center. Method: A single institution retrospective chart review from May 2005 through May 2015 was conducted. All patients diagnosed with stage I through III lung cancer who underwent surgical resection were selected. Patient and tumor characteristics recorded included age at diagnosis, sex, tobacco use, tumor location (side and lobe), stage, size and receipt of chemotherapy or radiotherapy. Chis-square and Wilcoxon-Mann-Whitney tests were used to compare demographics, tumor characteristics and LOS. Multiple logistic and Cox regression analyses were used to compute relative risk (RR) for positive margins and mortality hazard ratios along with 95 percent confidence intervals (95%CI), respectively. Results: Of the 235 patients, 101 subjects had VATS while OT was performed in 134 patients. Age at diagnosis, sex, tobacco use, tumor location, and size were comparable for VATS and OT. No significant difference was observed in the relative risk of positive margins for VATS versus OT, RR = 0.56 (95%CI = 0.26, 1.05). However, VATS had shorter median LOS compared to OT (4 vs. 6 days, respectively), p = 0.002. A comparison of VATS versus OT showed no significant difference in the risk of recurrence, HR = 1.21 (95%CI = 0.74, 2.00), or death, HR = 1.34 (95%CI = 0.88, 2.06), in the intent-to-treat population. Similarly, no significant differences in recurrence or mortality risk were observed between VATS versus OT for analyses conducted separately for each cancer stage group or those limited to patients with negative margins. Conclusion: Our study indicates that compared to OT, VATS leads to shorter LOS while achieving comparable margins status, recurrence-free and overall survival regardless of tumor stage at diagnosis.

The operations of digitization, transmission and storage of medical data, particularly images require increasingly effective encoding methods not only in terms of compression ratio and flow of information but also in terms of visual quality. At first, there was DCT (discrete cosine transform) then DWT (discrete wavelet transform) and their associated standards in terms of coding and image compression. After that, the 2nd generation wavelets seeks to be positioned and confronted to the image and video coding methods currently used. It is in this context that we suggested a method combining bandelets and SPIHT (set partitioning in hierarchical trees) algorithm. There are two main reasons for our approach: the first lies in the nature of the bandelet transform to take advantage by capturing the geometrical complexity of the image structure. The second reason stems in the suitability of encoding the bandelet coefficients by the SPIHT encoder. Quality measurements shows that in some cases (for low bit rates) the performances of the proposed coding compete with the well-established ones and opens up new application prospects in the field of medical imaging.

Superpixels are a representation of still images as pixel grids because of their more meaningful information compared with atomic pixels. However, their usefulness for video classification has been given little attention. In this paper, rather than using spatial RGB values as low-level features, we use optical flows mapped into hue-saturation-value (HSV) space to capture rich motion features over time. We introduce motion superpixels, which are superpixels generated from flow fields. After mapping flow fields into HSV space, independent superpixels are formed by iteration of seeded regions. Every grid of a motion superpixel is tracked over time using nearest neighbors in the histogram of flow (HOF) for consecutive flow fields. To define the temporal representation, the evolution of three features within the superpixel region, namely the HOF, HOG, and the center of superpixel mass are used as descriptors. The bag of features algorithm is used to quantify final features, and generalized histogram-kernel support vector machines are used as learning algorithms. We evaluate the proposed superpixel tracking on first-person videos and action sports videos.

Multi-object tracking (MOT) in airborne videos is a challenging problem due to the uncertain airborne vehicle motion, vibrations of the mounted camera, unreliable detections, size, appearance and motion of the moving objects as well as occlusions due to the interaction between the moving objects and with other static objects in the scene.To deal with these problems, this work proposes a four-stage Hierarchical Association framework for multiple object Tracking in Airborne video (HATA). The proposed framework combines data association-based tracking (DAT) methods and target tracking using a Compressive Tracking approach, to robustly track objects in complex airborne surveillance scenes. In each association stage, different sets of tracklets and detections are associated to efficiently handle local tracklet generation, local trajectory construction, global drifting tracklet correction and global fragmented tracklet linking. Experiments with challenging airborne video datasets show significant tracking improvement compared to existing state-of-art methods.

Aiming at the problems that large amount of video monitoring image data in underground coal mines leads to difficulties in transmission and storage, compressed sensing theory is introduced to encode and decode video images, and a new distributed video coding scheme is proposed. In order to obtain more sparse representation and more general applicability, a block-based adaptive sparse base scheme is proposed. For the acquisition of side information, fixed weight is usually used to synthesize side information and the correlation between different image blocks is neglected, a block-based classification weighted side information generation scheme is proposed. Experimental results show that the block-based classification codec scheme can make full use of inter-frame correlation. Under the appropriate sampling rate, the PSNR value of video reconstruction increases, which effectively improves the quality of video frame reconstruction.

Capsule endoscopy, which uses a wireless camera to take images of the digestive tract, is emerging as an alternative to traditional colonoscopy. The diagnostic values of these images depend on the quality of revealed underlying tissue surfaces. In this paper, we consider the problem of enhancing the visibility of detail and shadowed tissue surfaces for capsule endoscopy images. Using concentric circles at each pixel for random walks combined with stochastic sampling, the proposed method enhances the details of vessel and tissue surfaces. The framework decomposes the image into two detail layers that contain shadowed tissue surfaces and detail features. The target pixel value is recalculated for the smooth layer using similarity of the target pixel to neighboring pixels by weighting against the total gradient variation and intensity differences. In order to evaluate the diagnostic image quality of the proposed method, we used clinical subjective evaluation with a rank order on selected KID image database and compared to state of the art enhancement methods. The result showed that the proposed method provides a better result in terms of diagnostic image quality and objective quality contrast metrics and structural similarity index.

In this study, we examine microscale waterflooding in a randomly close-packed porous medium. Three different porosities are prepared in a microfluidic platform and saturated with silicone oil. Optical video fluorescence microscopy is used to track the water front as it flows through the porous packed bed. The degree of water saturation is compared to water containing two different types of chemical modifiers, sodium dodecyl sulfate (SDS) and polyvinylpyrrolidone (PVP), with water in the absence of a surfactant used as a control. Image analysis of our video data yield saturation curves and calculate fractal dimension, which we use to identify how morphology changes the way an invading water phase moves through the porous media. An inverse analysis based on the implicit pressure explicit saturation (IMPES) simulation technique uses mobility ratio as an adjustable parameter to fit our experimental saturation curves. The results from our inverse analysis combined with our image analysis show that this platform can be used to evaluate the effectiveness of surfactants or polymers as additives for enhancing the transport of water through an oil-saturated porous medium.

In the context of video background-foreground separation, we propose a compressive online Robust Principal Component Analysis (RPCA) with optical flow that separates recursively a sequence of video frames into foreground (sparse) and background (low-rank) components. This separation method can process per video frame from a small set of measurements, in contrast to conventional batch-based RPCA, which processes the full data. The proposed method also leverages multiple prior information by incorporating previously separated background and foreground frames in an n-l₁ minimization problem. Moreover, optical flow is utilized to estimate motions between the previous foreground frames and then compensate the motions to achieve higher quality prior foregrounds for improving the separation. Our method is tested on several video sequences in different scenarios for online background-foreground separation given compressive measurements. The visual and quantitative results show that the proposed method outperforms other existing methods.

Exponential increase in mobile video delivery will continue with the demand for higher resolution, multi-view and large-scale multicast video services. Novel fifth generation (5G) 3GPP New Radio (NR) standard will bring a number of new opportunities for optimizing video delivery across both 5G core and radio access network. One of the promising approaches for video quality adaptation, throughput enhancement and erasure protection is the use of packet-level random linear network coding (RLNC). In this work, we discuss the integration of RLNC into the 5G NR standard, building upon the ideas and opportunities identified in 4G LTE. We explicitly identify and discuss in detail novel 5G NR features that provide support for RLNC-based video delivery in 5G, thus pointing out to the promising avenues for future research.

Dictionary Learning (DL) and Sparse Representation (SR) based Classifier have impacted greatly on the classification performance and has had good recognition rate on image data. In Video Semantic Analysis (VSA), the local structure of video data contains more vital discriminative information needed for classification. However, this has not been fully exploited by the current DL based approaches. Besides, similar coding findings are not being realized from video features with the same video category. Based on the issues stated afore, a novel learning algorithm, called Sparsity based Locality-Sensitive Discriminative Dictionary Learning(SLSDDL) for VSA is proposed in this paper. In the proposed algorithm, a discriminant loss function for the category based on sparse coding of the sparse coefficients is introduced into structure of Locality-Sensitive Dictionary Learning (LSDL) algorithm. Finally, the sparse coefficients for the testing video feature sample are solved by the optimized method of SLSDDL and the classification result for video semantic is obtained by minimizing the error between the original and reconstructed samples. The experiment results show that, the proposed SLSDDL significantly improves the performance of video semantic detection compared with the comparative state-of-the-art approaches. Moreover, the robustness to various diverse environments in video is also demonstrated, which proves the universality of the novel approach.

Deck inclination and vertical displacements are among the most important technical parameters to evaluate the health status of a bridge and to verify its bearing capacity. Several methods, both conventional and innovative, are used for structural rotations and displacement monitoring; no one of these does allow, at the same time, precision, automation, static and dynamic monitoring without using high cost instrumentation. The proposed system uses a common laser pointer and image processing. The elastic line inclination is measured by analyzing the single frames of a HD video of the laser beam imprint projected on a flat target. For the image processing, a code was developed in Matlab® that provides instantaneous rotation and displacement of a bridge, charged by a mobile load. An important feature is the synchronization of the load positioning, obtained by a GNSS receiver or by a video. After the calibration procedures, a test was carried out during the movements of a heavy truck maneuvering on a bridge. Data acquisition synchronization allowed to relate the position of the truck on the deck to inclination and displacements. The inclination of elastic line was obtained with a precision of 0.01 mrad. The results demonstrate the suitability of the method for dynamic load tests, control and monitoring of bridges.

Clinical rehabilitation and sports performance analysis both require the objectification of movement. Kinovea^© is a free 2D motion analysis software that enables the establishment of kinematics parameters. This low-cost technology has been used in sports sciences, as well as clinical field and research work. Although it has been validated as a tool with which to assess time-related variables, this is not yet the case regarding angular and distance variables. The main objective of this study was to determine the validity and reliability of the Kinovea software in obtaining angular and distance data at different perspectives of 90°, 75°, 60° and 45°. For this purpose, a figure with 29 points was designed (in AutoCAD) and 24 frames analysed. Each frame was examined by three observers who each made two attempts. For each export data item, 20 angles and 20 distance variables were calculated, with intra- and inter-observer reliability also analysed. To evaluate Kinovea reliability and validity a multiple approach was applied involving the following analysis: -systematic error with a two-way ANOVA 2x4; -relative reliability with ICC and CV (95% confidence interval); -absolute reliability with Standard Error. The results thus obtained indicate that the Kinovea software is a valid and reliable tool that is able to measure accurately at distances up to 5 m from the object and at an angle range of 90°–45°. Nevertheless, for optimum results an angle of 90° is suggested.

Video quality assessment (VQA) plays an important role in video applications for quality evaluation and resource allocation. It aims to evaluate the video quality consistent with the human perception. In this letter, a hierarchical gradient similarity based VQA metric is proposed inspired by the structure of the primate visual cortex, in which visual information is processed through sequential visual areas. These areas are modeled with the corresponding measures to evaluate the overall perceptual quality. Experimental results on the LIVE database show that the proposed VQA metric significantly outperforms the state-of-the-art VQA metrics.

Depth-sensing technology has led to broad applications of inexpensive depth cameras that can capture human motion and scenes in 3D space. Background subtraction algorithms can be improved by fusing color and depth cues, thereby allowing many issues encountered in classical color segmentation to be solved. In this paper, we propose a new fusion method that combines depth and color information for foreground segmentation based on an advanced color-based algorithm. First, a background model and a depth model are developed. Then, based on these models, we propose a new updating strategy that can eliminate ghosting and black shadows almost completely. Extensive experiments have been performed to compare the proposed algorithm with other, conventional RGB-D algorithms. The experimental results suggest that our method extracts foregrounds with higher effectiveness and efficiency.