Automated Video Event Detection

AVEDIn order to study the distribution and abundance of oceanic animals, MBARI uses high-resolution video equipment on remotely operated vehicles. Quantitative video transects (QVTs) supplant traditional net tows to assess the quantity and diversity of organisms in the water column. QVTs are run from 50 m to 4000 m and provide high-resolution data at the scale of the individual animals as well as their natural aggregation patterns. However, the current, manual method of analyzing QVTs is labor intensive and tedious.

We have developed an automated system for detecting marine organisms visible in the videos. Video frames are processed with a neuromorphic selective attention algorithm. The candidate objects of interest are tracked across video frames using linear Kalman filters. If objects can be tracked successfully over several frames, they are labeled as potentially “interesting” and marked in the video frames. The plan is that the system will enhance the productivity of human video annotators and/or cue a subsequent object classification module by marking candidate objects.

The continued use of ROVs and future use of Autonomous Underwater Vehicles (AUVs) for QVTs offer potential for even more data, perhaps many times what we current collect and analyze. Hence, we see tremendous benefit in automating portions of the analysis. We also see great benefit in automating analysis of video from fixed ocean observatory cameras, where autonomous response to potential events (pan/zoom to events), and automated processing of largely “boring” (event sparse) video streams from 10s or 100s or even 1000s of network cameras could be key to those cameras being useful practical scientific instruments.

Team

Technology

Solving challenges
Taking the laboratory into the ocean
In Situ Ultraviolet Spectrophotometer
Midwater Respirometer System
Mobile flow cytometer
Enabling targeted sampling
Automated Video Event Detection
Gulper autonomous underwater vehicle
Advancing a persistent presence
Aerostat hotspot
Benthic event detectors
Benthic rover
Fault Prognostication
Long-range autonomous underwater vehicle Tethys
Marine “soundscape” for passive acoustic monitoring
Monterey Ocean-Bottom Broadband Seismometer
Shark Café camera
Vehicle Persistence
Wave Glider-based communications hotspot
Emerging and current tools
Communications
Aerostat hotspot
Wave Glider-based communications hotspot
Data management
Oceanographic Decision Support System
Spatial Temporal Oceanographic Query System (STOQS) Data
Video Annotation and Reference System
Instruments
Apex profiling floats
Benthic event detectors
Deep particle image velocimetry
Environmental Sample Processor
How the ESP Works
Genomic sensors
ESP Web Portal
The ESP in the news
Investigations of imaging for midwater autonomous platforms
Lagrangian sediment traps
Laser Raman Spectroscopy
Midwater Respirometer System
Mobile flow cytometer
Smart underwater connector
Power
Wave-Power Buoy
Vehicle technology
Benthic Rover
Gulper autonomous underwater vehicle
Imaging autonomous underwater vehicle
In Situ Ultraviolet Spectrophotometer
Seafloor mapping AUV
Long-range autonomous underwater vehicle Tethys
Mini remotely operated vehicle
ROV Doc Ricketts
ROV Ventana
Video
Automated Video Event Detection
Deep learning
SeeStar Imaging System
Shark Café camera
Video Annotation and Reference System
Technology publications
Technology transfer