MBARI deploys video cameras on many scientific platforms. Much of this video is high quality, from High Definition cameras on ROVs and AUVs, and is used for both qualitative observations and quantitative measurements. The video is curated and annotated by trained professional staff, and the resulting annotations and measurements stored in databases. Other video and still imagery is collected using consumer- or professional-grade cameras for science or operations.

VARS: MBARI video is curated using the Video Annotation and Reference System (VARS), a software interface and database system that provides tools for describing, cataloging, retrieving and viewing the visual, descriptive, and quantitative data associated with MBARI’s deep-sea video archives. The VARS system software was made available to users outside of MBARI in November 2005. MBARI continually updates and releases improvements to VARS. Researchers and institutions can use VARS for cataloging and analyzing large or complex observation data sets. The software is available as open source code that can be adapted and improved for specific research needs.

Video analysis at MBARI: The database of deep-sea observations that have been recorded by MBARI remotely operated vehicles (ROVs) during 25+ years of operation, and associated software system, VARS, represents MBARI collective knowledge and serves as an investigative tool that facilitates deep-sea research publications as well as technical, agency, educational, and outreach projects. From this unique dataset, MBARI’s video annotation team has developed the Deep-Sea Guide (DSG). The DSG is a web-based system that allows for the correlation of visual, descriptive, and observational data with environmental data from multiple sources by providing tools for searching, identifying, and examining occurrence data (e.g., depth, time, abundance) for biological, geological, and experimental observations.

AVED: MBARI has developed an automated system for detecting marine organisms visible in the videos. Video frames are processed with a selective attention algorithm; candidate objects of interest are tracked across video frames. Objects tracked successfully over several frames are labeled as potentially “interesting” and marked in the video frames. The objective of the system is to enhance the productivity of human video annotators and/or cue a subsequent object classification module by marking candidate objects. As part of this system, MBARI is exploring state of the art object classification systems and machine learning, to enable automated video analysis.