Monterey Bay Aquarium Research Institute
Video Annotation and Reference System (VARS)
VARS System Overview

MBARI uses high-resolution video equipment to record more than 300 remotely operated vehicle (ROV) dives per year. In MBARI's 21 years of ROV diving (1989-2010), over 16,000 hours of videotapes have been archived, annotated, and maintained as a centralized institutional resource. This video library contains detailed footage of the biological, chemical, geological, and physical aspects of the Monterey Bay submarine canyon and other areas including the Pacific Northwest, Northern California, Hawaii, and the Gulf of California. MBARI has developed the Video Annotation and Reference System (VARS), consisting of software and hardware to facilitate the creation, storage, and retrieval of video annotation records from the ROV dive tapes.

As shown in the flowchart below, video images are collected by cameras on MBARI ROVs (remotely operated vehicles). The actual recordings are made in the shipboard control rooms. Using the VARS Annotation Interface, researchers take frame grabs and enter annotations during the dives. Descriptions are created using the extensive knowledge database. When a ship returns to shore, the tapes are reviewed and annotated in more detail by MBARI Video lab personnel, also using the annotation interface. All of this information goes into the annotation database, and a copy of the completely annotated tape is stored in the video library. At this point, the information is now accessible through the VARS Query interface.

Knowledge database

The core of the system is a knowledge database of over 4,000 biological, geological and technical terms used to describe deep-sea research conducted by the institute. This database is comprised of objects, which are identifiable things—for example, Atolla wyvillei jelly, fault, myctophid fish, gas vent, or suction sampler. The knowledge base also contains associations, which serve as descriptors (such as blue, large, juvenile, or active) that can be associated with the objects being analyzed. The taxonomic data can be viewed in alphabetic or hierarchical order and is accompanied by descriptive information that serves as a marine reference guide for system users. The hierarchical information allows for consistent, rapid classification and description and complex querying of objects observed on video.

The dedicated user interface is currently being modified, so it may not look exactly like the illustration. The interface allows science personnel to easily add, view, and edit information contained in the knowledge base. To ensure consistency, all amendments must be authorized by a knowledge base manager. Functioning as an encyclopedia, the knowledge base is used with the annotation application to characterize video observations in a consistent fashion.

Knowledge Base Sample (text form, in hierarchical order)

VARS Annotation interface

The annotation interface allows researchers on MBARI ships to make annotations of video, as it is being recorded by the ROV cameras. However, it is rarely possible to get all the details recorded in real-time. The view through the camera is constantly changing, and a variety of creatures (and other objects) appear and disappear with great frequency. Every tape is later viewed in the video lab, and the rest of the details are added to the annotation data base by video lab personel. These annotations are what makes up the MBARI knowledge base.

The interface will soon be available to researchers and institutions outside MBARI, and workshops for users are being planned. Instructions and documentation are presented on these pages for those who might be using the interface on a cruise or doing research in the MBARI video lab.

Documentation and instructions for VARS annotation interface

VARS Query interface

The VARS Query is a software tool, developed to enable easy access to information from MBARI's video annotation database. Complex queries can be made by constraining temporal, spatial, or physical parameters (for example: season, location, or depth). The VARS Query references the knowledge base (described above). Using the query tool, users can identify the location of video sequences or access other information recorded during ROV research dives.

Use VARS Query to access VARS data

Data characteristics disclaimer

Though they have proven to have great inherent and inferred scientific value, the data were not always collected using systematic search procedures. Biases have been introduced by observation techniques, camera settings, mission planning, changing science terminology, and observation location and sequencing, among other factors. In no way should these data be viewed or analyzed as random, structured, or systematic observations.


Last updated: Jan. 28, 2010