Monterey Bay Aquarium Research Institute's
Video Information Management System
Project Overview

The Monterey Bay Aquarium Research Institiute (MBARI) maintains a video library consisting of video footage dealing with biological, chemical, geological, and physical aspects of the Monterey Bay submarine canyon and surrounding coastal California waters. Researchers use these data to determine descriptive and quantitative attributes of the environment (visibility, particulate distribution, etc.) and of objects in the environment (organisms, geologic features, etc.). These 5000-plus hours of catalogued and stored video have proven to be a valuable resource for scientists and collaborating researchers. The video and related ancillary data information are used to document deep-sea exploration techniques and to aid in the descriptions of new organisms, geological features, and oceanographic experiments in Monterey Bay. Recognizing the tremendous intellectual value of the extensive collection of deep-sea video captured by the remotely-operated vehicles (ROVs) at MBARI, the Video Information Management System (VIMS) project has undertaken the task of delivering several applications to assist scientists with their video-related research needs.

The program VICKI (Video Information Capture with Knowledge Inferencing) was developed at MBARI with the Smalltalk language to meet specific video annotation requirements of our scientists and video lab staff. VICKI was designed to manually annotate MBARI video in a real-time environment, at sea on one of our research vessels, and in the laboratories using objects to describe the ROV video content (figure 1). A knowledge base enriched with the deep-sea organisms and geological features found in Monterey Bay provides the user with consistent hierarchical information about objects and associations for input into the database. Upon describing the video stream with VICKI, the user creates an annotation file containing a series of observations referenced with videotape timecodes. An ASCII output file is generated from the annotation file and input into the shore-based, VIMS relational database. In VIMS, these data are matched with CTDO and navigation data providing ancillary data such as temperature, salinity, oxygen, depth, vehicle position, and camera state values (focus, zoom, iris, and fieldwidth).

The VIMS Query Interface, developed at MBARI with object-oriented technologies and the computer language Java, provides scientists with an interface for retrieving annotation and ancillary data from VIMS (figure 2). Queries can be constrained by objects, dates and times, or with parameters from ancillary data; for example, a user can search on the occurrence of an organism between 200 and 400 meters. Expedition information such as research vessel, vehicle, and chief scientist can be returned with each annotation record in addition to ancillary data.

Scientists at MBARI and collaborative researchers use information from the VIMS database to investigate relationships between organisms—documenting the depths, spatial relationships, seasonal occurrences, seawater conditions, and diversity of organisms in Monterey Bay waters. Scientists also use this information to return to archived videotapes for observations of organisms, to view a geological feature, or locate deep-sea equipment. With ten years of archived video data, MBARI houses a valuable resource for a successful look into the deep sea.

Holly Baum, Nancy Jacobsen Stout, Kristine Rodgers-Walz,   http://www.mbari.org

Figure 1. VICKI Annotation Interface.

Figure 2. VIMS Query Interface, Constraints Page.