Machine learning

MBARI was founded by David Packard to be an alternative to the traditional academic oceanographic research institution. The mission of MBARI is to achieve and maintain a position as a world center for advanced research and education in ocean science and technology, and to do so through the development of better instruments, systems, and methods for scientific research in the deep waters of the ocean.

MBARI’s video assets

The Monterey Bay Aquarium Research Institute’s (MBARI) remotely operated vehicles (ROVs) have been collecting deep-sea video in the Monterey Bay region and other Pacific waters since 1988. To date, approximately 25,000 hours of video tape have been collected and professionally annotated using MBARI’s Video Annotation and Reference System (VARS). These annotations, currently totaling over 6.2 million, provide an established baseline for research and present a perspective into the biodiversity of the deep sea that no other data set can rival.

An ever-increasing number of camera deployments on MBARI platforms are collecting a variety of visual data at an astonishing rate. The expectation from scientists, management, board members, policy makers, and the public are that these videos should be instantly accessible and swiftly and accurately processed and analyzed for science, policy, management, and educational uses. Unfortunately, the technology, infrastructure, and practical tools available to the science community have not caught up with these expectations. This gap creates a daunting challenge.

Machine learning goals

The commercial proliferation of new imaging technology developments—from collection, storage, distribution, manipulation, and, of particular interest, automated detection, tracking, and classification of objects—continues to be integrated into the marine science community in innovative and effective ways. And yet, because the real-world, visual data collected for science differs so drastically from the imagery found in the popular culture (e.g. cats, cars, faces, shoes) that drives the most significant advances in artificial intelligence (AI), the potential for AI to enable groundbreaking exploration and discovery in science has not yet been fully realized.

At MBARI, significant progress has been made in generating data sets to drive machine-learning models from our existing, expertly curated video data with a minimal amount of additional processing. Our engineers also continue to adapt new deep-learning techniques for processing deep-sea video data. We have already demonstrated success classifying objects in still images. Our current focus has shifted towards improving video understanding: object tracking, classification and counting, measurement, behavior analyses, and habitat surveys. Our goal is to develop a suite of video analysis tools relevant to ocean sciences and conservation. We see unlimited potential for analyzed imagery in the ocean domain—from basic science exploration and discovery, time-series studies, marine protected area assessment and monitoring, fisheries and aquaculture management, subsea industrial process control and compliance testing, and more.

MBARI seeks to strategically partner with leaders in industry and academia in mutually beneficial ways. MBARI can offer unique data sets and challenging objectives that help advance AI technologies. In turn, these partnerships would leverage large teams of dedicated researchers, algorithms, storage, computing infrastructure, and distribution mechanisms in the commercial realm and have the potential to produce the sea change needed for processing extensive amounts of complex visual and associated environmental data critical for understanding the health of our ocean.