The Shore Side Data System development involved requirements definition through 2001 and much of 2002. In 2002, however, developers began ad-hoc definition of a relational database schema which could accommodate all the different types of data MBARI might collect (and had been collecting).
A team of six developers (representing two decades of data management experience) worked on the schema part-time for approximately 18 months. During that period, they generated two major revisions and also participated in a multi-day consultation and training from experts at TogetherSoft (now Borland). For its first year, the schema was technically way ahead of the development on the core system, but this advantage was critical to establishing a clean, reusable core schema for tracking metadata and data sets.
Near the end of 2002, the team began its first effort to develop code. Kicking off with an "extreme week" of team programming, the developers collaborated for much of the month of November and released a working data management system (for the MOOS test mooring) in one month. Shortly after the mooring was deployed, a severe storm broke the mooring cable. Throughout the three days that the mooring was floating untethered, and after it was re-installed for further analysis on the MBARI campus, the mooring systems continued to collect and transmit data (via satellite) to the shore-based SSDS.
Another key milestone occurred in early 2003, when Brian Schlining took responsibility for migrating the design to make it suitable for AUV missions. Rather than providing streaming data, AUVs create data files from each sensor and then offload all the files from a mission when the AUV is recovered. This procedure naturally forced significant architectural changes. The system started successfully collecting AUV data that summer, a significant increase in scope and complexity. Domain experts and data management personnel are still refining the domain-specific and general-purpose processes to create an automatically post-processed data set from AUV data.
With the addition of data streams from the second MOOS test mooring (June 2004), the MOOS Lite (CIMT) mooring (June 2004), and the Benthic Instrument Node (connection scheduled later in 2004), the SSDS is handling about 500 megabytes of data (and post-processed data) from over 50 different sensors. The development process, and the design that resulted, are nearly past the feasibility stage and now must focus on operational products to satisfy many potential users.
Details of the development of SSDS may be found at the following links:
- The Design section describes the current architecture of the system. The information is limited at the moment, but more details will soon be available.
- The Methodology section provides more specific information about the development approach and technologies used.