The MB-System
Cookbook

The MB-System™ Cookbook provides detailed examples regarding the processing of swath mapping data using the MB-System™ open-source software package. Step-by-step instructions, explanations, code (scripts), data files and processed results are provided for each example. The examples use real data files, with typical data irregularities.

MB-System™ supports many of the data formats and their variations that have evolved over the years. Accommodating these changes is a forever and ongoing process. The MB-System™ Cookbook is an attempt, in part, to document when data formats require special techniques in the processing.

The most up to date version of this document can be obtained from the MB-System™ web page at

http://www.ldeo.columbia.edu/MB-System/MB-System.intro.html

where it can be found in both Portable Document Format (PDF) and Hypertext Markup Language (HTML). Alternatively, it can be downloaded via anonymous ftp from the Lamont-Doherty Earth Observatory at:

ftp.ldeo.columbia.edu

MB-System™ is a collection tools used to process research grade swath mapping sonar data in more than four-dozen formats from sonar equipment manufactured and operated around the world. MB-System™ is used in conjunction with the Generic Mapping Tools (GMT) created by Paul Wessel of the University of Hawaii and Walter Smith of NOAA. GMT is a powerful set of processes used to manipulate data and to create Encapsulated Post Script maps and charts.

MB-System™ rides atop a library of functions for reading and writing swath mapping sonar data files called MBIO,(Multi Beam Input-Output).While largely unseen by the user, MBIO is where the rubber meets the road. MBIO takes into account multitudes of details, including existence of side scan or and or amplitude data, interpolation of navigation to ping times, geometry of specific sonar models, and of course the various data formats themselves. While MBIO does not support every possible data type, it has grown to accommodate the bulk of the sonar data formats common the the multibeam community. Indeed, considerable development is ongoing to support ever greater variations in sonar data formats, created both by commercial vendors and research institutions.

In addition to the native data formats, MBIO defines many MB-System™-only data formats. These have been created out of necessity when vendor-native formats fail to accommodate the needs required of sonar processing, or are too unwieldy to store. For example, some Simrad sonars have traditionally stored navigation information separate from the bathymetry information, requiring interpolation of navigation to ping times each time the data is read. MBIO defines a data format stores the bathymetry and the interpolated navigation in a single composite file. Similarly, native Hydrosweep DS-2 data contain no less than 9 separate data files with multitudes of ancillary information regarding the sonar's operational settings, far more information than the sonar data itself. In this case, MBIO defines an alternate format that condenses the necessary navigation, bathymetry and sidescan information into a single, more manageable and considerably smaller file. Many research institutions have found the MB-System™ formats preferable to the native sonar formats and perform a real time conversion of sonar data for their users.

Stating which sonar and sonar formats are supported by MB-System™ is something of a moving target, however as of the time of this writing the following sonars were supported:

In an effort to demonstrate MB-System™ in as realistic manner as possible we have provided several real datasets as examples. These are available in three separate archives that may be optionally download from the MB-System™ ftp site.

The standard MB-System™ distribution comes with an archive of example data sets and scripts generated using MB-System™ (MB-SystemExamples.X.Y.Z.tar.Z). These are referenced on occasion in the Cookbook, and since their size is small they can readily be downloaded. When uncompressed and un-archived, the resulting directory tree will resemble the lines below.

[vschmidt@val-LDEO mbexamples]$ ls 
data mbbath mbgrid mbinfo mblist mbm_plot README xbt

The ~/data directory contains several sample data files used in these examples. The other directories contain scripts that briefly demonstrate the use of several of the tools in MB-System™ .

For the purposes of the MB-System™ Cookbook, however these data sets were not sufficient to demonstrate the intricacies of various processing techniques, nor the art of managing a data set within a larger archive. Therefore, for the purposes of the examples contained herein, you will find two additional collections of data that are freely download-able such that you may follow along with each step we demonstrate.

The first collection of data has been added to the mbexamples.tgz archive file which is a superset of the standard set of examples described above. That is to say, this archive contains all the examples in the standard set, plus a collection of "other data sets" for illustration of svp corrections, roll bias corrections and general survey plotting. A directory called ~mbexamples/cookbook_examples has been created which will serve as a home directory for calculations utilizing these data sets.

These smaller data sets do not illustrate the processing and management of data within a larger archive however. To this end, we have provided a completely separate archive containing several data sets mapping the volcanic seamount Loihi, south of the Big Island of Hawaii. Within the loihi data archive you will find four data sub directories, each with their own Loihi data sets. The entire archive is some 660 MB in size.

The basic measurement of most sonar systems is the time it takes a sound pressure wave, emitted from some source, to reach a reflector and return. The observed travel time is converted to distance using an estimate of the speed of sound along the sound waves travel path. Estimating the speed of sound in water and the path a sound wave travels is a complex topic that is really beyond the scope of this cookbook. However, understanding the basics provides invaluable insight into sonar operation and the processing of sonar data, and so we have included an abbreviated discussion here.

2.1.1. Speed of Sound

The speed of sound in sea water is roughly 1500 meters/second. However the speed of sound can vary with changes in temperature, salinity and pressure. These variations can drastically change the way sound travels through water, as changes in sound speed between bodies of water, either vertically in the water column or horizontally across the ocean, cause the trajectory of sound waves to bend. Therefore, it is imperative to understand the environmental conditions in which sonar data is taken.

2.1.1.1. Affects of Temperature, Salinity and Pressure

The speed of sound increases with increases in temperature, salinity and pressure. Although the relationship is much more complex, to a first approximation, the following tables provides constant coefficients of change in sound speed for each of these parameters. Since changes in pressure in the sea typically result from changes in depth, values are provided with respect to changes in depth.

• Change in Speed of Sound per Change in Degree C

  ---> + 3 meters/sec)

• Change in Speed of Sound per Change in ppt Salinity

  ---> + 1.2 meters/sec

• Change in Speed of Sound per Change in 30 Meters of Depth

  ---> + .5 meters/sec)

^[1]

Temperature has the largest effect on the speed of sound in the open ocean. Temperature variations range from 28 F near the poles to 90 F or more at the Equator. Of course, relevant temperature differences with regard to multibeam sonar systems are the variations that occur over relatively short distances, in particular those that occur with depth. These are discussed further below.

While the salinity of the world's oceans varies from roughly 32 to 38 ppt. These changes are gradual, such that within the range of a multibeam sonar, the impact on the speed of sound in the ocean is negligible. However near land masses or bodies of sea ice, salinity values can change considerably and can have significant effects on the way sound travels through water.

While the change in the speed of sound for a given depth change is small, in depth excursions where temperature and salinity are relatively constant, pressure changes as a result of increasing depth becomes the dominating factor in changes in the speed of sound.

2.1.1.2. The Sound Speed Profile

The composite effects of temperature, salinity and pressure are measured in a vertical column of water to provide what is known as the "sound speed profile or SSP . A typical SSP is shown below:

Figure 2.1. Typical SSP

From the SSP above, one can see an iso-speed layer from the surface down to a few tens of meters due to mixing of water from wave action. This layer is called the "mixed surface layer" and is characterized by a flat or slightly negative (slanting down to the right) sound speed profile. The iso-speed layer is followed by a seasonal thermocline down to about 250 meters. Below, a larger main thermocline exists. These variations in the SSP are almost entirely due to changes in temperature of the water. Below the main thermocline, the temperature becomes largely constant and changes in pressure due to depth have the dominant effect on the SSP causing it to gradually increase.

2.1.2. The Effects of Sound Speed Errors

There are two fundamental sound speed measurement inputs into multibeam sonar systems. These are 1) the speed of sound at the keel of the ship in the vicinity of the sonar array, and 2) the profile of sound speed changes vertically in the water column. The former is used in the sonar's beam forming while the latter is used more directly in the the bathymetry calculations.

2.1.2.1. Sound Speed Errors at the Keel

In an effort to understand the effect on sonar performance of an incorrect sound speed at the keel, we must discuss the process of beam forming. This discussion surrounds a single simplified beam forming method, of which there are many. However it illustrates the effects well and the results can be applied to any sonar system.

The sound speed at the keel of the ship, local to the array, affects the directivity of the beams produced by the sonar. The result is that the sonar is not exactly looking in the direction we expect, introducing considerable error.

In multibeam sonar, a beam is formed by summing the sounds measured by multiple hydrophones time delayed by a specific amount with respect to each other. The following illustration depicts an array of hydrophones with an incident planar sound wave.

Figure 2.2. Diagram of Planar Soundwaves Orthogonally Incident on a Linear Hydrophone Array.

From this figure above, it is clear that the hydrophones will measure the incoming sounds simultaneously. However when the sound wave is incident from some angle , the hydrophones closer to the source will detect the sound prior to those farther away. Look at the figure below.

Figure 2.3. Diagram of Beam Forming Performed by A Linear Array

To listen in that direction, therefore, sonar systems sum the sounds measured from each hydrophone after delaying its measurement by the amount of time it would take for the sound to travel from the closest hydrophone. The result is an imaginary hydrophone array shown in the figure above, where the angle with respect to the actual array might be considered the direction the sonar beam is pointing. In this way, sounds coming from this direction are measured at each hydrophone in the imaginary array simultaneously.

As an example, assume the array is attempting to listen in a direction 30 degrees from its orientation. The sonar calculates the distance between the first hydrophone and the last hydrophone in the direction it is listening. This distance, X is simply

X = Z Sin (30)

The sonar divides the speed of sound at the keel into this distance X to get the time it will take the sound return to travel from the closer hydrophone to the further one.

t=X/ss

The measurements of the closer hydrophone are then summed with those of the further hydrophone delayed by this interval such that the arrival of the sound signal at the last hydrophone coincides with the delayed measurement of this closer hydrophone.

If the sound speed used to calculate this delay time is too low, the calculated delay time will be too large. A larger delay time results in a beam that is pointed further away from orthogonal than has been assumed.

If all this is confusing, just try to remember:

SS(Keel) Too Low - > Beam Fan Pattern Too Wide

SS(Keel) Too High - > Beam Fan Pattern Too Narrow

It is important to realize that the beam perpendicular to the plane of the sonar array is directionally unaffected. Only the beams formed by delaying the signals from individual hydrophones look in the wrong direction when the speed of sound at the keel is in error. This implies, of course, that the effect a sound speed error has on any given data set are largely dependent on the orientation of the installed sonar.

Many sonar arrays are installed flush with the hull of the ship and parallel to the sea (nominal) floor. A sonar array installed parallel to the sea floor sees little or no error in the nadir beam due to errors in the keel sound speed. However, because of the high beam angles at the outer beams, the errors are exacerbated at the edges of the swath. When the value is too low, the beam fan shape is erroneously wide, causing the measured bathymetry at the outer beams to be too deep. (Sound travel times are erroneously longer for the wider beams.) This causes the cross track shape of a swath to "frown". The converse is true, of course, for keel sound speeds that are too high.

Other sonar arrays are installed on a V-shaped structure mounted to the bottom of the ship or towfish. For these sonars, the zero angle beam, whose direction is unaffected by the keel sound speed, is that which is formed perpendicular to the array at some angle from nadir. The effect on the beam pattern, is the same - keel sound speed too low -> beam pattern is too wide - keel sound speed too high -> beam pattern too narrow. However, in this case, the effects on the data set are slightly different. A wider beam pattern does indeed cause erroneously deep bathymetry values at the beams furthest from the ship's track, but the beams directly beneath the ship will be erroneously shallow. Similarly, a narrower beam pattern causes erroneously shallow bathymetry values at beams furthest from the ship's track and erroneously deep values directly beneath the ship.

In both sonar installation orientations positive keel sound speed errors result in "frowns" in the cross track swath profile, while negative errors result in "smiles". The distinction to make is that the beams unaffected by these errors are beneath the ship for the former and at an angle orthogonal to the sonar array for the latter.

One final effect to point out. Multibeam sonars use beam forming both for projection as well as for reception of sound pulses. Moreover, beam forming is done, not only in the athwartships direction to create a swath, but fore and aft to increase measurement resolution. Typically the beam footprint created by the projectors on the sea floor is large compared to that created by the receivers. In this way, errors in the receive footprint still fall within that of the projection footprint. However significant sound speed errors at the keel can upset this balance lowering the signal to noise ratio of the sonar and compromising data quality.

Here's the kicker. ERRORS IN KEEL SOUND SPEED ARE (typically) NOT RECOVERABLE! Let us say that again:

Warning

ERRORS IN KEEL SOUND SPEED ARE NOT RECOVERABLE!

Because sonar systems do not save wave forms from individual hydrophones, one cannot go back and apply corrected sound speed values to beam forming calculations. This value must be correct the first time. Therefore, and it cannot be stressed enough, sonar operators must always be conscience of the correct operation of any device that provides the keel sound speed measurement to the sonar.

2.1.2.2. Sound Speed Profile Errors

Bathymetric sonar systems calculate water depths by measuring the time it takes a sound pulse to travel to the bottom and back to the receiver. To translate these time measurements into distances, one must know the speed at which sound travels through water and the general trajectory the sound traveled. As we have seen, temperature, pressure, and salinity all contribute to the speed of sound in water. Moreover, differences in sound speed across the water column acts as a lens bending the path that sound travels. For these reasons, it is imperative to have accurate sound speed profiles for any data set.

Inaccurate sound speed profiles may be the single largest correctable cause of bathymetry errors in multibeam sonar data. Understanding the various effects errors in sound speed can have on sonar data is challenging and deserves careful consideration. Even more difficult, is recognizing these errors from other sources of error in the data. A solid theoretical understanding is essential.

Errors in the sound speed profile produce predictable, although often confused, results in the data.

For example, a simple step offset in the sound speed profile will cause the calculated bathymetry to be shallower for higher sound speeds, and deeper for lower sound speeds. Not so obvious, is that relative changes in sound speed down through the water column cause sound to bend, a process called refraction. Therefore, errors in the relative changes in sound speed through the water column cause errors in the calculated sound trajectory. Because the bending is larger for sound traveling at oblique angles to the gradient, oblique beams (usually the outer ones) have more error. All of this is further complicated by the fact that sound speed profile errors high in the water column can exacerbate or offset the effects of those lower in the water column.

2.1.2.2.1. Refraction

It is convenient, when talking about sound and its travel path from a point, to consider the path as a ray. This is a common technique in wave theory, used in optics and other sciences. In a homogeneous medium sound does indeed travel in a straight line. However, when a sound wave passes between two mediums having different sound speeds its direction is bent. This is a property of waves more than sound itself, and many theories and explanations, with varying degrees of success, have been put forth over they years. While not true in all cases a simple one for illustration is offered here.

Figure 2.4. Snell's Law

Consider sound wave incident on a boundary between two bodies of water having differing sound speeds as shown in the figure above. Snell's Law states that the Cosine of the initial angle of incidence divided by the initial sound speed is a constant as the sound passes from regions of differing sound speeds.

Cos (A)/ Co = Constant

Therefore, if the new region has lower sound speed, the Cosine of the angle of incidence must be less than that of the previous region. Hence the new angle of incidence is less. So one might imagine the sound bending "toward" regions of lower sound speed, in the sense that the angle of sound travel is more steep, and "away" from regions of higher sound speed in the sense that the angle of incidence is more shallow.

Then if one knows the angle that a sound wave left its source, and the sound speed profile of the medium it passes through, one can calculate the path the wave takes over its entire trip.

A slightly more complicated example is one where the sound speed of a single medium changes linearly with depth rather than at discrete depth intervals. In this scenario, the sound ray bends continuously over its travel path. It can be shown that the path the sound travels is that of the arc of a circle whose radius, R, is the ratio of the initial sound speed and the slope of the gradient (R = Co/g).^[2]

This process, of calculating the course of travel of a sound ray through a water column of varying sound speed, is called "ray tracing" and is essential to sonar performance. Since sound does not travel as a straight line though the ocean, sonar systems must calculate propagation paths for both the the ping and return to calculate the correct distance and direction of the reflecting object.

2.1.2.2.2. SSP Errors and the Resulting Bathymetry

To be sure, the most likely source of sound speed profile errors is simply not measuring it frequently enough. Sound speed profiles change as bodies of waters change and whether, due to insufficiently frequent measurements or plain old inattention, inevitably sound speeds change before sonar operators notice.

But assuming your monitoring the sonar performance, and measurements have been carefully planned, we can consider the likely sources of error. Since in the open ocean temperature and pressure are the largest contributors to changes in sound speed, errors in the sensors aboard XBT and CTD sensors are of central concern. Over the relatively small temperature variation seen by these devices, temperature measurement response is typically quite linear. However inexpensive and non-calibrated sensors common on XBT's and CTD's often have step offset errors (the measurement will be off by a fraction of a degree or more over the entire range). Conversely, because pressure sensors must operate over such a large range (over to 2000 psi), they are much more prone to non-linearity (the measurement error will typically increase with depth).

Consider how these two errors would affect a sound speed profile. An constant temperature error will most significantly affect the portions of the sound speed profile in the thermocline with a proportionally smaller affect in the other regions. Conversely, a depth sensor non-linearity will most significantly affect the deep depth areas and have less proportional affect on the regions where changes in temperature dominate.

The amount of sound reflected from the sea floor is highly variable. It is dependent on the angle of incidence of the sound wave, (called the grazing angle), the smoothness of the sea floor, the sea floor composition, and the frequency of the sound.

Sound energy is well reflected when it bounces off a flat surface normal to the sound waves path of travel. However at an oblique angle, much of the sound is reflected at a complementary angle away from the receiver. Similarly rough surfaces tend to scatter the sound energy in directions away from the source. This generally dissipates the received sound level, but can enhance it when the angle of interception with the surface would otherwise reflect most of the sound energy away.

Some of the sound energy is lost into the sea floor itself. The amount sound energy will propagate into the sea floor is highly dependent on the frequency and bottom composition. For a typical bottom type and nominal source level, frequencies above 10kHz penetrate very little. From 1kHz to 10kHz sound often penetrates to several meters of depth. From 100 Hz to 1kHz sound can penetrate to several 10s of meters or more. Below 100 Hz sound waves have been detected traveling at various depths in the earth's crust around the globe.

Acoustic interference is somewhat loosely defined as any unwanted acoustic source that increases the sound levels detect by the sonar with respect to the bottom signal.

In the preceding sections we have talked qualitatively about "signal to noise ratio" (SNR), but we have not yet defined it in any formal way. While our goal is to provide only a cursory introduction to acoustics as it applies to sonar systems, it is instructive to look at the equation for SNR, if only briefly.

First we can define a bunch of terms, each of which are represented in decibels (dB):

SL = Source Level or the amount of sound energy that we "ping" into the water.

TS = Target Strength or the amount of sound energy that is reflected from an object (in our case the sea floor).

PL = Propagation Loss or the amount of energy lost to absorption and spreading of sound energy in the water column.

N = Noise or the amount of sound energy from other sources including reverberation, other ships, own ship, sea life, you name it. This term also includes electronic noise that is not "acoustic" in origination.

Armed with these definitions we can then write an equation for SNR, but before we do, we could review some quick math so it makes more sense. Remember that these values are expressed in dB, which is the log of the ratio of the sound intensity level to some standard level. Our "ratio" is a ratio of sound intensities, so when writing these as logarithms, to multiply the sound intensities we add the logarithms and to divide the sound intensities we subtract the logarithms. Now for the equation:

SNR = SL - PL + TS - PL - N

Reading left to right this equation makes perfect sense. Our signal starts with the source level from the transducer array (SL). The level is then reduced during propagation to the sea floor target (PL). Some amount of signal is reflected from the target (TS). This target signal is then decreased again by propagation loss back to the transducers (PL). The received signal is then reduced by are ability to discern it from the surrounding noise (N).

Written more compactly:

SNR = (SL + TS) - (2PL + N)

It is helpful to consider this equation when considering the possible sources of poor sonar performance and data quality.

Blackinton(199_) defines a useful terminology for describing swath mapping sonar systems that produce bathymetric data...

It is common when working within a survey, to want to plot the whole thing, and then take a closer look at a few particular points of interest. "Lets look at the data around this sea mount!" "How about a plot of yesterday's data." "What about a plot of the old data, with the new data." We can do all this with mbdatalist.

We will first use mbdatalist to create a master file list of all the files in the survey. This list will have proper relative references, will contain the MBIO format of the files, and will have an appropriate grid weighting factor.

We will then use mbdatalist to create three ancillary files for each data file. These ancillary files will greatly speed the processing of subsequent tasks. The three files are referred to as "info", "fast bathymetry" and "fast navigation." The info (".inf" suffix") contains meta-data and statistics about its parent data file. The information it contains is, in fact, identical to that created by mbinfo as we will see later. The fast bathymetry (.fbt suffix) and fast navigation (.fnv) files contain bathymetry and navigation data, respectively, in a format that can be read and processed more quickly than the original swath file format.

We can use mbdatalist to create a geographically windowed list of the files we are interested in.

Finally, we can make some cool, quick plots.

Perhaps we need an example.

In the ~/mbexamples/cookbook_examples/other_data_sets/ew0204survey/ directory you will find a collection of data files recorded with the Atlas Hydrosweep DS2 sonar aboard the R/V Ewing. These files will be used for the following examples.

First we need an initial list of the data files. This list needs to be in the directory that contains the data files themselves. This is done easily enough with the following:

cd ew0203survey/

ls -1 | grep mb183$ > tmplist 

Now that we have a list of the files in our data directory, we can use mbdatalist to create a properly formatted data list for our subsequent MB-System™ processing.

mbdatalist -F-1 -I tmplist > datalist-1

Let us take a moment to peek inside and see what mbdatalist has done for us.

00020504090010.mb183 183 1.000000
00020504091010.mb183 183 1.000000
00020504092010.mb183 183 1.000000
00020504093010.mb183 183 1.000000
00020504094010.mb183 183 1.000000
00020504095010.mb183 183 1.000000
00020504100010.mb183 183 1.000000
...

Here we see, for each file, the file name, the format, and a default grid weight of 1. Perfect!

With our new data list in hand, we can go ahead and create the ancillary data files. Using mbdatalist again:

mbdatalist -F-1 -I other_data_sets/ew0204survey/filelist.124 -N 

The data directory now looks like this:

00020504090010.mb183
00020504090010.mb183.fbt
00020504090010.mb183.fnv
00020504090010.mb183.inf
00020504091010.mb183
00020504091010.mb183.fbt
00020504091010.mb183.fnv
00020504091010.mb183.inf
00020504092010.mb183
00020504092010.mb183.fbt
00020504092010.mb183.fnv
00020504092010.mb183.inf
...

Now that we have the ancillary files created (particularly the statistics file which is used by mbdatalist for geographic windowing) we can create a new file list geographically windowed around a particular area. In this instance, I know from my exemplary notes during the cruise, that a small survey was taken of an area bounded by the following coordinates: (W/E/S/N) 170.133/170.35/42.2/42.4. The following line creates a list of data files within the geographic bounds of interest, with all the proper details required by other MB-System™ tools.

mbdatalist -F-1 -I datalist-1 -R170.133/170.35/42.2/42.4 \
 > survey-datalist

The resulting survey-datalist looks something like this:

00020504100010.mb183 183 1.000000
00020504101010.mb183 183 1.000000
00020504102010.mb183 183 1.000000
00020504103010.mb183 183 1.000000
00020504104010.mb183 183 1.000000
00020504105010.mb183 183 1.000000
....

As you can see, each data file is listed appropriately, each format type (183) is specified, and the default weighting factor of 1 is included as well.

Now lets make some plots.

Now that we've got a list of the data files of interest, we want to make some quick plots to see what we've got. These will not be the prettiest plots, as none of the data has been edited. None-the-less, MB-System™ will create nicely formatted plots, even if the data is a little unpolished.

First, perhaps we'd like to look at just the navigation data and see the ship track for this survey. To do that we call mbm_plot as shown below:

mbm_plot -F-1 -I survey-datalist -N

Here, -F-1 specifies the format, in this case "-1" indicates that the input is a list of files rather than an individual file and that the format for each file is specified in the list. Of course, -I survey-datalist is our list of data files. A navigation plot is specified with -N. With this simple line, and nothing more, we get a script that will create a navigation plot with default annotations, grid lines, tick marks, etc.. The results of executing the line above are shown below.

Plot generation shellscript <survey-datalist.cmd> created.

Instructions:
  Execute <survey-datalist.cmd> to generate Postscript plot 
<survey-datalist.ps>.
  Executing <survey-datalist.cmd> also invokes ghostview to
  view the plot on the screen.

A script was created called survey-datalist.cmd. When executed the script will create the navigation plot as a post script file and then execute ghostview to view the plot immediately. When we execute the resulting script the following plot is created:

Figure 3.1. Survey Navigation Plot

Wasn't that easy? mbm_plot sets default tick marks, grid lines, and longitude and latitude annotations as well as track annotations. It takes care of centering your plot onto a single page and even adds a title. All of these details, of course, can be changed though the myriad of options to mbm_plot. Have a look at the man page for the details.

Now let us take a look at the bathymetry data itself. We might start with a color plot of the bathymetry data. One specifies plotting of the bathymetry data by indicating a "graphics" mode utilizing the "-G" flag to mbm_plot. Five graphics modes are supported:

Then to create a color fill of the bathymetry data, we execute the following:

mbm_plot -F-1 -I survey_filelist.124 -G1

Here is the resulting plot.

Figure 3.2. Color Bathymetry Plot

Now perhaps we'd like to add some contours to our color plot, and maybe add the navigation back in as well. A quick contour plot with default parameters can be specified with the "-C" flag.

mbm_plot -F-1 -I survey_filelist.124 -G1 -N -C

The resulting plot is shown below:

Figure 3.3. Color Bathymetry Plot with Contours

What a nice plot of the data set! We can quickly see the lay of the sea floor, maximum and minimum contours, the ship track's coverage, and oddly shaped contours here and there already give hints as to where we we might need to spend special attention in our data editing.

Just for fun, lets create the same plot with a different color scheme. mbm_plot has a few predefined palettes that do not require us to create our own color map. These can be specified with the "-W" flag followed by the color style (continuous or discrete intervals), and optionally the palette (1-5) and number of colors(11 default).The following will create the same plot with mbm_plot's "high intensity" color palette.

mbm_plot -F-1 -I survey_filelist.124 -G1 -N -C -W1/2

Here's the resulting plot:

Figure 3.4. High Intensity Color Bathymetry Plot with Contours

mbm_plot can also quickly create shaded relief maps. These are specified with graphics mode "2" listed above, and by default produce a plot with artificial illumination from the north. The direction of illumination may be modified with the -A flag. See "COMPLETE DISCRIPTION OF OPTIONS" in the man page for more details.

mbm_plot -F-1 -I survey_filelist.124 -G2

Here's the resulting plot.

Figure 3.5. Shaded Relief Color Bathymetry Plot

Finally we might wish to see a plot of the side scan data. Sidescan plotting is also specified by a graphics mode, in this case mode "5".

mbm_plot -F-1 -I survey_filelist.124 -G5

Here is the plot:

Figure 3.6. Sidescan Plot

The plot above is not as nice as we might like. Much of shading is lost on erroneously high sidescan values which have incorrectly weighted the grayscales. The result is a fuzzy, washed out side scan image. This particular data set covers such a large range of side scan values (due to a change of several 1000's of meters in depth plus normal noise) that the side scan image looks quite poor. In this case, we will have to edit the data to remove the erroneous data spikes and then renormalize the gray scaling to get a good side scan image. We leave that to another chapter.

Now that we've explored the data sets graphically one might be interested to extract some statistics about the data set. This leads us to the next section.

MB-System™ provides mbinfo as a simple tool to extract statistics about a data set. Actually we've already used mbinfo and didn't know it. It was called in the background on our first call to mbdatalist to create the ".inf" files for each data file. However we could have executed mbinfo on each file individually as shown below.

mbinfo -F 183 -I 00020504090010.mb183

The results are sent to STDOUT by default rather than an ".inf" file. One can optionally redirect the results to a file of the same name as the original with ".inf" appended to the end with the -O flag. The resulting file will be automatically read and utilized by several of the other MB-System™ processes.

Here is the result of executing the mbinfo line above.

Swath Data File:      00020504090010.mb183
MBIO Data Format ID:  183
Format name:          MBF_HSDS2LAM
Informal Description: L-DEO HSDS2 processing format
Attributes:           STN Atlas multibeam sonars, 
                      Hydrosweep DS2, Hydrosweep MD, 
                      Fansweep 10, Fansweep 20, 
                      bathymetry, amplitude, and sidescan,
                      up to 1440 beams and 4096 pixels,
                      XDR binary, L-DEO.

Data Totals:
Number of Records:               37
Bathymetry Data (140 beams):
  Number of Beams:             5180
  Number of Good Beams:        5106     98.57%
  Number of Zero Beams:          74      1.43%
  Number of Flagged Beams:        0      0.00%
Amplitude Data (140 beams):
  Number of Beams:             5180
  Number of Good Beams:        5106     98.57%
  Number of Zero Beams:          74      1.43%
  Number of Flagged Beams:        0      0.00%
Sidescan Data (2180 pixels):
  Number of Pixels:           73815
  Number of Good Pixels:      55276     74.88%
  Number of Zero Pixels:      18539     25.12%
  Number of Flagged Pixels:       0      0.00%

Navigation Totals:
Total Time:             0.1593 hours
Total Track Length:     3.4493 km
Average Speed:         21.6521 km/hr (11.7038 knots)

Start of Data:
Time:  05 04 2002 08:59:56.460000  JD124
Lon:  169.8792     Lat:   42.1343     Depth:  5057.7613 meters
Speed: 23.1530 km/hr (12.5151 knots)  Heading:  46.1206 degrees
Sonar Depth:    5.8000 m  Sonar Altitude: 5051.9613 m

End of Data:
Time:  05 04 2002 09:09:29.968000  JD124
Lon:  169.9084     Lat:   42.1563     Depth:  4950.8452 meters
Speed: 21.0345 km/hr (11.3700 knots)  Heading:  43.9014 degrees
Sonar Depth:    5.1000 m  Sonar Altitude: 4945.7452 m

Limits:
Minimum Longitude:     169.8268   Maximum Longitude:     169.9605
Minimum Latitude:       42.0966   Maximum Latitude:       42.1970
Minimum Sonar Depth:     4.6000   Maximum Sonar Depth:     6.7000
Minimum Altitude:     4817.2718   Maximum Altitude:     5051.9613
Minimum Depth:        4770.9467   Maximum Depth:        5347.7265
Minimum Amplitude:       7.0000   Maximum Amplitude:     241.0000
Minimum Sidescan:        2.0000   Maximum Sidescan:      255.0000

We could also have executed mbinfo on a list of data files to see summary statistics for all of them. As an example we'll use our survey list created earlier:

[vschmidt@val-ldeo cookbook_examples]$ mbinfo -F-1 -I survey-datalist

And the results...

...
Data Totals:
Number of Records:             2412
Bathymetry Data (140 beams):
  Number of Beams:           337680
  Number of Good Beams:      329580     97.60%
  Number of Zero Beams:        8100      2.40%
  Number of Flagged Beams:        0      0.00%
Amplitude Data (140 beams):
  Number of Beams:           337680
  Number of Good Beams:      329580     97.60%
  Number of Zero Beams:        8100      2.40%
  Number of Flagged Beams:        0      0.00%
Sidescan Data (4094 pixels):
  Number of Pixels:         8746103
  Number of Good Pixels:    7680282     87.81%
  Number of Zero Pixels:    1065821     12.19%
  Number of Flagged Pixels:       0      0.00%

Navigation Totals:
Total Time:             7.0001 hours
Total Track Length:   128.0750 km
Average Speed:         18.2963 km/hr ( 9.8899 knots)

Start of Data:
Time:  05 04 2002 09:59:44.485000  JD124
Lon:  170.0626     Lat:   42.2726     Depth:  4508.8810 meters
Speed:  0.0000 km/hr ( 0.0000 knots)  Heading:  44.6704 degrees
Sonar Depth:    6.0000 m  Sonar Altitude: 4502.8810 m

End of Data:
Time:  05 04 2002 16:59:44.722000  JD124
Lon:  170.2718     Lat:   42.4209     Depth:  1446.6716 meters
Speed: 21.4741 km/hr (11.6076 knots)  Heading:  35.9473 degrees
Sonar Depth:    5.4000 m  Sonar Altitude: 1441.2716 m

Limits:
Minimum Longitude:     170.0040   Maximum Longitude:     170.3616
Minimum Latitude:       42.1619   Maximum Latitude:       42.4469
Minimum Sonar Depth:     3.6000   Maximum Sonar Depth:     7.9000
Minimum Altitude:      990.7620   Maximum Altitude:     4740.1799
Minimum Depth:         973.2481   Maximum Depth:        6223.7055
Minimum Amplitude:       0.0000   Maximum Amplitude:     250.0000
Minimum Sidescan:        1.0000   Maximum Sidescan:      255.0000

Here mbinfo has created summary statistics for the entire survey . We can see that the survey lasted some seven hours covering some 128 km at an average ship speed of about 10 knots. We can see the number of beams recorded, and had any processing been done on these data files, we would see the number of beams that had been flagged for removal. We can also see the data bounds, in latitude, longitude, minimum and maximum water depth, and start and stop times.

Now that we've seen how to quickly make some nice plots and extract some basic meta-data about our data files, we can turn to the topic of processing that data.

The first step in processing sonar data files with MB-System™ is to identify the type of sonar that was used to record the data and the format the data files are in. MB-System™ uses this information not only for successfully reading from and writing to the data files, but behind the scenes details regarding these sonars are used in reprocessing the data files.

Hopefully this information has been recorded by the science party when the data was taken and is readily available. However at times those with the task of post processing the data are not always the ones who participated in the cruise.

MB-System™ uses a standard set of libraries for reading sonar data files. MB-System™ version 5 supports the following sonar systems, among others:

MB-System™ supports several file formats unique to each sonar type - more than can be easily listed here. However they are all clearly delineated in the man page for MBIO allowing one to easily determine the MB-System™ format for the specified sonar and file type.

You can use the mbformat tool provided by MB-System™ to determine the sonar data Format ID. Each sonar writes data files with standard naming conventions. mbformat, typically uses these naming conventions to identify the sonar data format from which it looks up the numerical Format ID.

In some cases, the naming convention doesn't provide enough clues to determine the correct format ID. For example, Simrad recently changed their sonar data format, although their file naming convention is the same (*.raw). mbformat must open these files and begin to read the data to determine if the correct format ID is 51 - for the old format, or 56 - for the new format.

If the files are renamed with some naming conventions other than that provided by the sonar system or the standard MB-System™ convention, mbformat will not recognize them. In this case one must know details regarding the sonar system and sift through the MBIO man page for the appropriate format.

For example, suppose it is known, from notes provided with the data set, that the data was taken using the SeaBeam™ - 19 beam sonar on the R/V Thomas. I might further know that the data is bathymetry only (no sidescan) and is stored in binary format. I could then look in the man page for MBIO and see the following:

           ...
           MBIO Data Format ID:  16
           Format name:          MBF_SBSIOSWB
           Informal Description: SIO Swath-bathy SeaBeam
           Attributes:           Sea Beam, bathymetry, 19 beams,
                                 binary, centered, SIO.
           ...

There we have it - "MBIO Data Format ID: 16".

With the MB-System™ Format ID identified and the files copied to a manageable format, the next step is to take a look at the data. The things we want to accomplish are:

4.4.1. Integrating Your Data into a Larger Set

Often research institutions may maintain an single archive of several multibeam data sets from various cruises and various surveys. We might quickly consider how one might organize such an archive, to ease the task of processing and data retrieval.

4.4.1.1. The Method

Consider our example Lo'ihi data set. The top level directory of the Lo'ihi archive provided is entitled loihi. This is actually an unfortunate name on our part. Data sets should not be organized based on geographic location, as we shall explain. It might have been more appropriately entitled multibeam_archive. But for the time being we'll stick with loihi.

Organization below the top level directory can be quite varied, however should follow a single axiom - Data sets should not be organized by parameters that are easily managed by MB-System™. Two particular examples come to mind: time and space.

Managing data sets manually, by the time or location, is difficult, if not impossible to do. A common way to organize data sets is by cruise. (Alternatively, some institutions will organize data sets by principle investigator to allow administrative control over "ownership" and release of the data.) Cruises can be further broken down into segments, such that data is grouped in portions that require like processing or are simply in smaller chunks.

An example follows with segment directories shown for just one of the cruises.

archive/ewing_0101
archive/ewing_0101/segment_001
archive/ewing_0101/segment_002
archive/ewing_0101/segment_003
archive/ewing_0102
archive/ewing_0103
archive/knorr_0101
archive/knorr_0102
archive/knorr_0103
archive/knorr_0104
archive/thom_0101
archive/thom_0102
archive/thom_0103

The granularity of the file system within the archive will differ greatly from implementation to implementation, but the general idea is the same. With the hierarchy of directories and data files in place, we then create a series of recursive datalists within each directory.

Now consider our example data. Within our top archive directory, loihi we have subdirectories for each cruise. We might have broken these down further into segments if the data warranted, but in this case it did not. A diagram of the structure and a listing of the loihi directory are provided below.

Figure 4.1. Lo'ihi Archive Data Structure

>ls -1 loihi
datalist.mb-1
datalistp.mb-1
HUGO97
MBARI1998EM300
SumnerLoihi1997
TUNE04WT

Note

For the moment disregard the file datalistp.mb-1. This data list will be used to specify that processed files are to take precedence over their unprocessed originals. The utility of this file will be covered in detail later on.

In the loihi directory we create the archive data list - datalist.mb-1. This file is a text file containing the relative path and name of each of the data lists in the successive lower level subdirectories. With each list file name is a special MB-System™ Format ID ("-1") denoting that formats will be specified in the lower recursed data lists. Also included a data weighting factor or "grid weight". The contents of datalist.mb-1 are shown below:

>cat datalist.mb-1
HUGO97/datalist.mb-1 -1 100.0
MBARI1998EM300/datalist.mb-1 -1 1.0
SumnerLoihi1997/datalist.mb-1 -1 0.1
TUNE04WT/datalist.mb-1 -1 0.001

Before we continue, consider the weighting factors above. The grid weighting factor is a way to merge multiple data sets over the same geographic area and prioritize which data will be extracted for display. Larger weighting factors take priority over smaller ones. While occasionally mitigating factors affect individual sonar performance, usually the difference in sonar quality between sonar vendor and even by vendor models are quite significant. Therefore, weighting data sets by "sonar model" is often a good practice. One might further refine the weighting factor by sonar model and ship, since differences in ship size and construction can significantly affect sonar data results. Increments between weights should be large to allow further refinement without added hassle. Toward that end, some organizations weight their datasets logarithmically as we have done here.

If we now consider the lower level data lists we see the format is similar, except now, instead of referring to lower level data lists, we refer directly to the data files themselves. For example, MBARI1998EM300/datalist.mb-1 contains a list of the data files and their format:

cat MBARI1998EM300/datalist.mb-1
mbari_1998_53_msn.mb57 57
mbari_1998_54_msn.mb57 57
mbari_1998_55_msn.mb57 57
mbari_1998_56_msn.mb57 57
mbari_1998_57_msn.mb57 57
mbari_1998_58_msn.mb57 57
mbari_1998_59_msn.mb57 57
mbari_1998_60_msn.mb57 57
mbari_1998_61_msn.mb57 57

Here no grid weight is included, although we could have specified one if a further level of granularity was required.

Note

The default behavior is for any grid weight in a particular datalist entry to override values derived from higher levels in the recursive structure. This behavior can be reversed if the text $NOLOCALWEIGHT is inserted in the data list (or in a datalist higher up in the structure).

In our Lo'ihi archive, we have only two layers of subdirectories, however we could have more. Recursive data lists would be created in a similar way, referring down through the structure until the actual data files are referenced. Typically upper level data lists are maintained manually, as they are small and easy to manipulate. However, in the lowest levels, creating a data list for a directory of hundreds of files can be cumbersome. mbdatalist can be used to generate these lists.

As an example, we'll create another data list for the MBARI1998EM300 data directory. First we need a text file with a list of the data files in the directory. To do that execute the following:

ls -1 | grep mb57 > list

We can then use this list with mbdatalist to create a data list as follows:

mbdatalist -F-1 -Ilist
mbari_1998_53_msn.mb57 57 1.000000
mbari_1998_54_msn.mb57 57 1.000000
mbari_1998_55_msn.mb57 57 1.000000
mbari_1998_56_msn.mb57 57 1.000000
mbari_1998_57_msn.mb57 57 1.000000
mbari_1998_58_msn.mb57 57 1.000000
mbari_1998_59_msn.mb57 57 1.000000
mbari_1998_60_msn.mb57 57 1.000000
mbari_1998_61_msn.mb57 57 1.000000

The results of the above command are sent to STDOUT, however the could have been redirected to a text file named datalist.mb-1 and we would essentially have the list we looked at earlier. We have now added a default grid weight of 1. Remember, the weights assigned here take precedence over those assigned higher in the recursion structure.

Note

Note that mbdatalist does not currently provide the ability to specify a weighting factor other than the default. It is primarily a data list parsing tool as we shall see in future examples. One can either manually specify the weight at a higher level in the data list recursion structure, or adjust the default grid weight via a simple shell script. The following example changes the default value to 100.

 cat datalist_old | sed 's/1\.000000/100/' > datalist_new

Finally we should remove our interim list

rm list

4.4.1.2. The Benefit

So what is the benefit of having our data in this archive, with all these recursive data lists?

Suppose you were interested in the highest quality data you have covering Lo'ihi and the surrounding area. You can now quickly make a mosaic of the various data sets, simply by referring to the archive data list rather than sorting through a myriad of data files to decide the best quality data to then refer to the files directly. Where two data sets are collocated, the data with the highest grid weight will be extracted.

As we continue with data processing, we'll create post-processed data files adjacent to the raw data files in the archive. mbdatalist allows us to insert a flag ("$PROCESSED") into any data list causing the processed versions of the data in that list, and those beneath it in the recursion structure, to be extracted rather than the raw, unprocessed data. In this way we may selectively extract the processed or non-processed data files within our data structure. As an illustration, you'll find a secondary data list in the highest level directory called datalistp.mb-1 which contains only two entries - the $PROCESSED flag and the original archive data list datalist.mb-1.

Note

Alternatively, mbdatalist can be called with the -P flag which sets the $PROCESSED flag and overrides all others.

Similar to the discussion above, a "$RAW" flag in a data list or the -U flag on the command line can be specified to explicitly extract only the unprocessed data files, either by data list or universally as above.

Suppose we want to know the geographic bounds of the entire Lo'ihi archive. We simply run mbinfo and refer to the highest level data list to get statistics regarding the entire archive.

 mbinfo -F-1 -I datalist.mb-1
...

Data Totals:
Number of Records:           191764
Bathymetry Data (135 beams):
  Number of Beams:         12119082
  Number of Good Beams:     8745488     72.16%
  Number of Zero Beams:     1590395     13.12%
  Number of Flagged Beams:  1783199     14.71%
Amplitude Data (135 beams):
  Number of Beams:          7285751
  Number of Good Beams:     5519648     75.76%
  Number of Zero Beams:      608084      8.35%
  Number of Flagged Beams:  1158019     15.89%
Sidescan Data (1024 pixels):
  Number of Pixels:        10155008
  Number of Good Pixels:    4023419     39.62%
  Number of Zero Pixels:    6131589     60.38%
  Number of Flagged Pixels:       0      0.00%

Navigation Totals:
Total Time:         -49797.0534 hours
Total Track Length: 36092.8118 km
Average Speed:          0.0000 km/hr ( 0.0000 knots)

Start of Data:
Time:  06 21 1997 19:01:07.393000  JD172
Lon: -155.2214     Lat:   18.8823     Depth:   301.0800 meters
Speed:  1.9631 km/hr ( 1.0611 knots)  Heading: 229.1400 degrees
Sonar Depth:    0.3000 m  Sonar Altitude:  300.7800 m

End of Data:
Time:  10 16 1991 21:57:55.000000  JD289
Lon: -157.8782     Lat:   21.2853     Depth:    33.0000 meters
Speed: 20.5175 km/hr (11.0906 knots)  Heading:  31.2014 degrees
Sonar Depth:    0.0000 m  Sonar Altitude:   33.0000 m

Limits:
Minimum Longitude:    -157.8801   Maximum Longitude:    -153.3829
Minimum Latitude:       18.5684   Maximum Latitude:       21.2853
Minimum Sonar Depth:    -0.4100   Maximum Sonar Depth:  1530.3800
Minimum Altitude:      -92.9500   Maximum Altitude:     5482.8000
Minimum Depth:          33.0000   Maximum Depth:        5605.0000
Minimum Amplitude:     -60.0000   Maximum Amplitude:     255.0000
Minimum Sidescan:        0.1600   Maximum Sidescan:       90.5000

In these results we see that information regarding track length, speed, and duration are incorrect. mbinfo as they become meaningless when considering multiple overlapping data sets taken on different ships at different times. However the geographic and depth "Limits" are reported correctly as are the beam statistics. For example, here we see the archive has a minimum depth of 33 meters and maximum depth of 5604 meters.

Suppose you are interested in which data files contain data covering the top of Lo'ihi. We can use mbdatalist to create an extraction data list that covers only data files containing data within specified geographic bounds, for example:

mbdatalist -F-1 -I datalist.mb-1 -R-155.5/-155/18.7/19.2

results in the following data list containing with data within the bounds -155.5/-155/18.7/19.2. :

HUGO97/mba97172L05.mb121 121 100.000000
HUGO97/mba97174L01.mb121 121 100.000000
HUGO97/mba97174L02.mb121 121 100.000000
HUGO97/mba97174L04.mb121 121 100.000000
HUGO97/mba97174L05.mb121 121 100.000000
HUGO97/mba97174L06.mb121 121 100.000000
HUGO97/mba97174L07.mb121 121 100.000000
HUGO97/mba97175L01.mb121 121 100.000000
HUGO97/mba97175L03.mb121 121 100.000000
MBARI1998EM300/mbari_1998_53_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_54_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_55_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_56_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_57_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_58_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_59_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_60_msn.mb57 57 1.000000
MBARI1998EM300/mbari_1998_61_msn.mb57 57 1.000000
SumnerLoihi1997/61mba97043.d01 121 0.10000