Summary of modifications made to Model I and Model II regressions

March 2016 — revision 4.0

It has now been 8+ years since any edits were made to these scripts: time to check to see if there are any problems or issues due to the many updates to the Matlab code.

  • Each of the 8 lsqfit scripts was checked for compatibility with Matlab 2014b and no errors were found.
  • However, some of the scripts had calculated variables which were not needed for the results. The code for these unused parameters was deleted from the scripts to improve computational efficiency.
  • All of the scripts were then error checked using the test data file and were found to produce the correct results.
  • All of the lsqfit script headers were revised with the new revision date & info.
  • The revised and updated scripts were then used to produce new graphs of the regression lines.

September 2007 — revision 3.1

I corrected an error in the y-intercept uncertainty calculation for the major axis algorithm:

  • The equation was initially derived correctly in Kermack & Haldane (1950), but they introduced a typo when the equation was reprinted in a table of summary formulae at the end of their text. York (1966) copied this typo from the summary formulae and I coded my equations using York’s notation. The algorithm now gives the correct uncertainty for the intercept in all four quadrants.
  • The Results Table was corrected as required. The single line data plots for the major axis calculations were corrected as well.
  • Minor typos in the header information for the lsqcubic were corrected and the revision date was updated.

April 2007 — revision 3

Additional minor changes were made to the lsqfit algorithms:

  • Minor typos in the header information in some of the files were corrected.
  • Revision dates on each file were updated.
  • Replaced GIF images of line plots with higher resolution JPG files.

January 2003 — revision 2

A minor change was made to the names of the Model I regressions:

  • Lsqfitxi.m was renamed lsqfitx.m to make it more consistent with the names of the other algorithms.
  • Minor changes were made to lsqbisec.m and lsqfitgm so that they are now compatible with this change.
  • The headers in each file were expanded to provide more information about each algorithm.

January 2000 — revision 1

The m-files for the Model I regressions were renamed to better reflect their applications:

  • Lsqfit1.m was renamed lsqfity.m since this regression minimizes the sum of the squares of the deviations in y.
  • Lsqfit1i.m was renamed lsqfitxi.m since this regression minimizes the sum of the squares of the deviations in x and is inverted so that the slope is directly comparable to the slopes from the other regression algorithms.
  • Lsqfit1w.m was renamed lsqfityw.m since this regression minimizes the weighted sum of the squares of the deviations in y.

At this time, it was also noticed that the standard deviations for the slope and intercept of the weighted model I regression, lsqfityw.m (formerly known as lsqfit1w.m), were considerably smaller than the standard deviations calculated by the other regression algorithms (see: results).

  • The source code was checked against the original and found to be correct.
  • The algorithm calculates the correct values according to the example given in Table 6.2 in Bevington and Robinson (1992).
  • Until this situation is resolved, an alternate calculation is presented: lsqfityz.m. This algorithm uses the same equations for slope and intercept as used by lsqfit1w.m and lsqfityw.m, but calculates the standard deviations for these terms based upon the more general equations derived by York (1966). The revised values are more in line with the standard deviations for these terms calculated by the other methods.

The algorithms for the Model II regressions were not changed although the header information for these files was revised for the purposes of clarity and completeness.

A new Model II regression algorithm was added, known here as the “least squares bisector:”lsqbisec.m.

  • Sprent and Dolby (1980) have objected to the ad hoc use of the geometric mean regression, because it is not a true unbiased expression for the functional relationship between X and Y.
  • They have suggested that an equally strong case can be made for using the line that bisects the minor angle between the two model I regressions.
  • While the difference in slope between the geometric mean regression and the “least squares bisector” are small and probably not statistically significant, this new “regression” line is included here for completeness.

And lastly, the test data file, data.txt, was edited and new results were calculated:

  • A simple plot of the original raw data revealed that it was not a bivariate normal distribution. It contained numerous outliers and data clusters. These have been removed to give a cleaner looking dataset.
  • The modified file is smaller, 110 points vs 139, and one can easily see the shape of the data ellipse.
  • Graphs of the regression lines are now available as well.


Data repository
Data policy
Deep-Sea Guide
What is happening in Monterey Bay today?
Central and Northern California Ocean Observing System
Chemical data
Ocean float data
Slough data
Mooring ISUS measurements
Southern Ocean Data
Mooring data
M1 Mooring Summary Data
M1 Asimet
M1 download Info
M1 EMeter
Molecular and genomics data
ESP Web Portal
Seafloor mapping
Soundscape Listening Room
Upper ocean data
Spatial Temporal Oceanographic Query System (STOQS) Data
Image gallery
Video library
Creature feature
Deep-sea wallpapers
Previous seminars
David Packard Distinguished Lecturers
Research software
Video Annotation and Reference System
System overview
Data Use Policy
Video Tape User Guide
Video File User Guide
Annotation Glossary
Query Interface
Basic User Guide
Advanced User Guide
Query Glossary
VARS publications
VARS datasets used in publications
Oceanographic Decision Support System
MB-System seafloor mapping software
How to download and install MB-System
MB-System Documentation
MB-System Announcements
MB-System Announcements (Archive)
MB-System FAQ
MB-System Discussion Lists
MB-System YouTube Tutorials
Matlab scripts: Linear regressions
Introduction to Model I and Model II linear regressions
A brief history of Model II regression analysis
Index of downloadable files
Summary of modifications
Regression rules of thumb
Results for Model I and Model II regressions
Graphs of the Model I and Model II regressions
Which regression: Model I or Model II?
Matlab scripts: Oceanographic calculations
Matlab scripts: Sound velocity
Visual Basic for Excel: Oceanographic calculations
Educational resources
Navigating STEM careers
MBARI Summer Internship Program
2017 Summer Interns Blog
Education and Research: Testing Hypotheses (EARTH)
EARTH workshops
2016—New Brunswick, NJ
2015—Newport, Oregon
2016 Satellite workshop—Pensacola, FL
2016 Satellite workshop—Beaufort, NC
EARTH resources
EARTH lesson plans
Lesson plans—published
Lesson plans—development
Lesson drafts—2015
Lesson drafts—2016 Pensacola
Adopt-A-Float Program
Center for Microbial Oceanography: Research and Education (C-MORE) Science Kits
Science at home: Curriculum and resources
Sample archive
SciComm Resources