Results for Model I and Model II regressions

Here are the results from the MATLAB® shell-script routines for fitting a line to the example datasets from Bevington and Robinson (2003):

Model I
Data File M-file Slope Intercept R
Y-on-X [1] BR3tab61 Lsqfity.m 0.02622 ± 0.00034 0.07139 ± 0.01917 0.99941
X-on-Y [1] BR3tab61 Lsqfitx.m 0.02625 ± 0.00034 0.06984 ± 0.02000 0.99941
Weighted Y-on-X [1,2] BR3tab62 Lsqfityw.m 30.6979 ± 1.0341 119.4963 ± 7.5676 0.99385
Weighted Y-on-X [1,3] BR3tab62 Lsqfityz.m 30.6979 ± 1.2096 119.4963 ± 8.8522 0.99385

Here are the results from my MATLAB® shell-script routines for fitting a line to the example dataset (data.txt) using the various linear regression models:

Model I
M-file Slope Intercept R
Y-on-X [1] Lsqfity.m    1.2931 ± 0.0598    9.3670 ± 1.4750 0.9014
X-on-Y [1] Lsqfitx.m 1.5916 ± 0.0736 3.0908 ± 1.8945 0.9014
Weighted Y-on-X [1,2] Lsqfityw.m 1.3322 ± 0.0064 4.4067 ± 0.0738 0.8973
Weighted Y-on-X [1,3] Lsqfityz.m 1.3322 ± 0.0630 4.4067 ± 0.7220 0.8973
Model II
M-file Slope Intercept R
Major axis Lsqfitma.m 1.4896 ± 0.0682 5.2356 ± 1.6455 0.9014
Geometric mean Lsqfitgm.m 1.4346 ± 0.0613 6.3917 ± 1.5128 0.9014
Least-squares-bisector [4] Lsqbisec.m 1.4320 ± 0.0613 6.4477 ± 1.5114 0.9014
Least-squares-cubic Lsqcubic.m 1.4948 ± 0.0649 5.2594 ± 0.6227 0.9583

Footnotes to table:

  1. All parameters were calculated with equations from Bevington and Robinson (1992).
  2. The standard deviations for the slope and intercept of the weighted regression (lsqfityw.m) seem small when compared to the uncertainties calculated with the other regression equations.
  3. Revised weighted model I regression: equations for the slope and intercept were again taken from Bevington and Robinson (1992), however, the standard deviations in these parameters are calculated according to equations derived by York (1966). The uncertainties calculated this way seem more in line with those calculated by the other regressions.
  4. The least-squares-bisector algorithm follows the suggestion of Sprent and Dolby (1980) that in the case of model II regressions, an equally strong case can be made for the line that bisects the minor angle between the two model I regressions: Y-on-X and X-on-Y. The uncertainties in the slope and intercept are calculated according to the equations derived by York (1966) in a manner analogous to that used for the geometric mean regression.

Also, please note that:

  • The answers in the table have not been rounded-off to the proper number of significant figures. Instead, I have choosen to list the results obtained with Matlab in standard precision mode so that one can compare these results to those from other programs without an undue emphasis on data truncation and round-off errors.
  • Equations for the Y-on-X, X-on-Y, major axis, geometric mean and least-squares-bisector regressions all run through the centroid: (mean-x,mean-y).
  • Likewise, the correlation coefficients for the Y-on-X, X-on-Y, major axis and geometric mean regressions are all the same. Thus, the correlation coefficient is properly interpreted as a measure of the linearity of the data and not how well the line fits the data.
  • For the weighted linear regressions and the least squares cubic the correlation coefficients are different owing to the varying weights applied to each data point.


  • Bevington & Robinson (1992). Data Reduction and Error Analysis for the Physical Sciences, Second Edition, McGraw-Hill, Inc., New York.
  • Sprent and Dolby (1980). The geometric mean functional relationship. Biometrics 36: 547-550.
  • York (1966). Least-squares fitting of a straight line. Canad. J. Phys. 44: 1079-1086.
  • Bevington & Robinson (2003). Data Reduction and Error Analysis for the Physical Sciences, Third Edition, McGraw-Hill, Inc., New York.


Data repository
Data policy
What is happening in Monterey Bay today?
Central and Northern California Ocean Observing System
Chemical data
Ocean float data
Slough data
Mooring ISUS measurements
M1 ISUS CTD Data Display
Southern Ocean Data
Mooring data
M1 Mooring Summary Data
M1 Asimet
M1 Download Info
M1 EMeter
M1 Flourometer (CeNCOOS)
M1 GPS Location
Molecular and genomics data
ESP Web Portal
Seafloor mapping
Upper ocean data
Spatial Temporal Oceanographic Query System (STOQS) Data
Tide prediction
Image gallery
Video library
Previous seminars
David Packard Distinguished Lecturers
Research software
Video Annotation and Reference System
System Overview
Annotation Interface
Video Tape User Guide
Video File User Guide
Still Images User Guide
Annotation Glossary
Query Interface
Basic User Guide
Advanced User Guide
Query Glossary
VARS Publications
Oceanographic Decision Support System
MB-System seafloor mapping software
MB-System Documentation
MB-System Announcements
MB-System Announcements (Archive)
How to Download and Install MB-System
MB-System Discussion Lists
MB-System FAQ
Matlab scripts: Linear regressions
Introduction to Model I and Model II linear regressions
A brief history of Model II regression analysis
Index of downloadable files
Summary of modifications
Regression rules of thumb
Results for Model I and Model II regressions
Graphs of the Model I and Model II regressions
Which regression: Model I or Model II?
Matlab scripts: Oceanographic calculations
Matlab scripts: Sound velocity
Visual Basic for Excel: Oceanographic calculations
Educational resources
MBARI Summer Internship Program
Education and Research: Testing Hypotheses (EARTH)
EARTH workshops
2016—New Brunswick, NJ
2015—Newport, Oregon
2016 Satellite workshop—Pensacola, FL
2016 Satellite workshop—Beaufort, NC
EARTH resources
EARTH lesson plans
Lesson plans—published
Lesson plans—development
Lesson drafts—2015
Lesson drafts—2016 Pensacola
Center for Microbial Oceanography: Research and Education (C-MORE) Science Kits
Sample archive