Rules of thumb
My “rules-of-thumb” for choosing which regression to use are as follows:
For situations where the X-parameter is controlled, as in making-up standards for instrument calibration or doing laboratory experiments where only one variable is changed, then the standard model-I regressions are required.
- When all data points are given equal weight, use lsqfity.
- When the data points are individually weighted, use lsqfityw or lsqfityz.
- To decide between using lsqfityw.m or lsqfityz.m, see footnotes #2 and #3 to my Results for Model I and Model II Regressions Table.
When comparing two methods that measure the same quantity [e.g., DOC(per) vs DOC(htc)] then the major axis [lsqfitma] provides the most simple, easy to calculate and straight-forward solution.
- One word of caution: the slope of major axis is sensitive to changes in scale (see: Kermack and Haldane, 1950, for an explanation).
- If this is a concern (as in cases where the choice of measurement units is arbitrary), then the geometric mean regression [lsqfitgm] may be used.
For comparing two different measured parameters (like DOC vs TCO2, AOU, etc.) and especially when the parameters have different units or widely varying metrics, then the reduced major axis is the method of choice.
- The reduced major axis method is both difficult and tedious to compute, so I recommend that the geometric mean regression [lsqfitgm] be used in this case as this gives the identical line (see Ricker, 1973, for the derivation). I have used the symmetrical limits for a model I regression to estimate the uncertainty in the slope and intercept of the geometric mean regression following Ricker’s (1973) treatment.
- Sprent and Dolby (1980) have taken exception to the ad hoc use of the geometric mean regression in model II cases. They recommend the use of the least squares bisector [lsqbisec], the line that bisects the angle between the two model I regressions. Unfortunately, they did not present a statistical treatment for the estimation of the uncertainty limits for the least squares bisector slope, or intercept.
When the analytical methods for the different parameters have disparate uncertainties or experimental precision, then a weighted Model II method may be required.
- The method developed by York called “the least squares cubic” [lsqcubic] can be used in this case.
- This method allows the uncertainty in each X and Y measurement be specified in order to appropriately weight the data.