Per the NIST guidelines, for the purposes of this report, an estimation result is deemed to pass the test if all of the coefficient and standard error results match the NIST results to at least 4.0 significant digits.
For the coefficient values, RATS passes all tests but one, where it falls just short.
For several of the tests, the standard errors results fail to meet the 4 significant digit criteria, although by a small amount in most cases.
For one test, both results (barely) fail to meet the 4 significant digit criteria.
The table below reports the minimum number of significant digits matched for each set of parameters.
| Passed | |||
| Dataset | Difficulty | Coeff. | S.E. |
| Lanczos3 | Lower | 6.58 | 5.68 |
| Gauss1 | Lower | 6.15 | 4.08 |
| Gauss2 | Lower | 7.37 | 4.06 |
| DanWood | Lower | 8.73 | 6.38 |
| Kirby2* | Average | 7.18 | 5.24 |
| Hahn1* | Average | 6.42 | 4.41 |
| Lanczos2 | Average | 7.34 | 5.67 |
| Gauss3 | Average | 7.14 | 4.06 |
| Roszman1 | Average | 6.71 | 6.74 |
| ENSO | Average | 5.91 | 6.48 |
| MGH09 | Higher | 6.57 | 5.34 |
| Thurber | Higher | 5.76 | 4.72 |
| BoxBOD | Higher | 7.56 | 6.19 |
| Rat42 | Higher | 6.59 | 4.61 |
| MGH10** | Higher | 7.10 | 6.15 |
| Eckerle4 | Higher | 8.57 | 6.61 |
| Rat43 | Higher | 6.45 | 4.75 |
| Min | 5.76 | 4.06 | |
| Average | 6.95 | 5.36 | |
| Passed for Coeffs, Failed for Std. Errors | |||
| Dataset | Difficulty | Coeff. | S.E. |
| Misra1a | Lower | 5.59 | 2.76 |
| Chwirut1 | Lower | 6.52 | 3.82 |
| Chwirut2 | Lower | 7.04 | 3.82 |
| Misra1b | Lower | 5.45 | 2.63 |
| Nelson | Average | 5.52 | 3.58 |
| MGH17*** | Average | 5.08 | 3.80 |
| Lanczos1 | Average | 10.56 | 3.90 |
| Misra1c | Average | 5.27 | 2.39 |
| Misra1d | Average | 5.37 | 2.54 |
| Min | 5.08 | 2.39 | |
| Average | 6.27 | 3.25 | |
| Failed | |||
| Dataset | Difficulty | Coeff. | S.E. |
| Bennett5 | Higher | 3.66 | 3.47 |
* For Hahn1, RATS warns of non-convergence for Start 1 and Start 2 values. A simple rescaling of the problem produces good fit. For Kirby2, the original model converges, but a similar rescaling produces much better results.
** RATS warns of non-convergence with Start 1 values. Results using Start 2 values shown.
*** Converges to a local optimum with Start 1 values. Results for Start 2 shown
The nonlinear regression programs, data files, and output files are available in RatsNistNonlin.Zip
Return to main NIST benchmark page