Fitting a new class of nonlinear regression models (created 2010-).

This page is moving to a new website.

If you are trying to fit a new class of nonlinear regression models that you have never fit before, do a bit of homework before you use SAS or any other package.

First, look at the function from an algebraic viewpoint. When does it equal zero? What value do you get when you input a zero? What happens as the curve approaches plus/minus infinity? If you're comfortable with calculus, take a couple of derivatives and see if you can find minima, maxima, and/or inflection points. Many functions that are useful in nonlinear regression can be expressed as differential equations. See if you can find a simple differential equation that will produce your function. If you can, try to see if the differential equation offers a simple mechanistic explanation for how the function performs.

Here's a simple model for growth where the growth gradually slows down as it approaches a threshold. The formula is

Notice that for x=0, you get y=c-a. As x approaches infinity, y approaches c. The first and second derivates are

The first derivative is always positive, which tells you that the function is monotone increasing. That's a nice property for growth unless your data involves declines in growth, as might be the case in osteoporosis. The second derivative is always negative, so the function is concave. An always increasing but always concave function means that the growth is decelerating over time.

If you play with the formulas for y and y', you can get the following simple differential equation

This shows that the rate of increase in y is proportional to the amount of growth left to be achieved.

Here's another class of nonlinear regression models.

The results are messy for x=0, but for x=u, you get y = c+0.5d. As x approaches plus infinity, y approaches c (if b is positive) or d (if b is negative). As x approaches minus infinity, y approaches either d or c (again, depending on the sign of b). The first and second derivatives are messy.

The first derivative is always positive if b is positive or always negative if b is negative. So the value of b determines whether the function is monotone increasing or monotone decreasing. There is a point of inflection where the second derivative equals zero (at x=u). This represents the transition from a concave to a convex function or from a convex to a concave function, again depending on the sign of b.

Again, a bit of playing around with the formulas gives you

The rate of change is y is very slow if y is close to the minimum value (c) or the maximum value (c+d). The derivative is largest when y is halfway between c and c+d.

Here's a third example. Many studies involving the uptake and elimination of drugs will use a difference of two exponentials.

For t=0, y is also zero. As t approaches plus infinity, y approaches zero also.

The first two derivatives are

This function achieves a maximum at

and has a point of inflection at

Second, draw a bunch of graphs varying one parameter at a time. Keep the other parameters at "simple" values like 0 or 1.

[Show examples]

Then plot your data and try to fit a curve by hand. The hard part is getting the curve and the points to come somewhat remotely within the same range of one another. Once you get a curve that is good enough that you can see what the distances are between the data points and the curve, then slowly tweak the curve until you get something that looks like the best fit to the data.

The reason you do this is two fold. First, you need decent starting values in many nonlinear regression models and the exercise above will give you decent starting values. If you start with bad values, the model may never converge.

Second, the exercise gets you comfortable with understanding how changes in each parameter will change the degree of bend, or how they will shift or stretch the curve.