This morning during the lecture on nonlinear regression, I mentioned (very) briefly the case of convex regression. Since I forgot to mention the codes in R, I will publish them here. Assume that where is some convex function.
Then is convex if and only if , , Hidreth (1954) proved that ifthen is unique.
Let , then where. I.e. is the projection of onto the (closed) convex cone . The projection theorem gives existence and unicity.
For convenience, in the application, we will consider the real-valued case, , i.e. . Assume that observations are ordered . Here
Hence, quadratic program with linear constraints.
is a piecewise linear function (interpolation of consecutive pairs ).
If is differentiable, is convex if
More generally, if is convex, then there exists such that
is a subgradient of at . And then
Hence, is solution of and . Now, to do it for real, use cobs package for constrained (b)splines regression,
To get a convex regression, use
1 2 3 4 5
plot(cars) x = cars$speed y = cars$dist rc = conreg(x,y,convex=TRUE) lines(rc, col = 2)
Here we can get the values of the knots
1 2 3 4 5 6 7
rc Call: conreg(x = x, y = y, convex = TRUE) Convex regression: From 19 separated x-values, using 5 inner knots, 7, 8, 9, 20, 23. RSS = 1356; R^2 = 0.8766; needed (5,0) iterations
and actually, if we use them in a linear-spline regression, we get the same output here
1 2 3 4
reg = lm(dist~bs(speed,degree=1,knots=c(4,7,8,9,,20,23,25)),data=cars) u = seq(4,25,by=.1) v = predict(reg,newdata=data.frame(speed=u)) lines(u,v,col="green")
Let us add vertical lines for the knots