# Single variable optimization

January 1, 2011
By

(This article was first published on YGC » R, and kindly contributed to R-bloggers)

Optimization means to seek minima or maxima of a funtion within a given defined domain.

If a function reach its maxima or minima, the derivative at that point is approaching to 0. If we apply Newton-Raphson method for root finding to f’, we can get the optimizing f.

?View Code RSPLUS
 ```1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 ``` ```f2df <- function(fun) { fun.list = as.list(fun) var <- names(fun.list[1]) fun.exp = fun.list[[2]] diff.fun = D(fun.exp, var) df = list(x=0, diff.fun) df = as.function(df) return(df) }   newton <- function(fun, x0, tol=1e-7, niter=100) { df = f2df(fun) for (i in 1:niter) { x = x0 - fun(x0)/df(x0) if (abs(fun(x)) < tol) return(x) x0 = x } stop("exceeded allowed number of iterations") }   newton_optimize <- function(fun, x0, tol=1e-7, niter=100) { df <- f2df(fun) x = newton(df, x0, tol, niter) ddf <- f2df(df) if (ddf(x) > 0) { cat ("minima:\t", x, "\n") } else { cat ("maxima:\t", x, "\n") } return(x) }```

The golden-section method does not need f’. And it is similar to the root-bracketing technique for root finding.

?View Code RSPLUS
 ```1 2 3 4 5 6 7 8 9 10 11 12 ``` ```gSection <- function(f, x1, x2, x3, tol=1e-7) { r <- 2 - (1+sqrt(5))/2 x4 <- x2 + (x3-x2)*r if ( abs(x3-x1) < tol ){ return(x2) } if (f(x4) < f(x2)) { gSection(f, x2, x4, x3, tol) } else { gSection(f, x4, x2, x1, tol) } }```
```> f <- function(x) (x-1/3)^2
> newton_optimize(f, 0, tol=1e-7)
minima:  0.3333333
[1] 0.3333333
> gSection(f, 0,0.5,1)
[1] 0.3333333
> optimize(f, c(0,1), tol=1e-7)
\$minimum
[1] 0.3333333

\$objective
[1] 0
```

### Related Posts

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Tags: , , ,