Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Disclaimer: I have no affiliation with The Next Web (cf. online article)

A few weeks ago I read this interesting and accessible article about explainable AI, discussing more specifically self-explainable AI issues. I’m not sure – anymore – if there’s a mandatory need for AI models that explain themselves, as there are model-agnostic tools such as the teller – among many others – for helping them in doing just that.

With that being said, the new `LSBoost` algorithm implemented in mlsauce does, explain itself. `LSBoost` is a cousin of the `LS_Boost` algorithm introduced in
GREEDY FUNCTION APPROXIMATION: A GRADIENT BOOSTING MACHINE (GFAGBM). GFAGBM’s `LS_Boost` is outlined below:

So, what makes the new `LSBoost` different? Would you be legitimately entitled to ask. Well, about the seemingly new name: I actually misspelled `LS_Boost` in my code in the first place! So, it’ll remain named as it is now and forever. Otherwise, in the new `LSBoost` we have:

• Page 1203, section 5 of GFAGBM is used: `LSBoost` contains a learning rate which could accelerate or slow down the convergence of residuals towards 0. Overfitting, fast or slow.
• Function h (referring to Algorithm 2 in GFAGBM) returns a columnwise concatenation of x and a – so called – neuron or node:

• a (referring to Algorithm 2 in GFAGBM) contains elements of a matrix of simulated uniform random numbers whose size can be controlled, in a randomized networks’ fashion.
• Both columns and rows of X (containing x’s) can be subsampled, in order to increase the diversity of the weak learners h fitting the successive residuals.
• Instead of optimizing least squares at line 4 of Algorithm 2, penalized least squares are used. Currently, ridge regression is implemented, and its bias has the effect of slowing down the convergence of residuals towards 0.
• An early stopping criterion is implemented, and is based on the magnitude of successive residuals.

Besides this, we can also remark that `LSBoost` is explainable as a linear model, while being a highly nonlinear one. Indeed by using some calculus, it’s possible to compute derivatives of F (still referring to Algorithm 2 outlined before) relative to x, wherever the function h does admit a derivative.

In the following Python+R examples appearing after the short survey (both tested on Linux and macOS so far), we’ll use `LSBoost` with default hyperparameters, for solving regression and classification problems. There’s still some room for improvement of models performance.

# I – Python version

## I – 0 – Install and import packages

Install mlsauce (command line)

``````

``````

Import packages

``````

import numpy as np
from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score
from time import time
from os import chdir
from sklearn import metrics

import mlsauce as ms

```
```

## I – 1 – Classification

### I – 1 – 1 Breast cancer dataset

``````

# data 1
X = breast_cancer.data
y = breast_cancer.target
# split data into training test and test set
np.random.seed(15029)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2)

print("dataset 1 -- breast cancer -----")

print(X.shape)
obj = ms.LSBoostClassifier()
# using default parameters
print(obj.get_params())

start = time()
obj.fit(X_train, y_train)
print(time()-start)
start = time()
print(obj.score(X_test, y_test))
print(time()-start)

# classification report
y_pred = obj.predict(X_test)
print(classification_report(y_test, y_pred))

```
```
``````
dataset 1 -- breast cancer -----

(569, 30)

{'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1}

0.16006875038146973
0.9473684210526315
0.015897750854492188

precision    recall  f1-score   support

0       1.00      0.86      0.92        42
1       0.92      1.00      0.96        72

accuracy                           0.95       114
macro avg       0.96      0.93      0.94       114
weighted avg       0.95      0.95      0.95       114

``````

### I – 1 – 2 Wine dataset

``````

# data 2
Z = wine.data
t = wine.target
np.random.seed(879423)
X_train, X_test, y_train, y_test = train_test_split(Z, t,
test_size=0.2)

print("dataset 2 -- wine -----")

print(Z.shape)
obj = ms.LSBoostClassifier()
# using default parameters
print(obj.get_params())

start = time()
obj.fit(X_train, y_train)
print(time()-start)
start = time()
print(obj.score(X_test, y_test))
print(time()-start)

# classification report
y_pred = obj.predict(X_test)
print(classification_report(y_test, y_pred))

```
```
``````
dataset 2 -- wine -----

(178, 13)

{'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1}

0.1548290252685547
0.9722222222222222
0.021778583526611328

precision    recall  f1-score   support

0       1.00      0.93      0.97        15
1       0.92      1.00      0.96        12
2       1.00      1.00      1.00         9

accuracy                           0.97        36
macro avg       0.97      0.98      0.98        36
weighted avg       0.97      0.97      0.97        36

``````

### I – 1 – 3 iris dataset

``````

# data 3
Z = iris.data
t = iris.target
np.random.seed(734563)
X_train, X_test, y_train, y_test = train_test_split(Z, t,
test_size=0.2)

print("dataset 3 -- iris -----")

print(Z.shape)
obj = ms.LSBoostClassifier()
# using default parameters
print(obj.get_params())

start = time()
obj.fit(X_train, y_train)
print(time()-start)
start = time()
print(obj.score(X_test, y_test))
print(time()-start)

# classification report
y_pred = obj.predict(X_test)
print(classification_report(y_test, y_pred))

```
```
``````
dataset 3 -- iris -----

(150, 4)

{'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1}

100%|██████████| 100/100 [00:00<00:00, 1157.03it/s]

0.0932917594909668
0.9666666666666667
0.007458209991455078

precision    recall  f1-score   support

0       1.00      1.00      1.00        13
1       1.00      0.90      0.95        10
2       0.88      1.00      0.93         7

accuracy                           0.97        30
macro avg       0.96      0.97      0.96        30
weighted avg       0.97      0.97      0.97        30

``````

## I – 2 – Regression

### I – 2 – 1 Boston dataset

``````

# data 1
X = boston.data
y = boston.target
# split data into training test and test set
np.random.seed(15029)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2)

print("dataset 4 -- boston -----")

print(X.shape)
obj = ms.LSBoostRegressor()
# using default parameters
print(obj.get_params())

start = time()
obj.fit(X_train, y_train)
print(time()-start)
start = time()
print(np.sqrt(np.mean(np.square(obj.predict(X_test) - y_test))))
print(time()-start)

```
```
``````
dataset 4 -- boston -----

(506, 13)

{'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1}

100%|██████████| 100/100 [00:00<00:00, 896.24it/s]
0%|          | 0/100 [00:00``````

### `I – 2 – 2 Diabetes dataset`

``````

# data 2
X = diabetes.data
y = diabetes.target
# split data into training test and test set
np.random.seed(15029)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2)

print("dataset 5 -- diabetes -----")

print(X.shape)
obj = ms.LSBoostRegressor()
# using default parameters
print(obj.get_params())

start = time()
obj.fit(X_train, y_train)
print(time()-start)
start = time()
print(np.sqrt(np.mean(np.square(obj.predict(X_test) - y_test))))
print(time()-start)

``````
``````
dataset 5 -- diabetes -----

(442, 10)

{'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1}

100%|██████████| 100/100 [00:00<00:00, 1000.60it/s]

0.10351037979125977
55.867989174555625
0.012843847274780273

``````

# `II – R version`

## `I – 0 – Install and import packages`

``````

library(devtools)
devtools::install_github("thierrymoudiki/mlsauce/R-package")
library(mlsauce)

``````

## `II – 1 – Classification`

``````

library(datasets)

X <- as.matrix(iris[, 1:4])
y <- as.integer(iris[, 5]) - 1L

n <- dim(X)[1]
p <- dim(X)[2]
set.seed(21341)
train_index <- sample(x = 1:n, size = floor(0.8*n), replace = TRUE)
test_index <- -train_index
X_train <- as.matrix(X[train_index, ])
y_train <- as.integer(y[train_index])
X_test <- as.matrix(X[test_index, ])
y_test <- as.integer(y[test_index])

# using default parameters
obj <- mlsauce::LSBoostClassifier()

start <- proc.time()[3]
obj\$fit(X_train, y_train)
print(proc.time()[3] - start)

start <- proc.time()[3]
print(obj\$score(X_test, y_test))
print(proc.time()[3] - start)

``````
``````elapsed
0.051
0.9253731
elapsed
0.011
``````

## `II – 2 – Regression`

``````

library(datasets)

X <- as.matrix(datasets::mtcars[, -1])
y <- as.integer(datasets::mtcars[, 1])

n <- dim(X)[1]
p <- dim(X)[2]
set.seed(21341)
train_index <- sample(x = 1:n, size = floor(0.8*n), replace = TRUE)
test_index <- -train_index
X_train <- as.matrix(X[train_index, ])
y_train <- as.double(y[train_index])
X_test <- as.matrix(X[test_index, ])
y_test <- as.double(y[test_index])

# using default parameters
obj <- mlsauce::LSBoostRegressor()

start <- proc.time()[3]
obj\$fit(X_train, y_train)
print(proc.time()[3] - start)

start <- proc.time()[3]
print(sqrt(mean((obj\$predict(X_test) - y_test)**2)))
print(proc.time()[3] - start)

``````
``````elapsed
0.044
6.482376
elapsed
0.01
``````
``` var vglnk = {key: '949efb41171ac6ec1bf7f206d57e90b8'}; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script')); RelatedShareTweetTo leave a comment for the author, please follow the link and comment on their blog: T. Moudiki's Webpage - R. R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.```
``` [{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/www.r-bloggers.com#Organization","name":"R-bloggers","url":"http:\/\/www.r-bloggers.com","sameAs":[],"logo":{"@type":"ImageObject","url":"http:\/\/www.r-bloggers.com\/wp-content\/uploads\/2020\/07\/R_blogger_logo_02.png","width":"1061","height":"304"},"contactPoint":{"@type":"ContactPoint","contactType":"technical support","telephone":"","url":"https:\/\/www.r-bloggers.com\/contact-us\/"}},{"@type":"WebSite","@id":"https:\/\/www.r-bloggers.com#website","headline":"R-bloggers","name":"R-bloggers","description":"R news and tutorials contributed by hundreds of R bloggers","url":"https:\/\/www.r-bloggers.com","potentialAction":{"@type":"SearchAction","target":"https:\/\/www.r-bloggers.com\/?s={search_term_string}","query-input":"required name=search_term_string"},"publisher":{"@id":"https:\/\/www.r-bloggers.com#Organization"}},{"@context":"https:\/\/schema.org","@type":"WebPage","@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#webpage","name":"LSBoost: Explainable &#039;AI&#039; using Gradient Boosted randomized networks (with examples in R and Pyt ...","url":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/","lastReviewed":"2020-07-23T18:00:00-06:00","reviewedBy":{"@type":"Organization","logo":{"@type":"ImageObject","url":"http:\/\/www.r-bloggers.com\/wp-content\/uploads\/2020\/07\/R_blogger_logo_02.png","width":"1061","height":"304"},"name":"R-bloggers"},"inLanguage":"en-US","description":"Explainable 'AI' with Gradient Boosted randomized networks.","primaryImageOfPage":{"@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#primaryimage"},"mainContentOfPage":[[{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"Home","url":"https:\/\/www.r-bloggers.com"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"About","url":"http:\/\/www.r-bloggers.com\/about\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"RSS","url":"https:\/\/feeds.feedburner.com\/RBloggers"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"add your blog!","url":"http:\/\/www.r-bloggers.com\/add-your-blog\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"Learn R","url":"https:\/\/www.r-bloggers.com\/how-to-learn-r-2\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"R jobs","url":"https:\/\/www.r-users.com\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"Submit a new job (it's free)","url":"https:\/\/www.r-users.com\/submit-job\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"Browse latest jobs (also free)","url":"https:\/\/www.r-users.com\/"},{"@context":"https:\/\/schema.org","@type":"SiteNavigationElement","@id":"https:\/\/www.r-bloggers.com\/#top nav","name":"Contact us","url":"http:\/\/www.r-bloggers.com\/contact-us\/"}]],"isPartOf":{"@id":"https:\/\/www.r-bloggers.com#website"},"breadcrumb":{"@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#breadcrumb"}},{"@type":"BreadcrumbList","@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"item":{"@id":"https:\/\/www.r-bloggers.com","name":"R-bloggers"}},{"@type":"ListItem","position":2,"item":{"@id":"https:\/\/www.r-bloggers.com\/category\/r-bloggers\/","name":"R bloggers"}},{"@type":"ListItem","position":3,"item":{"@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/","name":"LSBoost: Explainable &#039;AI&#039; using Gradient Boosted randomized networks (with examples in R and Pyt ..."}}]},{"@type":"Article","@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#article","url":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/","inLanguage":"en-US","mainEntityOfPage":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#webpage","headline":"LSBoost: Explainable &#039;AI&#039; using Gradient Boosted randomized networks (with examples in R and Pyt ...","description":"Explainable 'AI' with Gradient Boosted randomized networks.","articleBody":"Disclaimer: I have no affiliation with The Next Web (cf. online article) A few weeks ago I read this interesting and accessible article about explainable AI, discussing more specifically self-explainable AI issues. I\u2019m not sure \u2013 anymore \u2013 if there\u2019s a mandatory need for AI models that explain themselves, as there are model-agnostic tools such as the teller \u2013 among many others \u2013 for helping them in doing just that. With that being said, the new LSBoost algorithm implemented in mlsauce does, explain itself. LSBoost is a cousin of the LS_Boost algorithm introduced in GREEDY FUNCTION APPROXIMATION: A GRADIENT BOOSTING MACHINE (GFAGBM). GFAGBM\u2019s LS_Boost is outlined below: So, what makes the new LSBoost different? Would you be legitimately entitled to ask. Well, about the seemingly new name: I actually misspelled LS_Boost in my code in the first place! So, it\u2019ll remain named as it is now and forever. Otherwise, in the new LSBoost we have: Page 1203, section 5 of GFAGBM is used: LSBoost contains a learning rate which could accelerate or slow down the convergence of residuals towards 0. Overfitting, fast or slow. Function h (referring to Algorithm 2 in GFAGBM) returns a columnwise concatenation of x and a \u2013 so called \u2013 neuron or node: a (referring to Algorithm 2 in GFAGBM) contains elements of a matrix of simulated uniform random numbers whose size can be controlled, in a randomized networks\u2019 fashion. Both columns and rows of X (containing x\u2019s) can be subsampled, in order to increase the diversity of the weak learners h fitting the successive residuals. Instead of optimizing least squares at line 4 of Algorithm 2, penalized least squares are used. Currently, ridge regression is implemented, and its bias has the effect of slowing down the convergence of residuals towards 0. An early stopping criterion is implemented, and is based on the magnitude of successive residuals. Besides this, we can also remark that LSBoost is explainable as a linear model, while being a highly nonlinear one. Indeed by using some calculus, it\u2019s possible to compute derivatives of F (still referring to Algorithm 2 outlined before) relative to x, wherever the function h does admit a derivative. In the following Python+R examples appearing after the short survey (both tested on Linux and macOS so far), we\u2019ll use LSBoost with default hyperparameters, for solving regression and classification problems. There\u2019s still some room for improvement of models performance. Chargement\u2026 I - Python version I - 0 - Install and import packages Install mlsauce (command line) pip install mlsauce --upgrade Import packages import numpy as np from sklearn.datasets import load_boston, load_diabetes from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score from time import time from os import chdir from sklearn import metrics import mlsauce as ms I - 1 - Classification I - 1 - 1 Breast cancer dataset # data 1 breast_cancer load_breast_cancer() X breast_cancer.data y breast_cancer.target # split data into training test and test set np.random.seed(15029) X_train, X_test, y_train, y_test train_test_split(X, y, test_size0.2) print(\"dataset 1 -- breast cancer -----\") print(X.shape) obj ms.LSBoostClassifier() # using default parameters print(obj.get_params()) start time() obj.fit(X_train, y_train) print(time()-start) start time() print(obj.score(X_test, y_test)) print(time()-start) # classification report y_pred obj.predict(X_test) print(classification_report(y_test, y_pred))\t dataset 1 -- breast cancer ----- (569, 30) {'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1} 0.16006875038146973 0.9473684210526315 0.015897750854492188 precision recall f1-score support 0 1.00 0.86 0.92 42 1 0.92 1.00 0.96 72 accuracy 0.95 114 macro avg 0.96 0.93 0.94 114 weighted avg 0.95 0.95 0.95 114 I - 1 - 2 Wine dataset # data 2 wine load_wine() Z wine.data t wine.target np.random.seed(879423) X_train, X_test, y_train, y_test train_test_split(Z, t, test_size0.2) print(\"dataset 2 -- wine -----\") print(Z.shape) obj ms.LSBoostClassifier() # using default parameters print(obj.get_params()) start time() obj.fit(X_train, y_train) print(time()-start) start time() print(obj.score(X_test, y_test)) print(time()-start) # classification report y_pred obj.predict(X_test) print(classification_report(y_test, y_pred)) dataset 2 -- wine ----- (178, 13) {'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1} 0.1548290252685547 0.9722222222222222 0.021778583526611328 precision recall f1-score support 0 1.00 0.93 0.97 15 1 0.92 1.00 0.96 12 2 1.00 1.00 1.00 9 accuracy 0.97 36 macro avg 0.97 0.98 0.98 36 weighted avg 0.97 0.97 0.97 36 I - 1 - 3 iris dataset # data 3 iris load_iris() Z iris.data t iris.target np.random.seed(734563) X_train, X_test, y_train, y_test train_test_split(Z, t, test_size0.2) print(\"dataset 3 -- iris -----\") print(Z.shape) obj ms.LSBoostClassifier() # using default parameters print(obj.get_params()) start time() obj.fit(X_train, y_train) print(time()-start) start time() print(obj.score(X_test, y_test)) print(time()-start) # classification report y_pred obj.predict(X_test) print(classification_report(y_test, y_pred)) dataset 3 -- iris ----- (150, 4) {'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1} 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 100\/100 0.0932917594909668 0.9666666666666667 0.007458209991455078 precision recall f1-score support 0 1.00 1.00 1.00 13 1 1.00 0.90 0.95 10 2 0.88 1.00 0.93 7 accuracy 0.97 30 macro avg 0.96 0.97 0.96 30 weighted avg 0.97 0.97 0.97 30 I - 2 - Regression I - 2 - 1 Boston dataset # data 1 boston load_boston() X boston.data y boston.target # split data into training test and test set np.random.seed(15029) X_train, X_test, y_train, y_test train_test_split(X, y, test_size0.2) print(\"dataset 4 -- boston -----\") print(X.shape) obj ms.LSBoostRegressor() # using default parameters print(obj.get_params()) start time() obj.fit(X_train, y_train) print(time()-start) start time() print(np.sqrt(np.mean(np.square(obj.predict(X_test) - y_test)))) print(time()-start) dataset 4 -- boston ----- (506, 13) {'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1} 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 100\/100 0%| | 0\/100 0.1198277473449707 3.4934156173105206 0.01007080078125 I - 2 - 2 Diabetes dataset # data 2 diabetes load_diabetes() X diabetes.data y diabetes.target # split data into training test and test set np.random.seed(15029) X_train, X_test, y_train, y_test train_test_split(X, y, test_size0.2) print(\"dataset 5 -- diabetes -----\") print(X.shape) obj ms.LSBoostRegressor() # using default parameters print(obj.get_params()) start time() obj.fit(X_train, y_train) print(time()-start) start time() print(np.sqrt(np.mean(np.square(obj.predict(X_test) - y_test)))) print(time()-start) dataset 5 -- diabetes ----- (442, 10) {'backend': 'cpu', 'col_sample': 1, 'direct_link': 1, 'dropout': 0, 'learning_rate': 0.1, 'n_estimators': 100, 'n_hidden_features': 5, 'reg_lambda': 0.1, 'row_sample': 1, 'seed': 123, 'tolerance': 0.0001, 'verbose': 1} 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 100\/100 0.10351037979125977 55.867989174555625 0.012843847274780273 II - R version I - 0 - Install and import packages library(devtools) devtools::install_github(\"thierrymoudiki\/mlsauce\/R-package\") library(mlsauce)\t II - 1 - Classification library(datasets) X &lt;- as.matrix(iris) y &lt;- as.integer(iris) - 1L n &lt;- dim(X) p &lt;- dim(X) set.seed(21341) train_index &lt;- sample(x 1:n, size floor(0.8*n), replace TRUE) test_index &lt;- -train_index X_train &lt;- as.matrix(X) y_train &lt;- as.integer(y) X_test &lt;- as.matrix(X) y_test &lt;- as.integer(y) # using default parameters obj &lt;- mlsauce::LSBoostClassifier() start &lt;- proc.time() obj\$fit(X_train, y_train)\t print(proc.time() - start) start &lt;- proc.time() print(obj\$score(X_test, y_test)) print(proc.time() - start) elapsed 0.051 0.9253731 elapsed 0.011 II - 2 - Regression library(datasets) X &lt;- as.matrix(datasets::mtcars) y &lt;- as.integer(datasets::mtcars) n &lt;- dim(X) p &lt;- dim(X) set.seed(21341) train_index &lt;- sample(x 1:n, size floor(0.8*n), replace TRUE) test_index &lt;- -train_index X_train &lt;- as.matrix(X) y_train &lt;- as.double(y) X_test &lt;- as.matrix(X) y_test &lt;- as.double(y) # using default parameters obj &lt;- mlsauce::LSBoostRegressor() start &lt;- proc.time() obj\$fit(X_train, y_train) print(proc.time() - start) start &lt;- proc.time() print(sqrt(mean((obj\$predict(X_test) - y_test)**2))) print(proc.time() - start) elapsed 0.044 6.482376 elapsed 0.01","keywords":"","datePublished":"2020-07-23T18:00:00-06:00","dateModified":"2020-07-23T18:00:00-06:00","author":{"@type":"Person","name":"T. Moudiki","description":"","url":"https:\/\/www.r-bloggers.com\/author\/t-moudiki\/","sameAs":["https:\/\/thierrymoudiki.github.io\/"],"image":{"@type":"ImageObject","url":"https:\/\/secure.gravatar.com\/avatar\/b3f066e0c82b88e5148573f4d19f6601?s=96&d=mm&r=g","height":96,"width":96}},"publisher":{"@id":"https:\/\/www.r-bloggers.com#Organization"},"image":[{"@type":"ImageObject","url":"https:\/\/thierrymoudiki.github.io\/images\/2020-07-24\/2020-07-24-image1.png","width":996,"height":494,"@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#primaryimage"},{"@type":"ImageObject","url":"https:\/\/thierrymoudiki.github.io\/images\/2020-07-24\/2020-07-24-image2.png","width":972,"height":744}],"isPartOf":{"@id":"https:\/\/www.r-bloggers.com\/lsboost-explainable-ai-using-gradient-boosted-randomized-networks-with-examples-in-r-and-python\/#webpage"}}]}] var snp_f = []; var snp_hostname = new RegExp(location.host); var snp_http = new RegExp("^(http|https)://", "i"); var snp_cookie_prefix = ''; var snp_separate_cookies = false; var snp_ajax_url = 'https://www.r-bloggers.com/wp-admin/admin-ajax.php'; var snp_ajax_nonce = 'dff7afe6f2'; var snp_ignore_cookies = false; var snp_enable_analytics_events = false; var snp_enable_mobile = false; var snp_use_in_all = false; var snp_excluded_urls = []; snp_excluded_urls.push(''); Never miss an update! Subscribe to R-bloggers to receive e-mails with the latest R posts. (You will not see this message again.) Click here to close (This popup will not appear again).snp-pop-109583 .snp-theme6 { max-width: 700px;} .snp-pop-109583 .snp-theme6 h1 {font-size: 17px;} .snp-pop-109583 .snp-theme6 { color: #a0a4a9;} .snp-pop-109583 .snp-theme6 .snp-field ::-webkit-input-placeholder { color: #a0a4a9;} .snp-pop-109583 .snp-theme6 .snp-field :-moz-placeholder { color: #a0a4a9;} .snp-pop-109583 .snp-theme6 .snp-field :-ms-input-placeholder { color: #a0a4a9;} .snp-pop-109583 .snp-theme6 .snp-field input { border: 1px solid #a0a4a9;} .snp-pop-109583 .snp-theme6 .snp-field { color: #000000;} .snp-pop-109583 .snp-theme6 { background: #f2f2f2;}jQuery(document).ready(function() { }); var CaptchaCallback = function() { jQuery('.g-recaptcha').each(function(index, el) { grecaptcha.render(el, { 'sitekey' : '' }); }); }; window.FPConfig= { delay: 0, ignoreKeywords: ["\/wp-admin","\/wp-login.php","\/cart","add-to-cart","logout","#","?",".png",".jpeg",".jpg",".gif",".svg"], maxRPS: 3, hoverDelay: 50 }; _stq = window._stq || []; _stq.push([ 'view', {v:'ext',j:'1:7.3.2',blog:'11524731',post:'200206',tz:'-6',srv:'www.r-bloggers.com'} ]); _stq.push([ 'clickTrackerInit', '11524731', '200206' ]); jQuery(document).ready(function (\$) { //\$( document ).ajaxStart(function() { //}); for (var i = 0; i < document.forms.length; ++i) { var form = document.forms[i]; if (\$(form).attr("method") != "get") { \$(form).append('<input type="hidden" name="iLk_OoNZATdV" value="W84S_c1w*Ka7" />'); } if (\$(form).attr("method") != "get") { \$(form).append('<input type="hidden" name="pYbxk_z" value="0eJkL7w" />'); } if (\$(form).attr("method") != "get") { \$(form).append('<input type="hidden" name="xXGyZbrfKpq" value="gGW7HJCYOhl" />'); } } \$(document).on('submit', 'form', function () { if (\$(this).attr("method") != "get") { \$(this).append('<input type="hidden" name="iLk_OoNZATdV" value="W84S_c1w*Ka7" />'); } if (\$(this).attr("method") != "get") { \$(this).append('<input type="hidden" name="pYbxk_z" value="0eJkL7w" />'); } if (\$(this).attr("method") != "get") { \$(this).append('<input type="hidden" name="xXGyZbrfKpq" value="gGW7HJCYOhl" />'); } return true; }); jQuery.ajaxSetup({ beforeSend: function (e, data) { //console.log(Object.getOwnPropertyNames(data).sort()); //console.log(data.type); if (data.type !== 'POST') return; if (typeof data.data === 'object' && data.data !== null) { data.data.append("iLk_OoNZATdV", "W84S_c1w*Ka7"); data.data.append("pYbxk_z", "0eJkL7w"); data.data.append("xXGyZbrfKpq", "gGW7HJCYOhl"); } else { data.data = data.data + '&iLk_OoNZATdV=W84S_c1w*Ka7&pYbxk_z=0eJkL7w&xXGyZbrfKpq=gGW7HJCYOhl'; } } }); }); ```