# Boosting Win Probability accuracy with player embeddings

**R – Giga thoughts …**, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

In my previous post Computing Win Probability of T20 matches I had discussed various approaches on computing Win Probability of T20 matches. I had created ML models with glmnet and random forest using TidyModels. This was what I had achieved

- glmnet : accuracy – 0.67 and sensitivity/specificity – 0.68/0.65
- random forest : accuracy – 0.737 and roc_auc- 0.834
- DL model with Keras in Python : accuracy – 0.73

I wanted to see if the performance of the models could be further improved. I got a suggestion from a AI/DL whizkid, who is close to me, to include embeddings for batsmen and bowlers. He felt that win percentage is influenced by which batsman faces which bowler.

So, I started to explore this idea. Embeddings can be used to convert categorical variables to a vector of continuous floating point numbers.Fortunately R’s Tidymodels, has a convenient functionality to create embeddings. By including embeddings for batsman, bowler the performance of my ML models improved vastly. Now the performance is

- glmnet : accuracy – 0.728 and roc_auc – 0.81
- random forest : accuracy – 0.927 and roc_auc – 0.98
- mlp-dnn :accuracy – 0.762 and roc_auc – 0.854

As can be seem there is almost a 20% increase in accuracy with random forests with embeddings over the model without embeddings. Moreover, the feature importance which is plotted below shows that the bowler and batsman embeddings have a significant influence on the Win Probability

**Note**: The data for this analysis is taken from Cricsheet and has been processed with my R package yorkr.

A. **Win Probability using GLM with penalty and player embeddings**

Here Generalised Linear Model (GLMNET) for Logistic Regression is used. In the GLMNET the regularisation path is computed for the lasso or elastic net penalty at a grid of values for the regularisation parameter lambda. glmnet is extremely fast and gave an accuracy of 0.72 for an roc_auc of 0.81 with batsman, bowler embeddings. This was good improvement over my earlier implementation with glmnet without the batsman & bowler embeddings which had a

- Read the data

a) Read the data from 9 T20 leagues (BBL, CPL, IPL, NTB, PSL, SSM, T20 Men, T20 Women, WBB) and create a single data frame of ball-by-ball data. Display the data frame

library(dplyr) library(caret) library(e1071) library(ggplot2) library(tidymodels) library(embed) # Helper packages library(readr) # for importing data library(vip) df1=read.csv("output3/matchesBBL3.csv") df2=read.csv("output3/matchesCPL3.csv") df3=read.csv("output3/matchesIPL3.csv") df4=read.csv("output3/matchesNTB3.csv") df5=read.csv("output3/matchesPSL3.csv") df6=read.csv("output3/matchesSSM3.csv") df7=read.csv("output3/matchesT20M3.csv") df8=read.csv("output3/matchesT20W3.csv") df9=read.csv("output3/matchesWBB3.csv") #Bind all dataframes together df=rbind(df1,df2,df3,df4,df5,df6,df7,df8,df9) glimpse(df) Rows: 1,199,115 Columns: 10 $ batsman <chr> "JD Smith", "M Klinger", "M Klinger", "M Klinger", "JD … $ bowler <chr> "NM Hauritz", "NM Hauritz", "NM Hauritz", "NM Hauritz",… $ ballNum <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, … $ ballsRemaining <int> 125, 124, 123, 122, 121, 120, 119, 118, 117, 116, 115, … $ runs <int> 1, 1, 2, 3, 3, 3, 4, 4, 5, 5, 6, 7, 13, 14, 16, 18, 18,… $ runRate <dbl> 1.0000000, 0.5000000, 0.6666667, 0.7500000, 0.6000000, … $ numWickets <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1… $ runsMomentum <dbl> 0.08800000, 0.08870968, 0.08943089, 0.09016393, 0.09090… $ perfIndex <dbl> 11.000000, 5.500000, 7.333333, 8.250000, 6.600000, 5.50… $ isWinner <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0… df %>% count(isWinner) %>% mutate(prop = n/sum(n)) isWinner n prop 1 0 614237 0.5122419 2 1 584878 0.4877581

2) Create training.validation and test sets

b) Split to training, validation and test sets. The dataset is initially split into training and test in the ratio 80%:20%. The training data is again split into training and validation in the ratio 80:20

set.seed(123) splits <- initial_split(df,prop = 0.80) splits <Training/Testing/Total> <959292/239823/1199115> df_other <- training(splits) df_test <- testing(splits) set.seed(234) val_set <- validation_split(df_other,prop = 0.80) val_set # A tibble: 1 × 2 splits id <list> <chr> 1 <split [767433/191859]> validation

3) Create pre-processing recipe

a) Normalise the following predictors

- ballNum
- ballsRemaining
- runs
- runRate
- numWickets
- runsMomentum
- perfIndex

b) Create floating point embeddings for

- batsman
- bowler

4) Create a Logistic Regression Workflow by adding the GLM model and the recipe

5) Create grid of elastic penalty values for regularisation

6) Train all 30 models

7) Plot the ROC of the model against the penalty

# Use all 12 cores cores <- parallel::detectCores() cores # Create a Logistic Regression model with penalty lr_mod <- logistic_reg(penalty = tune(), mixture = 1) %>% set_engine("glmnet",num.threads = cores) # Create pre-processing recipe lr_recipe <- recipe(isWinner ~ ., data = df_other) %>% step_embed(batsman,bowler, outcome = vars(isWinner)) %>% step_normalize(ballNum,ballsRemaining,runs,runRate,numWickets,runsMomentum,perfIndex) # Set the workflow by adding the GLM model with the recipe lr_workflow <- workflow() %>% add_model(lr_mod) %>% add_recipe(lr_recipe) # Create a grid for the elastic net penalty lr_reg_grid <- tibble(penalty = 10^seq(-4, -1, length.out = 30)) lr_reg_grid %>% top_n(-5) # A tibble: 5 × 1 penalty <dbl> 1 0.0001 2 0.000127 3 0.000161 4 0.000204 5 0.000259 lr_reg_grid %>% top_n(5) # highest penalty values # A tibble: 5 × 1 penalty <dbl> 1 0.0386 2 0.0489 3 0.0621 4 0.0788 5 0.1 # Train 30 penalized models lr_res <- lr_workflow %>% tune_grid(val_set, grid = lr_reg_grid, control = control_grid(save_pred = TRUE), metrics = metric_set(accuracy,roc_auc)) # Plot the penalty versus ROC lr_plot <- lr_res %>% collect_metrics() %>% ggplot(aes(x = penalty, y = mean)) + geom_point() + geom_line() + ylab("Area under the ROC Curve") + scale_x_log10(labels = scales::label_number()) lr_plot

The Penalty vs ROC plot is shown below

8) Display the ROC_AUC of the top models with the penalty

9) Select the model with the best ROC_AUC and the associated penalty. It can be seen the best mean ROC_AUC is 0.81 and the associated penalty is 0.000530

top_models <- lr_res %>% show_best("roc_auc", n = 15) %>% arrange(penalty) top_models # A tibble: 15 × 7 penalty .metric .estimator mean n std_err .config <dbl> <chr> <chr> <dbl> <int> <dbl> <chr> 1 0.0001 roc_auc binary 0.810 1 NA Preprocessor1_Model01 2 0.000127 roc_auc binary 0.810 1 NA Preprocessor1_Model02 3 0.000161 roc_auc binary 0.810 1 NA Preprocessor1_Model03 4 0.000204 roc_auc binary 0.810 1 NA Preprocessor1_Model04 5 0.000259 roc_auc binary 0.810 1 NA Preprocessor1_Model05 6 0.000329 roc_auc binary 0.810 1 NA Preprocessor1_Model06 7 0.000418 roc_auc binary 0.810 1 NA Preprocessor1_Model07 8 0.000530 roc_auc binary 0.810 1 NA Preprocessor1_Model08 9 0.000672 roc_auc binary 0.810 1 NA Preprocessor1_Model09 10 0.000853 roc_auc binary 0.810 1 NA Preprocessor1_Model10 11 0.00108 roc_auc binary 0.810 1 NA Preprocessor1_Model11 12 0.00137 roc_auc binary 0.810 1 NA Preprocessor1_Model12 13 0.00174 roc_auc binary 0.809 1 NA Preprocessor1_Model13 14 0.00221 roc_auc binary 0.809 1 NA Preprocessor1_Model14 15 0.00281 roc_auc binary 0.809 1 NA Preprocessor1_Model15 #Picking the best model and the corresponding penalty lr_best <- lr_res %>% collect_metrics() %>% arrange(penalty) %>% slice(8) lr_best # A tibble: 1 × 7 penalty .metric .estimator mean n std_err .config <dbl> <chr> <chr> <dbl> <int> <dbl> <chr> 1 0.000530 roc_auc binary 0.810 1 NA Preprocessor1_Model08 # Collect predictions and generate the AUC curve lr_auc <- lr_res %>% collect_predictions(parameters = lr_best) %>% roc_curve(isWinner, .pred_0) %>% mutate(model = "Logistic Regression") autoplot(lr_auc)

7) Plot the Area under the Curve (AUC).

10) Build the final model with the best LR parameters value as found in lr_best

a) The best performance was for a penalty of 0.000530

b) The accuracy achieved is 0.72. Clearly using the embeddings for batsman, bowlers improves on the performance of the GLM model without the embeddings. The accuracy achieved was 0.72 whereas previously it was 0.67 see (Computing Win Probability of T20 Matches)

c) Create a fit with the best parameters

d) The accuracy is 72.8% and the ROC_AUC is 0.813

# Create a model with the penalty for best ROC_AUC last_lr_mod <- logistic_reg(penalty = 0.000530, mixture = 1) %>% set_engine("glmnet",num.threads = cores,importance = "impurity") #Update the workflow with this model last_lr_workflow <- lr_workflow %>% update_model(last_lr_mod) #Create a fit set.seed(345) last_lr_fit <- last_lr_workflow %>% last_fit(splits) #Generate accuracy, roc_auc last_lr_fit %>% collect_metrics() # A tibble: 2 × 4 .metric .estimator .estimate .config <chr> <chr> <dbl> <chr> 1 accuracy binary 0.728 Preprocessor1_Model1 2 roc_auc binary 0.813 Preprocessor1_Model1

11) Plot the feature importance

It can be seen that bowler and batsman embeddings are the most significant for the prediction followed by runRate.

runRate –

- runRate in 1st innings

- requiredRunRate in 2nd innings

12) Plot the ROC characteristics

last_lr_fit %>% collect_predictions() %>% roc_curve(isWinner, .pred_0) %>% autoplot()

13) Generate a confusion matrix

14) Create a final Generalised Linear Model for Logistic Regression with the penalty of 0.000530

15) Save the model

# generate predictions from the test set test_predictions <- last_lr_fit %>% collect_predictions() test_predictions # generate a confusion matrix test_predictions %>% conf_mat(truth = isWinner, estimate = .pred_class) Truth Prediction 0 1 0 90105 32658 1 32572 84488 final_lr_model <- fit(last_lr_workflow, df_other) final_lr_model obj_size(final_lr_model) 146.51 MB butcher::weigh(final_lr_model) A tibble: 305 × 2 object size <chr> <dbl> 1 pre.actions.recipe.recipe.steps.terms1 57.9 2 pre.actions.recipe.recipe.steps.terms2 57.9 3 pre.actions.recipe.recipe.steps.terms3 57.9 cleaned_lm <- butcher::axe_env(final_lr_model, verbose = TRUE) #✔ Memory released: "1.04 kB" #✔ Memory released: "1.62 kB" saveRDS(cleaned_lm, "cleanedLR.rds")

16) Compute Ball-by-ball Win Probability

- Chennai Super Kings-Lucknow Super Giants-2022-03-31

16a) The corresponding Worm-wicket graph for this match is as below

- Chennai Super Kings-Lucknow Super Giants-2022-03-31

**B) Win Probability using Random Forest with player embeddings**

In the 2nd approach I use Random Forest with batsman and bowler embeddings. The performance of the model with embeddings is quantum jump from the earlier performance without embeddings. However, the random forest is also computationally intensive.

1) Read the data

a) Read the data from 9 T20 leagues (BBL, CPL, IPL, NTB, PSL, SSM, T20 Men, T20 Women, WBB) and create a single data frame of ball-by-ball data. Display the data frame

2) Create training.validation and test sets

b) Split to training, validation and test sets. The dataset is initially split into training and test in the ratio 80%:20%. The training data is again split into training and validation in the ratio 80:20

library(dplyr) library(caret) library(e1071) library(ggplot2) library(tidymodels) library(tidymodels) library(embed) # Helper packages library(readr) # for importing data library(vip) library(ranger) # Read all the 9 T20 leagues df1=read.csv("output3/matchesBBL3.csv") df2=read.csv("output3/matchesCPL3.csv") df3=read.csv("output3/matchesIPL3.csv") df4=read.csv("output3/matchesNTB3.csv") df5=read.csv("output3/matchesPSL3.csv") df6=read.csv("output3/matchesSSM3.csv") df7=read.csv("output3/matchesT20M3.csv") df8=read.csv("output3/matchesT20W3.csv") df9=read.csv("output3/matchesWBB3.csv") # Bind into a single dataframe df=rbind(df1,df2,df3,df4,df5,df6,df7,df8,df9) set.seed(123) df$isWinner = as.factor(df$isWinner) #Split data into training, validation and test sets splits <- initial_split(df,prop = 0.80) df_other <- training(splits) df_test <- testing(splits) set.seed(234) val_set <- validation_split(df_other, prop = 0.80) val_set

2) Create a Random Forest model tuning for number of predictor nodes at each decision node (mtry) and minimum number of predictor nodes (min_n)

3) Use the ranger engine and set up for classification

4) Set up the recipe and include batsman and bowler embeddings

5) Create a workflow and add the recipe and the random forest model with the tuning parameters

# Use all 12 cores parallely cores <- parallel::detectCores() cores [1] 12 # Create the random forest model with mtry and min as tuning parameters rf_mod <- rand_forest(mtry = tune(), min_n = tune(), trees = 1000) %>% set_engine("ranger", num.threads = cores) %>% set_mode("classification") # Setup the recipe with batsman and bowler embeddings rf_recipe <- recipe(isWinner ~ ., data = df_other) %>% step_embed(batsman,bowler, outcome = vars(isWinner)) # Create the random forest workflow rf_workflow <- workflow() %>% add_model(rf_mod) %>% add_recipe(rf_recipe) rf_mod # show what will be tuned extract_parameter_set_dials(rf_mod) set.seed(345) # specify which values meant to tune # Build the model rf_res <- rf_workflow %>% tune_grid(val_set, grid = 10, control = control_grid(save_pred = TRUE), metrics = metric_set(accuracy,roc_auc)) # Pick the best roc_auc and the associated tuning parameters rf_res %>% show_best(metric = "roc_auc") # A tibble: 5 × 8 mtry min_n .metric .estimator mean n std_err .config <int> <int> <chr> <chr> <dbl> <int> <dbl> <chr> 1 4 4 roc_auc binary 0.980 1 NA Preprocessor1_Model08 2 9 8 roc_auc binary 0.979 1 NA Preprocessor1_Model03 3 8 16 roc_auc binary 0.974 1 NA Preprocessor1_Model10 4 7 22 roc_auc binary 0.969 1 NA Preprocessor1_Model09 5 5 19 roc_auc binary 0.969 1 NA Preprocessor1_Model06 rf_res %>% show_best(metric = "accuracy") # A tibble: 5 × 8 mtry min_n .metric .estimator mean n std_err .config <int> <int> <chr> <chr> <dbl> <int> <dbl> <chr> 1 4 4 accuracy binary 0.927 1 NA Preprocessor1_Model08 2 9 8 accuracy binary 0.926 1 NA Preprocessor1_Model03 3 8 16 accuracy binary 0.915 1 NA Preprocessor1_Model10 4 7 22 accuracy binary 0.906 1 NA Preprocessor1_Model09 5 5 19 accuracy binary 0.904 1 NA Preprocessor1_Model0

6) Select all models with the best roc_auc. It can be seen that the best roc_auc is 0.980 for mtry=4 and min_n=4

7) Get the model with the highest accuracy. The highest accuracy achieved is 0.927 or 92.7. This accuracy is also for mtry=4 and min_n=4

# Pick the best roc_auc and the associated tuning parameters rf_res %>% show_best(metric = "roc_auc") # A tibble: 5 × 8 mtry min_n .metric .estimator mean n std_err .config <int> <int> <chr> <chr> <dbl> <int> <dbl> <chr> 1 4 4 roc_auc binary 0.980 1 NA Preprocessor1_Model08 2 9 8 roc_auc binary 0.979 1 NA Preprocessor1_Model03 3 8 16 roc_auc binary 0.974 1 NA Preprocessor1_Model10 4 7 22 roc_auc binary 0.969 1 NA Preprocessor1_Model09 5 5 19 roc_auc binary 0.969 1 NA Preprocessor1_Model06 # Display the accuracy of the models in descending order and the parameters rf_res %>% show_best(metric = "accuracy") # A tibble: 5 × 8 mtry min_n .metric .estimator mean n std_err .config <int> <int> <chr> <chr> <dbl> <int> <dbl> <chr> 1 4 4 accuracy binary 0.927 1 NA Preprocessor1_Model08 2 9 8 accuracy binary 0.926 1 NA Preprocessor1_Model03 3 8 16 accuracy binary 0.915 1 NA Preprocessor1_Model10 4 7 22 accuracy binary 0.906 1 NA Preprocessor1_Model09 5 5 19 accuracy binary 0.904 1 NA Preprocessor1_Model0

8) Select the model with the best parameters for accuracy mtry=4 and min_n=4. For this the accuracy is 0.927. For this configuration the roc_auc is also the best at 0.980

9) Plot the Area Under the Curve (AUC). It can be seen that this model performs really well and it hugs the top left.

# Pick the best model rf_best <- rf_res %>% select_best(metric = "accuracy") # The best model has mtry=4 and min=4 rf_best mtry min_n .config <int> <int> <chr> 1 4 4 Preprocessor1_Model08 #Plot AUC rf_auc <- rf_res %>% collect_predictions(parameters = rf_best) %>% roc_curve(isWinner, .pred_0) %>% mutate(model = "Random Forest") autoplot(rf_auc)

10) Create the final model with the best parameters

11) Execute the final fit

12) Plot feature importance, The bowler and batsman embedding followed by perfIndex and runRate are features that contribute the most to the Win Probability

last_rf_mod <- rand_forest(mtry = 4, min_n = 4, trees = 1000) %>% set_engine("ranger", num.threads = cores, importance = "impurity") %>% set_mode("classification") # the last workflow last_rf_workflow <- rf_workflow %>% update_model(last_rf_mod) set.seed(345) last_rf_fit <- last_rf_workflow %>% last_fit(splits) last_rf_fit %>% collect_metrics() .metric .estimator .estimate .config <chr> <chr> <dbl> <chr> 1 accuracy binary 0.944 Preprocessor1_Model1 2 roc_auc binary 0.988 Preprocessor1_Model1 last_rf_fit %>% extract_fit_parsnip() %>% vip(num_features = 9)

13) Plot the ROC curve for the best fit

# Plot the ROC for the final model last_rf_fit %>% collect_predictions() %>% roc_curve(isWinner, .pred_0) %>% autoplot()

14) Create a confusion matrix

We can see that the number of false positives and false negatives is very low

15) Create the final fit with the Random Forest Model

# generate predictions from the test set test_predictions <- last_rf_fit %>% collect_predictions() test_predictions id .pred_0 .pred_1 .row .pred_class isWinner .config <chr> <dbl> <dbl> <int> <fct> <fct> <chr> 1 train/test split 0.838 0.162 1 0 0 Preprocessor1_Mo… 2 train/test split 0.463 0.537 11 1 0 Preprocessor1_Mo… 3 train/test split 0.846 0.154 14 0 0 Preprocessor1_Mo… 4 train/test split 0.839 0.161 22 0 0 Preprocessor1_Mo… 5 train/test split 0.846 0.154 36 0 0 Preprocessor1_Mo… 6 train/test split 0.848 0.152 37 0 0 Preprocessor1_Mo… 7 train/test split 0.731 0.269 39 0 0 Preprocessor1_Mo… 8 train/test split 0.972 0.0281 40 0 0 Preprocessor1_Mo… 9 train/test split 0.655 0.345 42 0 0 Preprocessor1_Mo… 10 train/test split 0.662 0.338 43 0 0 Preprocessor1_Mo… # generate a confusion matrix test_predictions %>% conf_mat(truth = isWinner, estimate = .pred_class) Truth Prediction 0 1 0 116576 7096 1 6391 109760 # Create the final model final_model <- fit(last_rf_workflow, df_other)

16) Computing Win Probability with Random Forest Model for match

- Pakistan-India-2022-10-23

17) Worm -wicket graph of match

- Pakistan-India-2022-10-23

C**) Win Probability using MLP – Deep Neural Network (DNN) with player embeddings**

In this approach the MLP package of Tidymodels was used. Multi-layer perceptron (MLP) with Deep Neural Network (DNN) was used to compute the Win Probability using player embeddings. An accuracy of 0.76 was obtained

1) Read the data

a) Read the data from 9 T20 leagues (BBL, CPL, IPL, NTB, PSL, SSM, T20 Men, T20 Women, WBB) and create a single data frame of ball-by-ball data. Display the data frame

2) Create training.validation and test sets

b) Split to training, validation and test sets. The dataset is initially split into training and test in the ratio 80%:20%. The training data is again split into training and validation in the ratio 80:20

library(dplyr) library(caret) library(e1071) library(ggplot2) library(tidymodels) library(embed) # Helper packages library(readr) # for importing data library(vip) library(ranger) df1=read.csv("output3/matchesBBL3.csv") df2=read.csv("output3/matchesCPL3.csv") df3=read.csv("output3/matchesIPL3.csv") df4=read.csv("output3/matchesNTB3.csv") df5=read.csv("output3/matchesPSL3.csv") df6=read.csv("output3/matchesSSM3.csv") df7=read.csv("output3/matchesT20M3.csv") df8=read.csv("output3/matchesT20W3.csv") df9=read.csv("output3/matchesWBB3.csv") df=rbind(df1,df2,df3,df4,df5,df6,df7,df8,df9) set.seed(123) df$isWinner = as.factor(df$isWinner) splits <- initial_split(df,prop = 0.80) df_other <- training(splits) df_test <- testing(splits) set.seed(234) val_set <- validation_split(df_other, prop = 0.80) val_set

3) Create a Deep Neural Network recipe

- Normalize parameters
- Add embeddings for batsman, bowler

4) Set the MLP-DNN hyperparameters

- epochs=100
- hidden units =5
- dropout regularization =0.1

5) Fit on Training data

cores <- parallel::detectCores() cores nn_recipe <- recipe(isWinner ~ ., data = df_other) %>% step_normalize(ballNum,ballsRemaining,runs,runRate,numWickets,runsMomentum,perfIndex) %>% step_embed(batsman,bowler, outcome = vars(isWinner)) %>% prep(training = df_other, retain = TRUE) # For validation: test_normalized <- bake(nn_recipe, new_data = df_test) set.seed(57974) # Set the hyper parameters for DNN # Use Keras # Fit on training data nnet_fit <- mlp(epochs = 100, hidden_units = 5, dropout = 0.1) %>% set_mode("classification") %>% # Also set engine-specific `verbose` argument to prevent logging the results: set_engine("keras", verbose = 0) %>% fit(isWinner ~ ., data = bake(nn_recipe, new_data = df_other)) nnet_fit parsnip model object Model:"sequential" ____________________________________________________________________________ Layer (type) Output Shape Param # ============================================================================ dense (Dense) (None, 5) 60 ____________________________________________________________________________ dense_1 (Dense) (None, 5) 30 ____________________________________________________________________________ dropout (Dropout) (None, 5) 0 ____________________________________________________________________________ dense_2 (Dense) (None, 2) 12 ============================================================================ Total params: 102 Trainable params: 102 Non-trainable params: 0

6) Test on Test data

- Check ROC_AUC. It is 0.854
- Check accuracy. The MLP-DNN gives a decent performance with an acuracy of 0.76
- Compute the Confusion Matrix

# Validate on test data val_results <- df_test %>% bind_cols( predict(nnet_fit, new_data = test_normalized), predict(nnet_fit, new_data = test_normalized, type = "prob") ) val_results # Check roc_auc val_results %>% roc_auc(truth = isWinner, .pred_0) .metric .estimator .estimate <chr> <chr> <dbl> 1 roc_auc binary 0.854 # Check accuracy val_results %>% accuracy(truth = isWinner, .pred_class) .metric .estimator .estimate <chr> <chr> <dbl> 1 accuracy binary 0.762 # Display confusion matrix val_results %>% conf_mat(truth = isWinner, .pred_class) Truth Prediction 0 1 0 97419 31564 1 25548 85292

Conclusion

- Of the 3 ML models, glmnet, random forest and Multi-layer Perceptron DNN, random forest had the best performance
- Random Forest ML model with batsman, bowler embeddings was able to achieve an accuracy of 92.4% and a ROC_AUC of 0.98 with very low false positives, negatives. This was a quantum jump from my earlier random forest model without embeddings which had an accuracy of 73.7% and an ROC_AUC of 0.834
- The glmnet and NN models are fairly light weight. Random Forest is computationally very intensive.

Check out my other posts

- Using Reinforcement Learning to solve Gridworld
- Deep Learning from first principles in Python, R and Octave – Part 8
- Introducing QCSimulator: A 5-qubit quantum computing simulator in R
- Big Data-5: kNiFi-ing through cricket data with yorkpy
- Singularity
- Practical Machine Learning with R and Python – Part 6
- GooglyPlusPlus2022 optimizes batting/bowling lineup
- Fun simulation of a Chain in Android
- Introducing cricpy:A python package to analyze performances of cricketers
- Programming languages in layman’s language

To see all posts click Index of posts

**leave a comment**for the author, please follow the link and comment on their blog:

**R – Giga thoughts …**.

R-bloggers.com offers

**daily e-mail updates**about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.