Digitalization, India, New Products & Forecasting

[This article was first published on R – Data quest, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Digitalization and Demonetization

India being the land of culture, land of opportunites and land of 130 cr consumers.
Over the years, India is not able to grow to its actual potential; major roadbloack was and is corruption.

This present Indian government’s very unpopular move of Demonetization aimed to curb corruption and push India towards Digitalization.

Although demonetization created lot of havoc and problems but it also created opportunities for the new product/s.

I don’t know whether it curbed the corruption or not but I can see a clear push towards digital banking echo system and new products.

plot of chunk unnamed-chunk-1

We can clearly observe during & after demonetization exponential jump of the usage of plastic money at the point of sale (POS). In December’2016, debt cards usage at POS is almost 4 times that of October usage.
We all expected that this increase will ease it out once situation become normal & new currency is easily available, even traders in India don’t encourage to take payments through plastic money as these transactions comes under income tax.

Though the situation had become normal now, usage of the plastic money at POS is still almost twice as that of before demonetization.

Mobile Money

Demonetization also created an environment where mobile cash can grow many folds and here mobile wallets and newly launched Unified payment Interface got the perfect opportunity to grow.

In October 2016, before demonetization, Mobile wallet + UPI together accounts for nearly 100 mn transactions which has now grown to 469 mn transactions in April 2018 and as per latest figures from RBI, alone UPI transactions in the month of June’18 accounts is little more than 400 mn transactions.

plot of chunk unnamed-chunk-2

Trend is clearly visible – India is slowly moving towards cashless economy.
But the bigger question of an hour is do we have such infrastructure ready for such kind of transition?

Forecast for the POS machines

Here is the small attempt to predict/forecast the number POS machines required for the next 1 year.
Data source is again RBI.

Before modeling the dataset, need to follow below guidelines (Courtsey Learnings from Time series Analysis @ Coursera)

Guidelines for modeling the time series

  • Trend suggests differencing
  • Variation in variance suggests transformation
  • Common transformation: log, then differencing -It is also known as log-return
  • ACF suggests order of moving average process (q)
  • PACF suggests order of autoregressive process (p)
  • Akaike Information Criterion (AIC)
  • Sum of squared errors (SSE)
  • Ljung-Box Q-statistics
  • Estimation!

data$Date <- as.Date(data$Date, format= "%d-%m-%Y") 

#top 5 rows of the data

kable (head(data,5))
Bank_Name Bank_Type Date Number_of_POS
Allahabad Bank Nationalised banks 2011-04-01 0
Andhra Bank Nationalised banks 2011-04-01 2113
Bank of Baroda Nationalised banks 2011-04-01 4537
Bank of India Nationalised banks 2011-04-01 2581
Bank of Maharashtra Nationalised banks 2011-04-01 481
pos_data<-aggregate(data$Number_of_POS, by=list(data$Date), sum)

names (pos_data)<- c("Date","POS")

#converting the data in ts format

ts_pos<- ts(pos_data[,-1],start = c(2011,4),end = c(2018,5), frequency = 12)

Ploting the data to figure out the trend

plot (ts_pos, main="Monthly POS machines deployed", ylab="Number of POS Machines deployed")

plot of chunk unnamed-chunk-4

Clearly we can see their is trend in the data & series is not Stationay.

We need to remove the trend.

Lets first check is there any auto- correlation between different lags of the time series.
For this we will use Box Test.

Box.test (ts_pos,lag=log(length(ts_pos)))

Box-Pierce test

data: ts_pos
X-squared = 273.29, df = 4.4543, p-value < 2.2e-16
The p- value is very small – which indicates there is definately auto-correaltion.

Now the next step is to remove the trend, I will use difference operator (diff)

plot (diff(ts_pos),main="Diferenced data of 1st order", ylab="Number of POS Machines deployed")

plot of chunk unnamed-chunk-6

Box.test (diff(ts_pos),lag=log(length(diff(ts_pos))))

Box-Pierce test

data: diff(ts_pos)
X-squared = 81.76, df = 4.4427, p-value < 2.2e-16

Still we can see the trend in the data. Again we have to apply difference operator

plot (diff(diff(ts_pos)),main="Diferenced data of order 2", ylab="Number of POS Machines deployed")

plot of chunk unnamed-chunk-7

Box.test (diff(diff(ts_pos)),lag=log(length(diff(diff(ts_pos)))))

Box-Pierce test

data: diff(diff(ts_pos))
X-squared = 15.703, df = 4.4308, p-value = 0.004967

Now let’s look at the ACF & PACF which also suggest auto-correaltion in the data
acf (diff(diff(ts_pos)), main="ACF of Differenced data")

plot of chunk unnamed-chunk-8

pacf(diff(diff(ts_pos)), main='PACF of differenced data')

plot of chunk unnamed-chunk-8

  • ACF – There is significant lag at lag 1 and 1 pick at lag 14 (may be we can ignore this and refer this as randomness).

  • PACF – There is significant lag at lag 1 (for autoregressive process)

Now let try different models

It is clear, our d=2 (we have performed difference twice)
Now let’s figure out other parameter of ARIMA model

diff.model<- c()
mod1<- c()
for(p in 1:4){
  for(q in 1:4){
          model<-arima(x=ts_pos, order = c((p-1),d,(q-1)))
          pval<-Box.test(model$residuals, lag=log(length(model$residuals)))
          mod1<- c(model$aic,sse,pval$p.value)
         diff.model<- cbind(diff.model,mod1)

colnames(diff.model)<- c ("Arima 0,2,0","Arima 0,2,1","Arima 0,2,2","Arima 0,2,3","Arima 1,2,0","Arima 1,2,1","Arima 1,2,2","Arima 1,2,3","Arima 2,2,0","Arima 2,2,1","Arima 2,2,2","Arima 2,2,3","Arima 3,2,0","Arima 3,2,1","Arima 3,2,2","Arima 3,2,3")
rownames (diff.model)=c('AIC', 'SSE', 'p-value')

kable (diff.model)
Arima 0,2,0 Arima 0,2,1 Arima 0,2,2 Arima 0,2,3 Arima 1,2,0 Arima 1,2,1 Arima 1,2,2 Arima 1,2,3 Arima 2,2,0 Arima 2,2,1 Arima 2,2,2 Arima 2,2,3 Arima 3,2,0 Arima 3,2,1 Arima 3,2,2 Arima 3,2,3
AIC 2.04390e+03 2.032641e+03 2.034152e+03 2.034000e+03 2.031575e+03 2.033573e+03 2.034886e+03 2.030729e+03 2.033574e+03 2.029861e+03 2.031232e+03 2.032559e+03 2.035330e+03 2.037349e+03 2.032914e+03 2.034753e+03
SSE 1.77329e+11 1.511417e+11 1.502817e+11 1.460151e+11 1.492217e+11 1.492186e+11 1.478676e+11 1.360945e+11 1.492195e+11 1.379507e+11 1.369194e+11 1.357870e+11 1.487718e+11 1.488064e+11 1.363015e+11 1.360618e+11
p-value 4.31830e-03 8.042071e-01 9.096711e-01 9.588175e-01 9.737261e-01 9.731882e-01 9.171468e-01 9.993006e-01 9.733108e-01 9.898481e-01 9.964327e-01 9.749469e-01 9.756955e-01 9.678883e-01 9.879784e-01 9.919547e-01
From the above we can conclude that
  • Model with least AIC is Arima 2,2,1.
  • Second best model is ARIMA 1,2,3
  • Third best model is ARIMA 1,2,0

Now the choice is between these 3 models.

Although there is a function in forecast library which automatically generate the parameter for least error.


Series: ts_pos

ar1 ar2 ma1
0.4638 0.2614 -0.9687
s.e. 0.1141 0.1113 0.0465

sigma2 estimated as 1.703e+09: log likelihood=-1010.93
AIC=2029.86 AICc=2030.37 BIC=2039.58
According to the above function also best model is ARIMA 2,2,1.

Now we will be using these parameters in our final model



initial value 10.747222
iter 2 value 10.726566
iter 3 value 10.663160
iter 4 value 10.660967
iter 5 value 10.660754
iter 6 value 10.660753
iter 7 value 10.660729
iter 8 value 10.660648
iter 9 value 10.660585
iter 10 value 10.660574
iter 11 value 10.660567
iter 12 value 10.660528
iter 13 value 10.660371
iter 14 value 10.660275
iter 15 value 10.660237
iter 16 value 10.660214
iter 17 value 10.659064
iter 18 value 10.658786
iter 19 value 10.658386
iter 20 value 10.657945
iter 21 value 10.652212
iter 22 value 10.649269
iter 23 value 10.638113
iter 24 value 10.633268
iter 25 value 10.627282
iter 26 value 10.625633
iter 27 value 10.623322
iter 28 value 10.621746
iter 29 value 10.621737
iter 30 value 10.621721
iter 30 value 10.621721
final value 10.621721
initial value 10.616198
iter 2 value 10.616101
iter 3 value 10.615955
iter 4 value 10.615953
iter 5 value 10.615951
iter 6 value 10.615951
iter 6 value 10.615950
iter 6 value 10.615950
final value 10.615950
plot of chunk unnamed-chunk-11$fit

stats::arima(x = xdata, order = c(p, d, q), seasonal = list(order = c(P, D,
Q), period = S), include.mean = !no.constant, optim.control = list(trace = trc,
REPORT = 1, reltol = tol))

ar1 ar2 ma1
0.4638 0.2614 -0.9687
s.e. 0.1141 0.1113 0.0465

sigma2 estimated as 1.642e+09: log likelihood = -1010.93, aic = 2029.86

[1] 81

Estimate SE t.value p.value
ar1 0.4638 0.1141 4.0652 0.0001
ar2 0.2614 0.1113 2.3494 0.0212
ma1 -0.9687 0.0465 -20.8369 0.0000

[1] 22.28911

[1] 22.31811

[1] 21.37472

Looking at the residual plots, we can conculde

  • There is no significant auto-correlation (ACF of Residual).
  • There is no small p value – which tell us there is no signifcant auto -correlation

We can assume at this point residuals are the white noise.

So the final model will be

model<-arima(x=ts_pos,order = c(1,2,1))


plot of chunk unnamed-chunk-12

Forecast for the next 12 months

pos_predict<- forecast(model,12)
kable (pos_predict)
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
Jun 2018 3322859 3268845 3376873 3240252 3405467
Jul 2018 3390114 3288152 3492077 3234177 3546052
Aug 2018 3460657 3297580 3623734 3211252 3710061
Sep 2018 3529929 3298050 3761808 3175300 3884558
Oct 2018 3599692 3291072 3908312 3127698 4071685
Nov 2018 3669265 3277079 4061451 3069469 4269061
Dec 2018 3738912 3256727 4221096 3001474 4476349
Jan 2019 3808530 3230390 4386669 2924342 4692718
Feb 2019 3878159 3198441 4557877 2838621 4917697
Mar 2019 3947784 3161166 4734401 2744756 5150812
Apr 2019 4017410 3118822 4915999 2643137 5391684
May 2019 4087036 3071626 5102447 2534100 5639973

4 mn POS machine will be required to deploy in May 2019.

In May 2016 – big 4 banks (SBI, HDFC, ICICI & Axis Bank) accounts for 75% of POS machine deployed. The same 4 banks accounts for 67% in May 2017 and 57% in May 2018. New players like RBL, corporation Banks getting very aggresive in deploying POS machines.

To leave a comment for the author, please follow the link and comment on their blog: R – Data quest. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)