Selenium is a powerful library available for both Python and R (the R version is called
RSelenium) which can automate tasks such as form filling, job applications, CRM system administration and many other tasks. That being said Selenium can be used as well to do a lot of harm such as filling up forms with fake answers, making bots to create fake views for Youtube and other nefarious purposes.
With this in mind, I can only think of what Peter Parker was told by Uncle Ben:
This blog post is about how setting up Selenium on R and Python went for me, If you can relate to this or have any insight, please leave a comment below!
Setting up Selenium on Python:
Learning how to use Selenium on Python took me about 10 minutes to figure out. All I needed to do was download chromedriver and install selenium
pip install selenium and I was ready to start working with it.
I was even able to do some form automation with it:
My experience with RSelenium
Coming from Python and wanting to do this in R presented an inconvenience for me as my main machine does not support virtualization- which disqualifies me from even being able to install Docker on the machine which I have been working on.
This left me with no other choice but to use Selenium strictly in Python.
While Chromedriver reccomends that it be ran on a VM, it is not a requirement, and I was able to use it in Python. My experience with RSelenium is that it is impossible to use it without Docker or something similar, which is disappointing as I wanted to see how
RSelenium matched up.
Getting past the limitations
If you are really set on wanting to use Selenium in an R framework (maybe because you need to do some data wrangling or want to use
tidyverse as part of your project, etc.), I would recommend writing the script in python and executing it in R with the the
reticulate package and have something like:
reticulate::py_run_file("path_to_python_file") ... ... (Rest of your R Code)
Let me reiterate you can learn how to use Selenium in Python in around 10 minutes, so the learning curve is as difficult as finding a solution for
RSelenium and will integrate in R code thanks to the
So as things look now- unless things change, my Selenium work will have to be written in Python.
This post originally was going to be one where I was going to compare the use and speed of Selenium in R and Python, but the inability to install Docker on my computer made me unable to do use the
I’m sure I am not the only one who faced this challenge, so I thought I would share my thoughts about how to get around it.
If you have a better solution- please feel free to share it with me as I would want to do a comparison between Python and R using Selenium!