Does weather cause accidents – part 1

[This article was first published on Adventures in Data, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Blending weather data

Scotland and other parts of the UK have some nicely curated open data on road traffic accidents. For individual cases, where and when they happened, how severe they were, the prevailing road conditions, which emergency services were involved and so on. It felt to me like this should be a very valuable resource and might help us learn about what the causes or maybe correlates of road traffic accidents are. This is partly inspired by Ben Moores work at http://blackspot.org.uk/.

One thought that sprang to mind was could we test the common belief that adverse weather conditions result in increased incidence and severity of road traffic accidents. The first part of this was to collect and blend some information on weather conditions for each crash recorded. In this blog, I will explain how I obtained / blended weather information relating to each road traffic incident.

The raw data can be found on Edinburgh City Council’s website or pinched from the github repository associated with this post, see the /data folder.

I used a very convenient service from http://forecast.io/ which I think is designed for weather app makers but I think it can be reasonably employed for small amounts of data plundering. Head to https://developer.forecast.io/ and get yourself a free API key which you will need for this. Most importantly we are using the Rforecastio package which is housed here thanks to hrbrmstr for providing it. Before starting you should install this with devtools::install_github("hrbrmstr/Rforecastio").

Now you’re ready to go fire up R and load in some libraries.

<span class="n">library</span><span class="p">(</span><span class="n">dplyr</span><span class="p">)</span>
<span class="n">library</span><span class="p">(</span><span class="n">lubridate</span><span class="p">)</span>
<span class="n">library</span><span class="p">(</span><span class="n">Rforecastio</span><span class="p">)</span>

I had to set the scipen option to avoid an error (i’ll explain later on).

<span class="n">options</span><span class="p">(</span><span class="n">scipen</span> <span class="o">=</span> <span class="m">100</span><span class="p">)</span>

Next you want to read in the data and I am also going to sample 1000 rows. This is because the forecast API will let you make 1000 free API calls per day. Ideally it would be nice to get the weather data for all of the rows but lets just go with a subset for now. Note I also create a numeric version of the date variable, for later use with Rforecastio.

<span class="n">d</span> <span class="o"><-</span> <span class="n">read.csv</span><span class="p">(</span><span class="s2">"data/acc-scot-2005-2012.csv"</span><span class="p">)</span>
<span class="n">d</span><span class="o">$</span><span class="n">Date</span> <span class="o"><-</span> <span class="n">dmy</span><span class="p">(</span><span class="n">d</span><span class="o">$</span><span class="n">Date</span><span class="p">)</span> <span class="o">%>%</span> <span class="n">as.numeric</span>
<span class="n">d</span> <span class="o"><-</span> <span class="n">d</span> <span class="o">%>%</span> <span class="n">sample_n</span><span class="p">(</span><span class="m">1000</span><span class="p">)</span>

Now we need to connect our session to the API, you can visit the Rforecastio documentation for this but all you need to do is place your key in the system environment as follows.

<span class="n">Sys.setenv</span><span class="p">(</span><span class="n">FORECASTIO_API_KEY</span> <span class="o">=</span> <span class="s2">"<Insert your API key here>"</span><span class="p">)</span> 
<span class="n">forecastio_api_key</span><span class="p">()</span>

Now I am going to set up the data collection as follows; first I will define wrapper that calls the API through the Rforecastio::get_forecast_for() function. This function will call the API using longitude, latitude and date/time information from the road traffic accidents. I do this because I know that certain things will be fixed such as the level of granularity and what aspect of the results we want to keep. I will then call this function once on the first row of our road traffic accidents, mainly to get an object of the with the correct size and attributes to then collect our results in.

<span class="n">getDailyWeather</span> <span class="o"><-</span> <span class="k">function</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="n">lon</span><span class="p">,</span> <span class="n">time</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">this</span> <span class="o"><-</span> <span class="n">get_forecast_for</span><span class="p">(</span>
    <span class="n">lat</span><span class="p">,</span> <span class="n">lon</span><span class="p">,</span> <span class="n">time</span><span class="p">,</span>
    <span class="n">exclude</span> <span class="o">=</span> <span class="s2">"minutely,hourly,currently,alerts,flags"</span>
  <span class="p">)</span>
  <span class="k">return</span><span class="p">(</span><span class="n">this</span><span class="o">$</span><span class="n">daily</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">weather</span> <span class="o"><-</span> <span class="n">get_forecast_for</span><span class="p">(</span>
  <span class="n">d</span><span class="o">$</span><span class="n">Latitude</span><span class="p">[</span><span class="m">1</span><span class="p">],</span> 
  <span class="n">d</span><span class="o">$</span><span class="n">Longitude</span><span class="p">[</span><span class="m">1</span><span class="p">],</span> 
  <span class="n">d</span><span class="o">$</span><span class="n">Date</span><span class="p">[</span><span class="m">1</span><span class="p">],</span>
  <span class="n">exclude</span> <span class="o">=</span> <span class="s2">"minutely,hourly,alerts,flags,current"</span>
<span class="p">)</span><span class="o">$</span><span class="n">daily</span>
<span class="n">weather</span><span class="p">[</span><span class="m">1</span><span class="p">,]</span> <span class="o"><-</span> <span class="n">NA</span>

The object called weather is a data.frame in which we will store our results. Now all we need to do is loop through this object, each time collecting a result with our wrapper for get_forecast_for. At this point I was getting an error if the numeric date was too round and being formatted in scientific notation. That is why we set the scipen option earlier.

<span class="k">for</span><span class="p">(</span><span class="n">i</span> <span class="k">in</span> <span class="m">1</span><span class="o">:</span><span class="n">nrow</span><span class="p">(</span><span class="n">d</span><span class="p">))</span> <span class="p">{</span>
  <span class="n">weather</span><span class="p">[</span><span class="n">i</span><span class="p">,]</span> <span class="o"><-</span> <span class="n">getDailyWeather</span><span class="p">(</span>
    <span class="n">d</span><span class="o">$</span><span class="n">Latitude</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> 
    <span class="n">d</span><span class="o">$</span><span class="n">Longitude</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> 
    <span class="n">d</span><span class="o">$</span><span class="n">Date</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
  <span class="p">)</span>
  <span class="k">if</span><span class="p">(</span><span class="n">i</span><span class="o">%%</span><span class="m">10</span> <span class="o">==</span> <span class="m">0</span><span class="p">)</span> <span class="n">cat</span><span class="p">((</span><span class="n">i</span><span class="o">/</span><span class="n">nrow</span><span class="p">(</span><span class="n">d</span><span class="p">))</span><span class="o">*</span><span class="m">100</span><span class="p">,</span> <span class="s2">"% donen"</span><span class="p">)</span>
<span class="p">}</span>

Finally we can rejoin this to the original data and its done. We have now blended together the road traffic accidents with historical weather from forecast.io.

<span class="n">names</span><span class="p">(</span><span class="n">weather</span><span class="p">)</span> <span class="o"><-</span> <span class="n">paste0</span><span class="p">(</span><span class="s2">"WTR_"</span><span class="p">,</span> <span class="n">names</span><span class="p">(</span><span class="n">weather</span><span class="p">))</span>
<span class="n">weather_d</span> <span class="o"><-</span> <span class="n">cbind</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">weather</span><span class="p">)</span>
<span class="n">write.csv</span><span class="p">(</span><span class="n">weather_d</span><span class="p">,</span> <span class="s2">"data/sample_weather.csv"</span><span class="p">,</span> <span class="n">row.names</span> <span class="o">=</span> <span class="n">FALSE</span><span class="p">)</span>

To leave a comment for the author, please follow the link and comment on their blog: Adventures in Data.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)