Bad Stock Photos of My Job? Data Science on Pexels

[This article was first published on Maëlle, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

I couldn’t miss the fun Twitter hashtag #BadStockPhotosOfMyJob thanks to a tweet by Julia Silge and another one by Colin Fay. The latter inspired me to actually go and look for what makes a data science photo… What characterizes “data science” stock photos?

My (not bad) stock photo source

Pexels metadata for the win

Where to find information related to stock photos? In two previous blog posts of mine I used Pexels, a website providing CC0 pictures which is quite nice. My goal was to obtain the titles and the tags of stock photos of “data science”: for instance if you look at this picture, its tags are “business”, “contemporary”, “computer”, etc. Pexels tags are very useful metadata, saving me the effort to use machine learning methods to analyse images.

Responsible webscraping

When researching this post I discovered that Pexels has an API, documented here but this API does not get you the title nor the tags associated to a picture so only webscraping could get me what I needed.

Webscraping is a powerful tool allowing one to rectangle webpages but with great power comes great responsability. Being able to scrape a webpage does not mean you are allowed to. You could get sued or your IP could get blocked. I am far from being an expert but I often read Bob Rudis’ blog where I learnt about rOpenSci’s robotstxt package that does “robots.txt file parsing and checking for R” which in plain language means it checks for you what a webpage legally allows you to do. See below,

<span class="c1"># how I'll find pictures</span><span class="w">
</span><span class="n">robotstxt</span><span class="o">::</span><span class="n">paths_allowed</span><span class="p">(</span><span class="s2">"https://www.pexels.com/search"</span><span class="p">)</span><span class="w">
</span>
## [1] TRUE
<span class="c1"># where tags live</span><span class="w">
</span><span class="n">robotstxt</span><span class="o">::</span><span class="n">paths_allowed</span><span class="p">(</span><span class="s2">"https://www.pexels.com/photo"</span><span class="p">)</span><span class="w">
</span>
## [1] TRUE

robots.txt files often also tell you how often you can hit a page by defining a “crawling delay”. Sadly Pexels robots.txt doesn’t:

<span class="n">robotstxt</span><span class="o">::</span><span class="n">get_robotstxt</span><span class="p">(</span><span class="s2">"https://www.pexels.com"</span><span class="p">)</span><span class="w">
</span>
## Sitemap: https://s3.amazonaws.com/pexels/sitemaps/sitemap.xml.gz

But Bob Rudis, who was patient and nice enough to answer my questions, told me that I should probably respect the rate limit defined in Pexels API docs. “Do not abuse the API. The API is rate-limited to 200 requests per hour and 20,000 requests per month.” As I recently explained in a post on Locke Data’s blog, these days to limit rate of a function I use the very handy ratelimitr package by Tarak Shah.

<span class="n">limited_get</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">ratelimitr</span><span class="o">::</span><span class="n">limit_rate</span><span class="p">(</span><span class="n">httr</span><span class="o">::</span><span class="n">GET</span><span class="p">,</span><span class="w">
                                      </span><span class="n">ratelimitr</span><span class="o">::</span><span class="n">rate</span><span class="p">(</span><span class="m">200</span><span class="p">,</span><span class="w"> </span><span class="m">60</span><span class="o">*</span><span class="m">60</span><span class="p">),</span><span class="c1"># not more than 200 times an hour</span><span class="w">
                                      </span><span class="n">ratelimitr</span><span class="o">::</span><span class="n">rate</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="w"> </span><span class="m">5</span><span class="p">))</span><span class="c1">#not more than 1 time every 5 seconds</span><span class="w">
</span>

Elegant webscraping

At the time of the two aforelinked blog posts I had used RSelenium to scroll down and get the download link of many pictures, but Bob Rudis wrote an elegant and cool alternative using query parameters, on which I’ll build in this post.

I first re-wrote the function to get all 15 pictures of each page of results.

<span class="n">get_page</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1</span><span class="p">,</span><span class="w"> </span><span class="n">seed</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
  </span><span class="n">message</span><span class="p">(</span><span class="n">num</span><span class="p">)</span><span class="w">
  </span><span class="n">limited_get</span><span class="p">(</span><span class="w">
    </span><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"https://www.pexels.com/search/data science/"</span><span class="p">,</span><span class="w">
    </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">page</span><span class="o">=</span><span class="n">num</span><span class="p">,</span><span class="w">
      </span><span class="n">seed</span><span class="o">=</span><span class="n">seed</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">res</span><span class="w">
  
  </span><span class="n">httr</span><span class="o">::</span><span class="n">stop_for_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  
  </span><span class="n">pg</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">httr</span><span class="o">::</span><span class="n">content</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  
  </span><span class="n">tibble</span><span class="o">::</span><span class="n">tibble</span><span class="p">(</span><span class="w">
    </span><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">rvest</span><span class="o">::</span><span class="n">html_attr</span><span class="p">(</span><span class="n">rvest</span><span class="o">::</span><span class="n">html_nodes</span><span class="p">(</span><span class="n">pg</span><span class="p">,</span><span class="w"> </span><span class="n">xpath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"//a[@class='js-photo-link']"</span><span class="p">),</span><span class="w"> </span><span class="s2">"href"</span><span class="p">),</span><span class="w">
    </span><span class="n">title</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">rvest</span><span class="o">::</span><span class="n">html_attr</span><span class="p">(</span><span class="n">rvest</span><span class="o">::</span><span class="n">html_nodes</span><span class="p">(</span><span class="n">pg</span><span class="p">,</span><span class="w"> </span><span class="n">xpath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"//a[@class='js-photo-link']"</span><span class="p">),</span><span class="w"> </span><span class="s2">"title"</span><span class="p">),</span><span class="w">
    </span><span class="n">tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">purrr</span><span class="o">::</span><span class="n">map</span><span class="p">(</span><span class="n">url</span><span class="p">,</span><span class="w"> </span><span class="n">get_tags</span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w">
  
</span><span class="p">}</span><span class="w"> 

</span>

I re-wrote it because I needed the “href” and because it seems that the structure of each page changed a bit since the day on which the gist was written. To find out I had to write “a[@class=’js-photo-link’]” I inspected the source of a page.

Then I wrote a function getting tags for each picture.

<span class="n">get_tags</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">url</span><span class="p">){</span><span class="w">
  </span><span class="n">message</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="w">
  </span><span class="n">url</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">paste0</span><span class="p">(</span><span class="s2">"https://www.pexels.com"</span><span class="p">,</span><span class="w"> </span><span class="n">url</span><span class="p">)</span><span class="w">
  </span><span class="n">res</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">limited_get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="w">
  </span><span class="n">httr</span><span class="o">::</span><span class="n">stop_for_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  </span><span class="n">pg</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">httr</span><span class="o">::</span><span class="n">content</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  </span><span class="n">nodes</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">rvest</span><span class="o">::</span><span class="n">html_nodes</span><span class="p">(</span><span class="n">pg</span><span class="p">,</span><span class="w"> </span><span class="n">xpath</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'//a[@data-track-label="tag" ]'</span><span class="p">)</span><span class="w">
  </span><span class="n">rvest</span><span class="o">::</span><span class="n">html_text</span><span class="p">(</span><span class="n">nodes</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

And finally I got results for 20 pages. I chose 20 without thinking too much. It seemed enough for my needs, and each of these pages had pictures.

<span class="n">ds_stock</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">purrr</span><span class="o">::</span><span class="n">map_df</span><span class="p">(</span><span class="m">1</span><span class="o">:</span><span class="m">20</span><span class="p">,</span><span class="w"> </span><span class="n">get_page</span><span class="p">)</span><span class="w">
</span><span class="n">ds_stock</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">unique</span><span class="p">(</span><span class="n">ds_stock</span><span class="p">)</span><span class="w">
</span><span class="n">ds_stock</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">tidyr</span><span class="o">::</span><span class="n">unnest</span><span class="p">(</span><span class="n">ds_stock</span><span class="p">,</span><span class="w"> </span><span class="n">tags</span><span class="p">)</span><span class="w">
</span>

I got 300 unique pictures.

What’s in a data science stock photo?

Now that I have all this information at hand, I can describe data science stock photos!

Data science tags

<span class="n">library</span><span class="p">(</span><span class="s2">"ggplot2"</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="s2">"ggalt"</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="s2">"hrbrthemes"</span><span class="p">)</span><span class="w">
</span><span class="n">tag_counts</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">dplyr</span><span class="o">::</span><span class="n">count</span><span class="p">(</span><span class="n">ds_stock</span><span class="p">,</span><span class="w"> </span><span class="n">tags</span><span class="p">,</span><span class="w"> </span><span class="n">sort</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">)[</span><span class="m">1</span><span class="o">:</span><span class="m">10</span><span class="p">,]</span><span class="w">

</span><span class="n">dplyr</span><span class="o">::</span><span class="n">mutate</span><span class="p">(</span><span class="n">tag_counts</span><span class="p">,</span><span class="w">
              </span><span class="n">tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">reorder</span><span class="p">(</span><span class="n">tags</span><span class="p">,</span><span class="w"> </span><span class="n">n</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
</span><span class="n">ggplot</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">geom_lollipop</span><span class="p">(</span><span class="n">aes</span><span class="p">(</span><span class="n">tags</span><span class="p">,</span><span class="w"> </span><span class="n">n</span><span class="p">),</span><span class="w">
                </span><span class="n">size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="p">,</span><span class="w"> </span><span class="n">col</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"salmon"</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">hrbrthemes</span><span class="o">::</span><span class="n">theme_ipsum</span><span class="p">(</span><span class="n">base_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">16</span><span class="p">,</span><span class="w">
                          </span><span class="n">axis_title_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">16</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">xlab</span><span class="p">(</span><span class="s2">"Tag"</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">ggtitle</span><span class="p">(</span><span class="s2">"300 data science stock photos"</span><span class="p">,</span><span class="w">
          </span><span class="n">subtitle</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Most frequent tags. Source: https://www.pexels.com"</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">coord_flip</span><span class="p">()</span><span class="w">
</span>

plot of chunk unnamed-chunk-4

So the most common tags are data, technology, business and computer. Not too surprising!

Data science scenes

Now, let’s have a look at titles that are in general more descriptive of what’s happening/present on the photo (i.e. is the computer near a cup of coffee or is someone working on it). I tried using a technique described in Julia Silge’s and David Robinson’s Tidy text mining book: “Counting and correlating pairs of words with the widyr package” described in this section of the book but it wasn’t too interesting because most correlation values were too low. One issue was probably my having too few titles: only half of pictures have titles! So I’ll resort to plotting most common bigrams, which I learnt in the Tidy text mining book as well.

<span class="n">stopwords</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">rcorpora</span><span class="o">::</span><span class="n">corpora</span><span class="p">(</span><span class="s2">"words/stopwords/en"</span><span class="p">)</span><span class="o">$</span><span class="n">stopWords</span><span class="w">

</span><span class="n">ds_stock</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">filter</span><span class="p">(</span><span class="o">!</span><span class="nf">is.na</span><span class="p">(</span><span class="n">title</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">select</span><span class="p">(</span><span class="n">title</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">unique</span><span class="p">()</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">tidytext</span><span class="o">::</span><span class="n">unnest_tokens</span><span class="p">(</span><span class="n">bigram</span><span class="p">,</span><span class="w"> </span><span class="n">title</span><span class="p">,</span><span class="w">
                          </span><span class="n">token</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"ngrams"</span><span class="p">,</span><span class="w"> </span><span class="n">n</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="p">)</span><span class="w">  </span><span class="o">%>%</span><span class="w">
  </span><span class="n">tidyr</span><span class="o">::</span><span class="n">separate</span><span class="p">(</span><span class="n">bigram</span><span class="p">,</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"word1"</span><span class="p">,</span><span class="w"> </span><span class="s2">"word2"</span><span class="p">),</span><span class="w"> </span><span class="n">sep</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">" "</span><span class="p">,</span><span class="w">
                  </span><span class="n">remove</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">)</span><span class="w">  </span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">filter</span><span class="p">(</span><span class="o">!</span><span class="n">word1</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="n">stopwords</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">filter</span><span class="p">(</span><span class="o">!</span><span class="n">word2</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="n">stopwords</span><span class="p">)</span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">count</span><span class="p">(</span><span class="n">bigram</span><span class="p">,</span><span class="w"> </span><span class="n">sort</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">dplyr</span><span class="o">::</span><span class="n">mutate</span><span class="p">(</span><span class="n">bigram</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">reorder</span><span class="p">(</span><span class="n">bigram</span><span class="p">,</span><span class="w"> </span><span class="n">n</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w">
  </span><span class="n">head</span><span class="p">(</span><span class="n">n</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">10</span><span class="p">)</span><span class="o">%>%</span><span class="w">
</span><span class="n">ggplot</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">geom_lollipop</span><span class="p">(</span><span class="n">aes</span><span class="p">(</span><span class="n">bigram</span><span class="p">,</span><span class="w"> </span><span class="n">n</span><span class="p">),</span><span class="w">
                </span><span class="n">size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="p">,</span><span class="w"> </span><span class="n">col</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"salmon"</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">hrbrthemes</span><span class="o">::</span><span class="n">theme_ipsum</span><span class="p">(</span><span class="n">base_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">16</span><span class="p">,</span><span class="w">
                          </span><span class="n">axis_title_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">16</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">ggtitle</span><span class="p">(</span><span class="s2">"300 data science stock photos"</span><span class="p">,</span><span class="w">
          </span><span class="n">subtitle</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Most frequent bigrams in titles. Source: https://www.pexels.com"</span><span class="p">)</span><span class="o">+</span><span class="w">
  </span><span class="n">coord_flip</span><span class="p">()</span><span class="w">
</span>

plot of chunk unnamed-chunk-5

So there’s a lot of holding computer happening, and these laptops are either black or white… And well Macbook Pro probably looks more professional?

Hold my laptop and watch…

my trying to find a good post conclusion! In this post, I tried to responsibly and elegantly scrape rich photo metadata from Pexels to characterize stock photos of data science. Using tags, and most common bigrams in titles, I found that data science stock photos are associated with data, business and computers; and that they often show people holding computers. Now, you’ll excuse me while I try and comfort my poor pink laptop that feels a bit too un-data-sciency.

To leave a comment for the author, please follow the link and comment on their blog: Maëlle.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)