Fun with Twitter

[This article was first published on Frank Portman, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

I’ve been playing around with the ‘twitteR’ package for R ever since I heard of its existence. Twitter is great and easy to mine because the messages are all-text and most people’s profiles are public. This process is made even easier with the ‘twitteR’ package, which takes advantage of the Twitter API.

After exploring some of the package’s capabilities, I decided to conduct a pretty basic sentiment analysis on some tweets with various hashtags. Specifically, I analyzed the polarity of each tweet – whether the tweet is positive, negative, or neutral.

The hashtags I used were: #YOLO, #FML, #blessed, #bacon

The actual script is fairly simple and repetitive but does yield some interesting results:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
<span class="line"><span class="kn">library</span><span class="p">(</span>twitteR<span class="p">)</span>
</span><span class="line"><span class="kn">library</span><span class="p">(</span>sentiment<span class="p">)</span>
</span><span class="line"><span class="kn">library</span><span class="p">(</span>ggplot2<span class="p">)</span>
</span><span class="line"><span class="kn">library</span><span class="p">(</span>RJSONIO<span class="p">)</span>
</span><span class="line"><span class="kn">library</span><span class="p">(</span>wordcloud<span class="p">)</span>
</span><span class="line">
</span><span class="line">yolo_tweets <span class="o"><-</span> searchTwitter<span class="p">(</span><span class="s">"#yolo"</span><span class="p">,</span> n <span class="o">=</span> <span class="m">1500</span><span class="p">)</span>
</span><span class="line">yolo_tweets <span class="o"><-</span> twListToDF<span class="p">(</span>yolo_tweets<span class="p">)</span>
</span><span class="line">yolo_tweets <span class="o"><-</span> yolo_tweets<span class="o">$</span>text
</span><span class="line">yolo_emotions <span class="o"><-</span> classify_emotion<span class="p">(</span>yolo_tweets<span class="p">)</span>
</span><span class="line">yolo_polarity <span class="o"><-</span> classify_polarity<span class="p">(</span>yolo_tweets<span class="p">)</span>
</span><span class="line">yolo_polarity.new <span class="o"><-</span> <span class="kt">matrix</span><span class="p">(</span>nrow <span class="o">=</span> <span class="m">1500</span><span class="p">,</span> ncol <span class="o">=</span> <span class="m">2</span><span class="p">)</span>
</span><span class="line">yolo_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span> <span class="o"><-</span> yolo_polarity<span class="p">[,</span> <span class="m">4</span><span class="p">]</span>
</span><span class="line">yolo_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">2</span><span class="p">]</span> <span class="o"><-</span> <span class="s">"yolo"</span>
</span><span class="line">
</span><span class="line">fml_tweets <span class="o"><-</span> searchTwitter<span class="p">(</span><span class="s">"#fml"</span><span class="p">,</span> n <span class="o">=</span> <span class="m">1500</span><span class="p">)</span>
</span><span class="line">fml_tweets <span class="o"><-</span> twListToDF<span class="p">(</span>fml_tweets<span class="p">)</span>
</span><span class="line">fml_tweets <span class="o"><-</span> fml_tweets<span class="o">$</span>text
</span><span class="line">fml_emotions <span class="o"><-</span> classify_emotion<span class="p">(</span>fml_tweets<span class="p">)</span>
</span><span class="line">fml_polarity <span class="o"><-</span> classify_polarity<span class="p">(</span>fml_tweets<span class="p">)</span>
</span><span class="line">fml_polarity.new <span class="o"><-</span> <span class="kt">matrix</span><span class="p">(</span>nrow <span class="o">=</span> <span class="m">1500</span><span class="p">,</span> ncol <span class="o">=</span> <span class="m">2</span><span class="p">)</span>
</span><span class="line">fml_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span> <span class="o"><-</span> fml_polarity<span class="p">[,</span> <span class="m">4</span><span class="p">]</span>
</span><span class="line">fml_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">2</span><span class="p">]</span> <span class="o"><-</span> <span class="s">"fml"</span>
</span><span class="line">
</span><span class="line">
</span><span class="line">blessed_tweets <span class="o"><-</span> searchTwitter<span class="p">(</span><span class="s">"#blessed"</span><span class="p">,</span> n <span class="o">=</span> <span class="m">1500</span><span class="p">)</span>
</span><span class="line">blessed_tweets <span class="o"><-</span> twListToDF<span class="p">(</span>blessed_tweets<span class="p">)</span>
</span><span class="line">blessed_tweets <span class="o"><-</span> blessed_tweets<span class="o">$</span>text
</span><span class="line">blessed_emotions <span class="o"><-</span> classify_emotion<span class="p">(</span>blessed_tweets<span class="p">)</span>
</span><span class="line">blessed_polarity <span class="o"><-</span> classify_polarity<span class="p">(</span>blessed_tweets<span class="p">)</span>
</span><span class="line">blessed_polarity.new <span class="o"><-</span> <span class="kt">matrix</span><span class="p">(</span>nrow <span class="o">=</span> <span class="m">1500</span><span class="p">,</span> ncol <span class="o">=</span> <span class="m">2</span><span class="p">)</span>
</span><span class="line">blessed_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span> <span class="o"><-</span> blessed_polarity<span class="p">[,</span> <span class="m">4</span><span class="p">]</span>
</span><span class="line">blessed_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">2</span><span class="p">]</span> <span class="o"><-</span> <span class="s">"blessed"</span>
</span><span class="line">
</span><span class="line">
</span><span class="line">bacon_tweets <span class="o"><-</span> searchTwitter<span class="p">(</span><span class="s">"#bacon"</span><span class="p">,</span> n <span class="o">=</span> <span class="m">1500</span><span class="p">)</span>
</span><span class="line">bacon_tweets <span class="o"><-</span> twListToDF<span class="p">(</span>bacon_tweets<span class="p">)</span>
</span><span class="line">bacon_tweets <span class="o"><-</span> bacon_tweets<span class="o">$</span>text
</span><span class="line">bacon_emotions <span class="o"><-</span> classify_emotion<span class="p">(</span>bacon_tweets<span class="p">)</span>
</span><span class="line">bacon_polarity <span class="o"><-</span> classify_polarity<span class="p">(</span>bacon_tweets<span class="p">)</span>
</span><span class="line">bacon_polarity.new <span class="o"><-</span> <span class="kt">matrix</span><span class="p">(</span>nrow <span class="o">=</span> <span class="m">1500</span><span class="p">,</span> ncol <span class="o">=</span> <span class="m">2</span><span class="p">)</span>
</span><span class="line">bacon_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span> <span class="o"><-</span> bacon_polarity<span class="p">[,</span> <span class="m">4</span><span class="p">]</span>
</span><span class="line">bacon_polarity.new<span class="p">[</span><span class="m">1</span><span class="o">:</span><span class="m">1500</span><span class="p">,</span> <span class="m">2</span><span class="p">]</span> <span class="o"><-</span> <span class="s">"bacon"</span>
</span><span class="line">
</span><span class="line">
</span><span class="line">polarities <span class="o"><-</span> <span class="kp">rbind</span><span class="p">(</span>yolo_polarity.new<span class="p">,</span> fml_polarity.new<span class="p">,</span>
</span><span class="line">                    blessed_polarity.new<span class="p">,</span> bacon_polarity.new<span class="p">)</span>
</span><span class="line"><span class="kp">colnames</span><span class="p">(</span>polarities<span class="p">)</span> <span class="o"><-</span> <span class="kt">c</span><span class="p">(</span><span class="s">"Polarities"</span><span class="p">,</span> <span class="s">"Hashtag"</span><span class="p">)</span>
</span><span class="line">
</span><span class="line">qplot<span class="p">(</span>polarities<span class="p">[,</span> <span class="m">2</span><span class="p">],</span> fill <span class="o">=</span> polarities<span class="p">[,</span> <span class="m">1</span><span class="p">])</span> <span class="o">+</span> xlab<span class="p">(</span><span class="s">"Hashtags"</span><span class="p">)</span> <span class="o">+</span>
</span><span class="line">      scale_fill_discrete<span class="p">(</span>name <span class="o">=</span> <span class="s">"Text Polarity"</span><span class="p">)</span> <span class="o">+</span>
</span><span class="line">      ggtitle<span class="p">(</span><span class="s">"Polarities of Different Hashtags on Twitter"</span><span class="p">)</span>
</span>

The histogram portrays some peculiar information. For one, all of these hashtags seemed to associate with positive messages. I did not expect #fml to be associated with positive results since the fmylife site is a place for people to post negative things that have happened to them. Nevertheless, the other hashtags had more positives which was expected.

Next, I decided to explore some of the functions of the ‘wordcloud’ package in R. In order to do so, I mined tweets that contained #rstats, and built a wordcloud that sized and placed words based on their frequencies.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<span class="line">rstats_tweets <span class="o"><-</span> searchTwitter<span class="p">(</span><span class="s">"#rstats"</span><span class="p">,</span> n <span class="o">=</span> <span class="m">1500</span><span class="p">)</span>
</span><span class="line">rstats_tweets <span class="o"><-</span> twListToDF<span class="p">(</span>rstats_tweets<span class="p">)</span>
</span><span class="line">rstats_tweets <span class="o"><-</span> rstats_tweets<span class="o">$</span>text
</span><span class="line">
</span><span class="line">
</span><span class="line">rstats_corpus <span class="o"><-</span> Corpus<span class="p">(</span>VectorSource<span class="p">(</span>rstats_tweets<span class="p">))</span>
</span><span class="line">rstats_corpus <span class="o"><-</span> tm_map<span class="p">(</span>rstats_corpus<span class="p">,</span>
</span><span class="line">                        <span class="kr">function</span><span class="p">(</span>x<span class="p">)</span> <span class="kp">iconv</span><span class="p">(</span><span class="kp">enc2utf8</span><span class="p">(</span>x<span class="p">),</span> sub <span class="o">=</span> <span class="s">"byte"</span><span class="p">))</span>
</span><span class="line">
</span><span class="line">tdm <span class="o"><-</span> TermDocumentMatrix<span class="p">(</span>rstats_corpus<span class="p">,</span>
</span><span class="line">                          control <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>removePunctuation <span class="o">=</span> <span class="kc">TRUE</span><span class="p">,</span>
</span><span class="line">                                         stopwords <span class="o">=</span> <span class="kt">c</span><span class="p">(</span><span class="s">"rstats"</span><span class="p">,</span>
</span><span class="line">                                                       stopwords<span class="p">(</span><span class="s">"english"</span><span class="p">)),</span>
</span><span class="line">                                         removeNumbers <span class="o">=</span> <span class="kc">TRUE</span><span class="p">,</span>
</span><span class="line">                                         tolower <span class="o">=</span> <span class="kc">TRUE</span><span class="p">))</span>
</span><span class="line">
</span><span class="line">m <span class="o">=</span> <span class="kp">as.matrix</span><span class="p">(</span>tdm<span class="p">)</span>
</span><span class="line">
</span><span class="line">word_freqs <span class="o">=</span> <span class="kp">sort</span><span class="p">(</span><span class="kp">rowSums</span><span class="p">(</span>m<span class="p">),</span> decreasing<span class="o">=</span><span class="kc">TRUE</span><span class="p">)</span>
</span><span class="line">
</span><span class="line">dm <span class="o">=</span> <span class="kt">data.frame</span><span class="p">(</span>word<span class="o">=</span><span class="kp">names</span><span class="p">(</span>word_freqs<span class="p">),</span> freq<span class="o">=</span>word_freqs<span class="p">)</span>
</span><span class="line">
</span><span class="line">wordcloud<span class="p">(</span>dm<span class="o">$</span>word<span class="p">,</span> dm<span class="o">$</span>freq<span class="p">,</span> random.order<span class="o">=</span><span class="kc">FALSE</span><span class="p">,</span> colors<span class="o">=</span>brewer.pal<span class="p">(</span><span class="m">8</span><span class="p">,</span> <span class="s">"Dark2"</span><span class="p">))</span>
</span>

I had to use the tm_map function to ensure all tweets were encoded properly, before using TermDocumentMatrix. As we can see, ‘shiny’ is by far the most popular word tweeted with ‘#rstats’. This should come as no surprise – Shiny is RStudio’s new and exciting way to integrate R with web applications.

To leave a comment for the author, please follow the link and comment on their blog: Frank Portman.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)