# How divided is the Senate?

July 29, 2013
By

(This article was first published on Category: R | Vik's Blog, and kindly contributed to R-bloggers)

I very seldom pay attention to politics directly, because politics have always seemed a bit circular and cyclical to me. Most of the political news that I take in ends up worming its way into the news sources that I do consume, like the excellent longform.org. Even given my limited intake of political news, one trend that I have noticed lately is the increasing number of references to the Senate as “polarized” or “divided.” Here is a link to an interesting series of charts on polarization. Is it possible to quantify this polarization? Can quantifying the polarization enable us to draw interesting conclusions?

As I started to walk down this road, I figured that it would be tough to find the data that I needed. My time in the US foreign service showed me just how slow the government can be at effectively publishing and using data. Imagine my surprise when I found that the senate website has a very convenient listing of all of the votes from the 101st congress to the 113th (current) congress. This data tells us, for each vote, whether each senator voted yes, no, or abstained.

From the vote data, we can generate plots showing how polarized the Senate is. We will assume that two people are not polarized if they have similar voting patterns. If we take only this vote, we would assume that Senator Ayotte and Senator Alexander, who both voted no, are not polarized, as they share the same opinion. This is well and good, but one bill isn’t really reflective of the voting records of the two Senators. If we really want to figure out where they stand, we would need to perform the analysis across all votes. I will describe the process further down, but for now, let’s jump to a polarization chart:

The above chart has a dot for each Senator, although only some senators are labelled due to space constraints. The further apart the dots are, the more the views of the two senators contrast. Dots are shaded by political affiliation. How can we generate this chart? Keep reading to find out.

## Getting senate data

The first thing that we need to do is get the Senate data. We can start on this page. We see a roll call table in the bottom right. Each roll call table has a listing of each vote in a given congress:

If we click on the vote, we can see the results of the vote.

We can easily write a web scraper, such as the one i wrote to grab this data and store it. This will give us a file of all the voting data.

We can read the file into R (see this script), which gives us a list of lists containing all the vote information. Here is an excerpt:

“`r

senate[[2]]
\$number
[1] “00187”

\$session
[1] “1”

\$data

``````  Coons (D-DE),       McCain (R-AZ),    Chambliss (R-GA),      Franken (D-MN),       Inhofe (R-OK),      Johnson (D-SD),       Tester (D-MT),       Carper (D-DE),
"Yea"                "Nay"                "Nay"                "Yea"                "Nay"                "Yea"                "Yea"                "Yea"
Schumer (D-NY),   Gillibrand (D-NY),      Fischer (R-NE),       Bennet (D-CO),     Barrasso (R-WY),      Shaheen (D-NH),      Boozman (R-AR),         Kirk (R-IL),
"Yea"                "Yea"                "Nay"                "Yea"                "Nay"                "Yea"                "Nay"                "Nay"
``````

“`

We can reformat this into a dataframe that has a record for each vote by each senator, and the congress/session/vote metadata:

sen vote congress number session
1 Coons (D-DE) Yea 113 188 1
2 McCain (R-AZ) Yea 113 188 1
3 Chambliss (R-GA) Yea 113 188 1
4 Franken (D-MN) Yea 113 188 1
5 Inhofe (R-OK) Yea 113 188 1
6 Johnson (D-SD) Yea 113 188 1
7 Tester (D-MT) Yea 113 188 1
8 Carper (D-DE) Yea 113 188 1
9 Schumer (D-NY) Yea 113 188 1
10 Gillibrand (D-NY) Yea 113 188 1

This is just a short excerpt from the actual dataframe, which has `798,835` individual vote records.

## Generating a vote matrix

Now we have a long list of all the votes by the senators, but we really want something like this:

“`

``````           Vote1      Vote2      Vote3
``````

Senator1 (Yes/No) (Yes/No) (Yes/No)
Senator2 (Yes/No) (Yes/No) (Yes/No)
Senator3 (Yes/No) (Yes/No) (Yes/No)
“`

To accomplish this, we will reformat our data by looping through each congress, then looping through each session in the congress, then looping through each vote in the session, and extracting the vote information.

We end up with this:

X.113.1.1. X.113.1.2. X.113.1.3. X.113.1.4. X.113.1.5. X.113.1.188. name party state
Coons (D-DE) 1 1 0 1 1 1 Coons D DE
McCain (R-AZ) 1 1 0 0 1 1 McCain R AZ
Chambliss (R-GA) 2 2 1 0 1 1 Chambliss R GA
Franken (D-MN) 1 1 0 1 1 1 Franken D MN
Inhofe (R-OK) 1 1 0 0 0 1 Inhofe R OK
Johnson (D-SD) 0 0 0 1 1 1 Johnson D SD
Tester (D-MT) 1 1 0 1 1 1 Tester D MT
Carper (D-DE) 1 1 0 1 1 1 Carper D DE
Schumer (D-NY) 0 0 0 1 1 1 Schumer D NY
Gillibrand (D-NY) 2 2 0 1 1 1 Gillibrand D NY

The above is an excerpt, so we are missing a lot of columns. As you can see, the leading column names are in the format congress.session.vote.

## Decomposing vote matrix

Once we have a vote matrix, we can use singular value decomposition to reduce the vote matrix to two dimensions so that we can plot points for each senator. SVD works by trying to combine information (variance) from the multiple columns into less columns.

We end up with this for the 113th Congress:

x y label_code label state name full_name
1 -0.0348802242728007 -0.0885011150722473 1 D DE Coons Coons (D-DE)
2 0.0139333391911509 0.0619623474516745 3 R AZ McCain McCain (R-AZ)
3 -0.00545157078068947 0.10221895824691 3 R GA Chambliss Chambliss (R-GA)
4 -0.0362627872805723 -0.0939015240228452 1 D MN Franken Franken (D-MN)
5 -0.0077981548158297 0.142184554607829 3 R OK Inhofe Inhofe (R-OK)
6 -0.0426066858139772 -0.0904442823377511 1 D SD Johnson Johnson (D-SD)
7 -0.044554280398656 -0.0545406205845917 1 D MT Tester Tester (D-MT)
8 -0.0356973327991686 -0.0826384224297392 1 D DE Carper Carper (D-DE)
9 -0.0371022993314578 -0.0968419990947506 1 D NY Schumer Schumer (D-NY)
10 -0.0331094361241657 -0.0970600008256449 1 D NY Gillibrand Gillibrand (D-NY)
11 -0.0156804216297059 0.117291940698239 3 R NE Fischer Fischer (R-NE)
12 -0.0289778208474004 -0.0824901409317521 1 D CO Bennet Bennet (D-CO)
13 -0.0180558122137665 0.136687567805345 3 R WY Barrasso Barrasso (R-WY)
14 -0.0364473257536752 -0.0839036582733608 1 D NH Shaheen Shaheen (D-NH)
15 -0.00815268808751543 0.113170007558517 3 R AR Boozman Boozman (R-AR)
16 -0.0216403625336666 0.0719986192299611 3 R IL Kirk Kirk (R-IL)
17 -0.029568048844845 -0.0858533841278676 1 D FL Nelson Nelson (D-FL)
18 -0.016413732654005 0.128746686935355 3 R TX Cornyn Cornyn (R-TX)
19 -0.0298919244925452 -0.0880029993263122 1 D MN Klobuchar Klobuchar (D-MN)
20 0.0129837663631103 0.081881278983249 3 R AZ Flake Flake (R-AZ)
21 0.00426679790452548 0.0906706864320044 3 R NE Johanns Johanns (R-NE)
22 0.00273613995205289 0.108642449568633 3 R KS Moran Moran (R-KS)
23 -0.019258803509291 0.13062638910029 3 R IA Grassley Grassley (R-IA)
24 0.547215825548056 -0.0593289149366795 1 D MA Markey Markey (D-MA)
25 -0.0353227186543595 -0.0976376341230047 1 D HI Schatz Schatz (D-HI)
26 -0.00956413282205122 0.129548524717885 3 R ID Risch Risch (R-ID)
27 -0.0229079628265486 -0.0869471509735395 1 D PA Casey Casey (D-PA)
28 -0.0115860482094747 0.131875057650696 3 R WY Enzi Enzi (R-WY)
29 0.00411918634805181 0.09322702104917 3 R MS Wicker Wicker (R-MS)
30 -0.026944901403195 -0.0707999020236366 2 I ME King King (I-ME)
31 -0.0361826736711891 -0.0964110651487557 1 D WI Baldwin Baldwin (D-WI)
32 -0.0268215357509806 -0.0890432282928414 1 D OR Wyden Wyden (D-OR)
33 -0.0122248435323734 -0.0754120081322735 1 D AK Begich Begich (D-AK)
34 -0.0245751728057898 0.141367663303047 3 R KS Roberts Roberts (R-KS)
35 -0.00345843619743554 0.113606389164264 3 R NC Burr Burr (R-NC)
36 -0.0292497277663385 -0.085407101553328 1 D NM Heinrich Heinrich (D-NM)
37 -0.0394437787124538 -0.10050126652756 1 D HI Hirono Hirono (D-HI)
38 -0.0192640058583086 -0.0370242407531159 1 D AR Pryor Pryor (D-AR)
39 -0.0430280450634182 -0.0933459229654493 1 D VT Leahy Leahy (D-VT)
40 -0.0109758213217893 0.103451003006734 3 R NH Ayotte Ayotte (R-NH)
41 -0.0404919136594251 -0.0943042109057099 1 D MI Stabenow Stabenow (D-MI)
42 -0.0273071654738363 -0.0924267270931892 1 D MD Mikulski Mikulski (D-MD)
43 -0.00834005326900628 0.11718205053358 3 R NV Heller Heller (R-NV)
44 -0.0228120494502621 -0.0501247061910197 1 D NC Hagan Hagan (D-NC)
45 -0.0398873356531087 -0.0933244585933005 1 D IL Durbin Durbin (D-IL)
46 -0.0381855188001876 -0.086229921349195 1 D OR Merkley Merkley (D-OR)
47 -0.0195976054340086 -0.091788450374095 1 D CO Udall Udall (D-CO)
48 -0.00926246130655879 -0.0188892715011417 1 D WV Manchin Manchin (D-WV)
49 0.00966111542420832 0.0914499893026106 3 R ND Hoeven Hoeven (R-ND)
50 0.0031092086697181 0.0851622652898145 3 R MO Blunt Blunt (R-MO)
51 -0.00562041044072087 0.0103468267205128 3 R ME Collins Collins (R-ME)
52 -0.00825702591654079 0.127243594535592 3 R PA Toomey Toomey (R-PA)
53 0.0250101385807152 0.0895482161472582 3 R TN Alexander Alexander (R-TN)
54 -0.0248599821563081 -0.095729706434113 1 D RI Whitehouse Whitehouse (D-RI)
55 -0.0333199376653913 -0.0972202109926888 1 D WA Cantwell Cantwell (D-WA)
56 -0.0237285974111554 0.1426337409077 3 R SC Scott Scott (R-SC)
57 0.00208864306527005 0.0880785386131045 3 R MS Cochran Cochran (R-MS)
58 0.00510452859342998 0.109059248449882 3 R IN Coats Coats (R-IN)
59 -0.0325060738924746 0.148436542924344 3 R TX Cruz Cruz (R-TX)
60 -0.0168944687463986 -0.100169858331491 1 D MA Warren Warren (D-MA)
61 -0.00504985813984682 0.116103678690317 3 R SD Thune Thune (R-SD)
62 0.00093559740162602 0.113313222911704 3 R AL Shelby Shelby (R-AL)
63 -0.00465838024989644 0.134113692758071 3 R OK Coburn Coburn (R-OK)
64 -0.00664117170127769 0.10622730554457 3 R UT Hatch Hatch (R-UT)
65 -0.00500337050624833 0.110290478839911 3 R OH Portman Portman (R-OH)
66 -0.0372190241620294 0.126847954536274 3 R UT Lee Lee (R-UT)
67 -0.0131122681852286 0.136328117053898 3 R WI Johnson Johnson (R-WI)
68 -0.0325452232363857 -0.0989242984488763 1 D CT Blumenthal Blumenthal (D-CT)
69 -0.0325608835753324 -0.0965579874994683 1 D MD Cardin Cardin (D-MD)
70 -0.0362197521229333 -0.0885084465604916 1 D NV Reid Reid (D-NV)
71 -0.0377614697054327 -0.0537364230302328 1 D MT Baucus Baucus (D-MT)
72 -0.0231248134323342 -0.0690827754890821 1 D LA Landrieu Landrieu (D-LA)
73 -0.0360776026332512 -0.0966153524943709 1 D RI Reed Reed (D-RI)
74 -0.0334062122015807 -0.0968346202863056 1 D CT Murphy Murphy (D-CT)
75 -0.0386494891823663 -0.0924318691119419 1 D NM Udall Udall (D-NM)
76 -0.034261765962939 -0.0971306029763209 1 D NJ Menendez Menendez (D-NJ)
77 0.0044452992945155 0.0906954517520155 3 R TN Corker Corker (R-TN)
78 -0.0352178910737286 -0.10132777843478 1 D OH Brown Brown (D-OH)
79 -0.0338688838171066 -0.0905868378033902 1 D CA Feinstein Feinstein (D-CA)
80 -0.0136465202405107 0.125521018575589 3 R AL Sessions Sessions (R-AL)
81 -0.0198635645247844 -0.0592441352221849 1 D ND Heitkamp Heitkamp (D-ND)
82 0.00851703548561369 0.11183895075283 3 R LA Vitter Vitter (R-LA)
83 -0.019576243340423 -0.0481775344083724 1 D IN Donnelly Donnelly (D-IN)
84 -0.00163774827120045 0.0936077455073306 3 R GA Isakson Isakson (R-GA)
86 -0.0357000480753136 -0.0735127065323662 1 D VA Warner Warner (D-VA)
87 -0.0292309014580987 -0.0828483216919547 1 D IA Harkin Harkin (D-IA)
88 0.00966030494209233 0.0636741890603597 3 R SC Graham Graham (R-SC)
89 -0.0222634405364073 0.112531903150154 3 R FL Rubio Rubio (R-FL)
90 -0.0299927709252352 -0.0702086799731241 1 D VA Kaine Kaine (D-VA)
91 -0.01976308643714 -0.107959054848897 1 D CA Boxer Boxer (D-CA)
92 -0.0119432394768267 -0.106879637174371 1 D WA Murray Murray (D-WA)
93 -0.0158697808112034 0.128969947769056 3 R ID Crapo Crapo (R-ID)
94 -0.0306370175141373 -0.095789033771327 1 D WV Rockefeller Rockefeller (D-WV)
95 0.0190442180517968 0.012248461541754 3 R AK Murkowski Murkowski (R-AK)
96 -0.0448332633666639 -0.092421654963586 2 I VT Sanders Sanders (I-VT)
97 -0.0242713912858675 0.126333955122549 3 R KY Paul Paul (R-KY)
98 0.491565045440006 0.00808162827350793 3 R NJ Chiesa Chiesa (R-NJ)
99 -0.0213028527791772 0.132160347234302 3 R KY McConnell McConnell (R-KY)
100 -0.0415935326847892 -0.0973163163238865 1 D MI Levin Levin (D-MI)
101 0.551432788755709 -0.071981264312893 1 D MA Kerry Kerry (D-MA)
102 0.301028043276857 -0.0845041483494266 1 D NJ Lautenberg Lautenberg (D-NJ)
103 0.0222177184793535 -0.163230940742793 1 D MA Cowan Cowan (D-MA)

`x` and `y` are our two dimensional singular values that represent our vote matrices. `label` is the party of the senator. `label_code` is the numeric representation of the party (1 is Democrat, 3 is Republican, 2 is Independent). ``state` is the state the senator is from.

Once we have these singular values, we can use them to plot our original chart:

## Interesting observations

• From the chart, we can see that there is significant polarization in the Senate. In fact, there is a dividing line between the two parties.
• Both independents seem to vote solidly democrat.
• Massachussetts has some really out there senators (full disclosure: I live in MA right now)
• So does New Jersey
• Collins (R-ME), Murkowski (R-AK), Chiesa (R-NJ), Machin (D-WV), and Pryor (D-AR), are the closest things to centrists in the Senate.
• There are solid voting clusters around the party leaderships of both parties.
• The party line seems to come before all else, judging by how closely voting aligns by party.

There are other interesting things in this chart. Feel free to let me know if you notice anything good.

## But wait, there’s more!

Now that we have these vote matrices, we can do all manner of cool things. One of the cool things we can do is calculate the euclidean distance between the votes of each Senator and the average votes on all issues. The greater the distance, the more “radical”, or extreme in their views, a senator is.

Here are all the senators, this time sorted by their distances:

x y label_code label state name full_name distances
101 0.551432788755709 -0.071981264312893 1 D MA Kerry Kerry (D-MA) 0.564834814542764
24 0.547215825548056 -0.0593289149366795 1 D MA Markey Markey (D-MA) 0.559309579320913
98 0.491565045440006 0.00808162827350793 3 R NJ Chiesa Chiesa (R-NJ) 0.50402702343082
102 0.301028043276857 -0.0845041483494266 1 D NJ Lautenberg Lautenberg (D-NJ) 0.339392958602243
103 0.0222177184793535 -0.163230940742793 1 D MA Cowan Cowan (D-MA) 0.186545210134961
59 -0.0325060738924746 0.148436542924344 3 R TX Cruz Cruz (R-TX) 0.165896480749798
56 -0.0237285974111554 0.1426337409077 3 R SC Scott Scott (R-SC) 0.158640662947416
34 -0.0245751728057898 0.141367663303047 3 R KS Roberts Roberts (R-KS) 0.157729205632881
5 -0.0077981548158297 0.142184554607829 3 R OK Inhofe Inhofe (R-OK) 0.157161641390102
13 -0.0180558122137665 0.136687567805345 3 R WY Barrasso Barrasso (R-WY) 0.152621988932664
67 -0.0131122681852286 0.136328117053898 3 R WI Johnson Johnson (R-WI) 0.151926482983146
66 -0.0372190241620294 0.126847954536274 3 R UT Lee Lee (R-UT) 0.151014401165793
63 -0.00465838024989644 0.134113692758071 3 R OK Coburn Coburn (R-OK) 0.150563032650489
99 -0.0213028527791772 0.132160347234302 3 R KY McConnell McConnell (R-KY) 0.149572657072359
28 -0.0115860482094747 0.131875057650696 3 R WY Enzi Enzi (R-WY) 0.148273587508265
23 -0.019258803509291 0.13062638910029 3 R IA Grassley Grassley (R-IA) 0.148018476577859
26 -0.00956413282205122 0.129548524717885 3 R ID Risch Risch (R-ID) 0.146533324110168
93 -0.0158697808112034 0.128969947769056 3 R ID Crapo Crapo (R-ID) 0.146312632504688
97 -0.0242713912858675 0.126333955122549 3 R KY Paul Paul (R-KY) 0.146223353635669
18 -0.016413732654005 0.128746686935355 3 R TX Cornyn Cornyn (R-TX) 0.146216404936958
52 -0.00825702591654079 0.127243594535592 3 R PA Toomey Toomey (R-PA) 0.144926057670348
80 -0.0136465202405107 0.125521018575589 3 R AL Sessions Sessions (R-AL) 0.143756581273559
11 -0.0156804216297059 0.117291940698239 3 R NE Fischer Fischer (R-NE) 0.138960750355783
43 -0.00834005326900628 0.11718205053358 3 R NV Heller Heller (R-NV) 0.138362287865482
82 0.00851703548561369 0.11183895075283 3 R LA Vitter Vitter (R-LA) 0.138184156829855
61 -0.00504985813984682 0.116103678690317 3 R SD Thune Thune (R-SD) 0.137854672693276
89 -0.0222634405364073 0.112531903150154 3 R FL Rubio Rubio (R-FL) 0.137806098357527
62 0.00093559740162602 0.113313222911704 3 R AL Shelby Shelby (R-AL) 0.137064565612541
35 -0.00345843619743554 0.113606389164264 3 R NC Burr Burr (R-NC) 0.136557433557291
15 -0.00815268808751543 0.113170007558517 3 R AR Boozman Boozman (R-AR) 0.136101853780071
58 0.00510452859342998 0.109059248449882 3 R IN Coats Coats (R-IN) 0.135690894512148
22 0.00273613995205289 0.108642449568633 3 R KS Moran Moran (R-KS) 0.134923581714883
65 -0.00500337050624833 0.110290478839911 3 R OH Portman Portman (R-OH) 0.134686530230351
53 0.0250101385807152 0.0895482161472582 3 R TN Alexander Alexander (R-TN) 0.134450487044903
64 -0.00664117170127769 0.10622730554457 3 R UT Hatch Hatch (R-UT) 0.132740120998976
40 -0.0109758213217893 0.103451003006734 3 R NH Ayotte Ayotte (R-NH) 0.131773301516089
3 -0.00545157078068947 0.10221895824691 3 R GA Chambliss Chambliss (R-GA) 0.131093015114917
92 -0.0119432394768267 -0.106879637174371 1 D WA Murray Murray (D-WA) 0.130153978588983
49 0.00966111542420832 0.0914499893026106 3 R ND Hoeven Hoeven (R-ND) 0.129407219690427
91 -0.01976308643714 -0.107959054848897 1 D CA Boxer Boxer (D-CA) 0.129264357312157
29 0.00411918634805181 0.09322702104917 3 R MS Wicker Wicker (R-MS) 0.128684118769704
84 -0.00163774827120045 0.0936077455073306 3 R GA Isakson Isakson (R-GA) 0.128132382150922
77 0.0044452992945155 0.0906954517520155 3 R TN Corker Corker (R-TN) 0.127877085086648
21 0.00426679790452548 0.0906706864320044 3 R NE Johanns Johanns (R-NE) 0.127836560969117
20 0.0129837663631103 0.081881278983249 3 R AZ Flake Flake (R-AZ) 0.127583769293001
57 0.00208864306527005 0.0880785386131045 3 R MS Cochran Cochran (R-MS) 0.126766732463057
50 0.0031092086697181 0.0851622652898145 3 R MO Blunt Blunt (R-MO) 0.126156637213949
16 -0.0216403625336666 0.0719986192299611 3 R IL Kirk Kirk (R-IL) 0.124078130072497
60 -0.0168944687463986 -0.100169858331491 1 D MA Warren Warren (D-MA) 0.124001435882044
37 -0.0394437787124538 -0.10050126652756 1 D HI Hirono Hirono (D-HI) 0.123889758913328
2 0.0139333391911509 0.0619623474516745 3 R AZ McCain McCain (R-AZ) 0.123705285262252
78 -0.0352178910737286 -0.10132777843478 1 D OH Brown Brown (D-OH) 0.123412758640486
88 0.00966030494209233 0.0636741890603597 3 R SC Graham Graham (R-SC) 0.122980572370755
100 -0.0415935326847892 -0.0973163163238865 1 D MI Levin Levin (D-MI) 0.122403925388497
68 -0.0325452232363857 -0.0989242984488763 1 D CT Blumenthal Blumenthal (D-CT) 0.121121264263079
96 -0.0448332633666639 -0.092421654963586 2 I VT Sanders Sanders (I-VT) 0.121104059523429
39 -0.0430280450634182 -0.0933459229654493 1 D VT Leahy Leahy (D-VT) 0.120661628709076
25 -0.0353227186543595 -0.0976376341230047 1 D HI Schatz Schatz (D-HI) 0.120516995483394
9 -0.0371022993314578 -0.0968419990947506 1 D NY Schumer Schumer (D-NY) 0.120411986046654
41 -0.0404919136594251 -0.0943042109057099 1 D MI Stabenow Stabenow (D-MI) 0.120043219408707
73 -0.0360776026332512 -0.0966153524943709 1 D RI Reed Reed (D-RI) 0.119977187394233
76 -0.034261765962939 -0.0971306029763209 1 D NJ Menendez Menendez (D-NJ) 0.119965878040237
55 -0.0333199376653913 -0.0972202109926888 1 D WA Cantwell Cantwell (D-WA) 0.119905053274074
31 -0.0361826736711891 -0.0964110651487557 1 D WI Baldwin Baldwin (D-WI) 0.119870079027214
10 -0.0331094361241657 -0.0970600008256449 1 D NY Gillibrand Gillibrand (D-NY) 0.119775857565045
74 -0.0334062122015807 -0.0968346202863056 1 D CT Murphy Murphy (D-CT) 0.119651535736152
69 -0.0325608835753324 -0.0965579874994683 1 D MD Cardin Cardin (D-MD) 0.119412340011159
54 -0.0248599821563081 -0.095729706434113 1 D RI Whitehouse Whitehouse (D-RI) 0.119390224208126
95 0.0190442180517968 0.012248461541754 3 R AK Murkowski Murkowski (R-AK) 0.119338650035987
45 -0.0398873356531087 -0.0933244585933005 1 D IL Durbin Durbin (D-IL) 0.119264416633329
6 -0.0426066858139772 -0.0904442823377511 1 D SD Johnson Johnson (D-SD) 0.119082469790906
94 -0.0306370175141373 -0.095789033771327 1 D WV Rockefeller Rockefeller (D-WV) 0.118904362580451
4 -0.0362627872805723 -0.0939015240228452 1 D MN Franken Franken (D-MN) 0.118425828858068
75 -0.0386494891823663 -0.0924318691119419 1 D NM Udall Udall (D-NM) 0.118366034195972
47 -0.0195976054340086 -0.091788450374095 1 D CO Udall Udall (D-CO) 0.118236088253773
42 -0.0273071654738363 -0.0924267270931892 1 D MD Mikulski Mikulski (D-MD) 0.117150656585951
79 -0.0338688838171066 -0.0905868378033902 1 D CA Feinstein Feinstein (D-CA) 0.116358543323594
70 -0.0362197521229333 -0.0885084465604916 1 D NV Reid Reid (D-NV) 0.115971985332404
46 -0.0381855188001876 -0.086229921349195 1 D OR Merkley Merkley (D-OR) 0.115774234530289
1 -0.0348802242728007 -0.0885011150722473 1 D DE Coons Coons (D-DE) 0.115670517369723
32 -0.0268215357509806 -0.0890432282928414 1 D OR Wyden Wyden (D-OR) 0.115563613050844
27 -0.0229079628265486 -0.0869471509735395 1 D PA Casey Casey (D-PA) 0.115286524384875
19 -0.0298919244925452 -0.0880029993263122 1 D MN Klobuchar Klobuchar (D-MN) 0.114982973280414
14 -0.0364473257536752 -0.0839036582733608 1 D NH Shaheen Shaheen (D-NH) 0.114595708701108
51 -0.00562041044072087 0.0103468267205128 3 R ME Collins Collins (R-ME) 0.114302134456818
33 -0.0122248435323734 -0.0754120081322735 1 D AK Begich Begich (D-AK) 0.114243839017192
17 -0.029568048844845 -0.0858533841278676 1 D FL Nelson Nelson (D-FL) 0.114194289812714
8 -0.0356973327991686 -0.0826384224297392 1 D DE Carper Carper (D-DE) 0.114101444562881
36 -0.0292497277663385 -0.085407101553328 1 D NM Heinrich Heinrich (D-NM) 0.114048209132161
7 -0.044554280398656 -0.0545406205845917 1 D MT Tester Tester (D-MT) 0.113815375991399
87 -0.0292309014580987 -0.0828483216919547 1 D IA Harkin Harkin (D-IA) 0.113321234065552
12 -0.0289778208474004 -0.0824901409317521 1 D CO Bennet Bennet (D-CO) 0.113231028586499
86 -0.0357000480753136 -0.0735127065323662 1 D VA Warner Warner (D-VA) 0.112526922656803
71 -0.0377614697054327 -0.0537364230302328 1 D MT Baucus Baucus (D-MT) 0.111755195704732
48 -0.00926246130655879 -0.0188892715011417 1 D WV Manchin Manchin (D-WV) 0.111750853665797
90 -0.0299927709252352 -0.0702086799731241 1 D VA Kaine Kaine (D-VA) 0.111203430282439
30 -0.026944901403195 -0.0707999020236366 2 I ME King King (I-ME) 0.11110709047993
72 -0.0231248134323342 -0.0690827754890821 1 D LA Landrieu Landrieu (D-LA) 0.111001469145908
81 -0.0198635645247844 -0.0592441352221849 1 D ND Heitkamp Heitkamp (D-ND) 0.110305612650745
38 -0.0192640058583086 -0.0370242407531159 1 D AR Pryor Pryor (D-AR) 0.110231120430787
83 -0.019576243340423 -0.0481775344083724 1 D IN Donnelly Donnelly (D-IN) 0.109949132073788
44 -0.0228120494502621 -0.0501247061910197 1 D NC Hagan Hagan (D-NC) 0.109873667752502

We can also make a graphic of the most “extreme” senators given this distance:

Note that the minority party is more likely to be extreme by this distance metric, because the typical view is titled towards the party with more votes.

## Using historical data

We can also calculate how polarized the parties have been by calculating how “extreme” the average member in each party was at any given time.

We can see how polarization has changed over time, and the average distance of each member to the typical voting pattern has shifted.

## Thoughts

This analysis was interesting to do, and I hope to do more in the future. You can find all of my code here.

Some cautions:

• I would hesitate to make any sweeping generalizations from this that are not supported by the data.
• What you see is pretty much what you get. All of this data is publicly available, and I highly encourage you to look at it if you are interested.

Ideas:

• It could be interesting to analyze voting patterns as compared to the text of bills. (Does senator X always vote for bills with the phrase “increase defense spending” in them?)
• Voting patterns vs demographic shifts in the US.
• Linking voting patterns and the rise and fall of political parties.
• Doing similar analysis for the House.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...