How divided is the Senate?

July 29, 2013
By

(This article was first published on Vik Paruchuri, and kindly contributed to R-bloggers)

I very seldom pay attention to politics directly, because politics have always seemed a bit circular and cyclical to me. Most of the political news that I take in ends up worming its way into the news sources that I do consume, like the excellent longform.org. Even given my limited intake of political news, one trend that I have noticed lately is the increasing number of references to the Senate as "polarized" or "divided." Here is a link to an interesting series of charts on polarization. Is it possible to quantify this polarization? Can quantifying the polarization enable us to draw interesting conclusions?

As I started to walk down this road, I figured that it would be tough to find the data that I needed. My time in the US foreign service showed me just how slow the government can be at effectively publishing and using data. Imagine my surprise when I found that the senate website has a very convenient listing of all of the votes from the 101st congress to the 113th (current) congress. This data tells us, for each vote, whether each senator voted yes, no, or abstained.

From the vote data, we can generate plots showing how polarized the Senate is. We will assume that two people are not polarized if they have similar voting patterns. If we take only this vote, we would assume that Senator Ayotte and Senator Alexander, who both voted no, are not polarized, as they share the same opinion. This is well and good, but one bill isn't really reflective of the voting records of the two Senators. If we really want to figure out where they stand, we would need to perform the analysis across all votes. I will describe the process further down, but for now, let's jump to a polarization chart:

senate polarization

The above chart has a dot for each Senator, although only some senators are labelled due to space constraints. The further apart the dots are, the more the views of the two senators contrast. Dots are shaded by political affiliation. How can we generate this chart? Keep reading to find out.

Getting senate data

The first thing that we need to do is get the Senate data. We can start on this page. We see a roll call table in the bottom right. Each roll call table has a listing of each vote in a given congress:

roll call table

If we click on the vote, we can see the results of the vote.

We can easily write a web scraper, such as the one i wrote to grab this data and store it. This will give us a file of all the voting data.

We can read the file into R (see this script), which gives us a list of lists containing all the vote information. Here is an excerpt:

> senate[[2]]
$number
[1] "00187"

$session
[1] "1"

$data
      Coons (D-DE), McCain (R-AZ), Chambliss (R-GA), Franken (D-MN), Inhofe (R-OK), Johnson (D-SD), Tester (D-MT), Carper (D-DE),
               "Yea" "Nay" "Nay" "Yea" "Nay" "Yea" "Yea" "Yea"
    Schumer (D-NY), Gillibrand (D-NY), Fischer (R-NE), Bennet (D-CO), Barrasso (R-WY), Shaheen (D-NH), Boozman (R-AR), Kirk (R-IL),
               "Yea" "Yea" "Nay" "Yea" "Nay" "Yea" "Nay" "Nay"

We can reformat this into a dataframe that has a record for each vote by each senator, and the congress/session/vote metadata:

senvotecongressnumbersession
1Coons (D-DE)Yea1131881
2McCain (R-AZ)Yea1131881
3Chambliss (R-GA)Yea1131881
4Franken (D-MN)Yea1131881
5Inhofe (R-OK)Yea1131881
6Johnson (D-SD)Yea1131881
7Tester (D-MT)Yea1131881
8Carper (D-DE)Yea1131881
9Schumer (D-NY)Yea1131881
10Gillibrand (D-NY)Yea1131881


This is just a short excerpt from the actual dataframe, which has 798,835 individual vote records.

Generating a vote matrix

Now we have a long list of all the votes by the senators, but we really want something like this:

               Vote1 Vote2 Vote3
Senator1 (Yes/No) (Yes/No) (Yes/No)
Senator2 (Yes/No) (Yes/No) (Yes/No)
Senator3 (Yes/No) (Yes/No) (Yes/No)

To accomplish this, we will reformat our data by looping through each congress, then looping through each session in the congress, then looping through each vote in the session, and extracting the vote information.

We end up with this:

X.113.1.1.X.113.1.2.X.113.1.3.X.113.1.4.X.113.1.5.X.113.1.188.namepartystate
Coons (D-DE)110111CoonsDDE
McCain (R-AZ)110011McCainRAZ
Chambliss (R-GA)221011ChamblissRGA
Franken (D-MN)110111FrankenDMN
Inhofe (R-OK)110001InhofeROK
Johnson (D-SD)000111JohnsonDSD
Tester (D-MT)110111TesterDMT
Carper (D-DE)110111CarperDDE
Schumer (D-NY)000111SchumerDNY
Gillibrand (D-NY)220111GillibrandDNY


The above is an excerpt, so we are missing a lot of columns. As you can see, the leading column names are in the format congress.session.vote.

Decomposing vote matrix

Once we have a vote matrix, we can use singular value decomposition to reduce the vote matrix to two dimensions so that we can plot points for each senator. SVD works by trying to combine information (variance) from the multiple columns into less columns.

We end up with this for the 113th Congress:

xylabel_codelabelstatenamefull_name
1-0.0348802242728007-0.08850111507224731DDECoonsCoons (D-DE)
20.01393333919115090.06196234745167453RAZMcCainMcCain (R-AZ)
3-0.005451570780689470.102218958246913RGAChamblissChambliss (R-GA)
4-0.0362627872805723-0.09390152402284521DMNFrankenFranken (D-MN)
5-0.00779815481582970.1421845546078293ROKInhofeInhofe (R-OK)
6-0.0426066858139772-0.09044428233775111DSDJohnsonJohnson (D-SD)
7-0.044554280398656-0.05454062058459171DMTTesterTester (D-MT)
8-0.0356973327991686-0.08263842242973921DDECarperCarper (D-DE)
9-0.0371022993314578-0.09684199909475061DNYSchumerSchumer (D-NY)
10-0.0331094361241657-0.09706000082564491DNYGillibrandGillibrand (D-NY)
11-0.01568042162970590.1172919406982393RNEFischerFischer (R-NE)
12-0.0289778208474004-0.08249014093175211DCOBennetBennet (D-CO)
13-0.01805581221376650.1366875678053453RWYBarrassoBarrasso (R-WY)
14-0.0364473257536752-0.08390365827336081DNHShaheenShaheen (D-NH)
15-0.008152688087515430.1131700075585173RARBoozmanBoozman (R-AR)
16-0.02164036253366660.07199861922996113RILKirkKirk (R-IL)
17-0.029568048844845-0.08585338412786761DFLNelsonNelson (D-FL)
18-0.0164137326540050.1287466869353553RTXCornynCornyn (R-TX)
19-0.0298919244925452-0.08800299932631221DMNKlobucharKlobuchar (D-MN)
200.01298376636311030.0818812789832493RAZFlakeFlake (R-AZ)
210.004266797904525480.09067068643200443RNEJohannsJohanns (R-NE)
220.002736139952052890.1086424495686333RKSMoranMoran (R-KS)
23-0.0192588035092910.130626389100293RIAGrassleyGrassley (R-IA)
240.547215825548056-0.05932891493667951DMAMarkeyMarkey (D-MA)
25-0.0353227186543595-0.09763763412300471DHISchatzSchatz (D-HI)
26-0.009564132822051220.1295485247178853RIDRischRisch (R-ID)
27-0.0229079628265486-0.08694715097353951DPACaseyCasey (D-PA)
28-0.01158604820947470.1318750576506963RWYEnziEnzi (R-WY)
290.004119186348051810.093227021049173RMSWickerWicker (R-MS)
30-0.026944901403195-0.07079990202363662IMEKingKing (I-ME)
31-0.0361826736711891-0.09641106514875571DWIBaldwinBaldwin (D-WI)
32-0.0268215357509806-0.08904322829284141DORWydenWyden (D-OR)
33-0.0122248435323734-0.07541200813227351DAKBegichBegich (D-AK)
34-0.02457517280578980.1413676633030473RKSRobertsRoberts (R-KS)
35-0.003458436197435540.1136063891642643RNCBurrBurr (R-NC)
36-0.0292497277663385-0.0854071015533281DNMHeinrichHeinrich (D-NM)
37-0.0394437787124538-0.100501266527561DHIHironoHirono (D-HI)
38-0.0192640058583086-0.03702424075311591DARPryorPryor (D-AR)
39-0.0430280450634182-0.09334592296544931DVTLeahyLeahy (D-VT)
40-0.01097582132178930.1034510030067343RNHAyotteAyotte (R-NH)
41-0.0404919136594251-0.09430421090570991DMIStabenowStabenow (D-MI)
42-0.0273071654738363-0.09242672709318921DMDMikulskiMikulski (D-MD)
43-0.008340053269006280.117182050533583RNVHellerHeller (R-NV)
44-0.0228120494502621-0.05012470619101971DNCHaganHagan (D-NC)
45-0.0398873356531087-0.09332445859330051DILDurbinDurbin (D-IL)
46-0.0381855188001876-0.0862299213491951DORMerkleyMerkley (D-OR)
47-0.0195976054340086-0.0917884503740951DCOUdallUdall (D-CO)
48-0.00926246130655879-0.01888927150114171DWVManchinManchin (D-WV)
490.009661115424208320.09144998930261063RNDHoevenHoeven (R-ND)
500.00310920866971810.08516226528981453RMOBluntBlunt (R-MO)
51-0.005620410440720870.01034682672051283RMECollinsCollins (R-ME)
52-0.008257025916540790.1272435945355923RPAToomeyToomey (R-PA)
530.02501013858071520.08954821614725823RTNAlexanderAlexander (R-TN)
54-0.0248599821563081-0.0957297064341131DRIWhitehouseWhitehouse (D-RI)
55-0.0333199376653913-0.09722021099268881DWACantwellCantwell (D-WA)
56-0.02372859741115540.14263374090773RSCScottScott (R-SC)
570.002088643065270050.08807853861310453RMSCochranCochran (R-MS)
580.005104528593429980.1090592484498823RINCoatsCoats (R-IN)
59-0.03250607389247460.1484365429243443RTXCruzCruz (R-TX)
60-0.0168944687463986-0.1001698583314911DMAWarrenWarren (D-MA)
61-0.005049858139846820.1161036786903173RSDThuneThune (R-SD)
620.000935597401626020.1133132229117043RALShelbyShelby (R-AL)
63-0.004658380249896440.1341136927580713ROKCoburnCoburn (R-OK)
64-0.006641171701277690.106227305544573RUTHatchHatch (R-UT)
65-0.005003370506248330.1102904788399113ROHPortmanPortman (R-OH)
66-0.03721902416202940.1268479545362743RUTLeeLee (R-UT)
67-0.01311226818522860.1363281170538983RWIJohnsonJohnson (R-WI)
68-0.0325452232363857-0.09892429844887631DCTBlumenthalBlumenthal (D-CT)
69-0.0325608835753324-0.09655798749946831DMDCardinCardin (D-MD)
70-0.0362197521229333-0.08850844656049161DNVReidReid (D-NV)
71-0.0377614697054327-0.05373642303023281DMTBaucusBaucus (D-MT)
72-0.0231248134323342-0.06908277548908211DLALandrieuLandrieu (D-LA)
73-0.0360776026332512-0.09661535249437091DRIReedReed (D-RI)
74-0.0334062122015807-0.09683462028630561DCTMurphyMurphy (D-CT)
75-0.0386494891823663-0.09243186911194191DNMUdallUdall (D-NM)
76-0.034261765962939-0.09713060297632091DNJMenendezMenendez (D-NJ)
770.00444529929451550.09069545175201553RTNCorkerCorker (R-TN)
78-0.0352178910737286-0.101327778434781DOHBrownBrown (D-OH)
79-0.0338688838171066-0.09058683780339021DCAFeinsteinFeinstein (D-CA)
80-0.01364652024051070.1255210185755893RALSessionsSessions (R-AL)
81-0.0198635645247844-0.05924413522218491DNDHeitkampHeitkamp (D-ND)
820.008517035485613690.111838950752833RLAVitterVitter (R-LA)
83-0.019576243340423-0.04817753440837241DINDonnellyDonnelly (D-IN)
84-0.001637748271200450.09360774550733063RGAIsaksonIsakson (R-GA)
85-0.011292726591166-0.06251294436694251DMOMcCaskillMcCaskill (D-MO)
86-0.0357000480753136-0.07351270653236621DVAWarnerWarner (D-VA)
87-0.0292309014580987-0.08284832169195471DIAHarkinHarkin (D-IA)
880.009660304942092330.06367418906035973RSCGrahamGraham (R-SC)
89-0.02226344053640730.1125319031501543RFLRubioRubio (R-FL)
90-0.0299927709252352-0.07020867997312411DVAKaineKaine (D-VA)
91-0.01976308643714-0.1079590548488971DCABoxerBoxer (D-CA)
92-0.0119432394768267-0.1068796371743711DWAMurrayMurray (D-WA)
93-0.01586978081120340.1289699477690563RIDCrapoCrapo (R-ID)
94-0.0306370175141373-0.0957890337713271DWVRockefellerRockefeller (D-WV)
950.01904421805179680.0122484615417543RAKMurkowskiMurkowski (R-AK)
96-0.0448332633666639-0.0924216549635862IVTSandersSanders (I-VT)
97-0.02427139128586750.1263339551225493RKYPaulPaul (R-KY)
980.4915650454400060.008081628273507933RNJChiesaChiesa (R-NJ)
99-0.02130285277917720.1321603472343023RKYMcConnellMcConnell (R-KY)
100-0.0415935326847892-0.09731631632388651DMILevinLevin (D-MI)
1010.551432788755709-0.0719812643128931DMAKerryKerry (D-MA)
1020.301028043276857-0.08450414834942661DNJLautenbergLautenberg (D-NJ)
1030.0222177184793535-0.1632309407427931DMACowanCowan (D-MA)


x and y are our two dimensional singular values that represent our vote matrices. label is the party of the senator. label_code is the numeric representation of the party (1 is Democrat, 3 is Republican, 2 is Independent). `state is the state the senator is from.

Once we have these singular values, we can use them to plot our original chart:

senate polarization

Interesting observations

  • From the chart, we can see that there is significant polarization in the Senate. In fact, there is a dividing line between the two parties.
  • Both independents seem to vote solidly democrat.
  • Massachussetts has some really out there senators (full disclosure: I live in MA right now)
  • So does New Jersey
  • Collins (R-ME), Murkowski (R-AK), Chiesa (R-NJ), Machin (D-WV), and Pryor (D-AR), are the closest things to centrists in the Senate.
  • There are solid voting clusters around the party leaderships of both parties.
  • The party line seems to come before all else, judging by how closely voting aligns by party.

There are other interesting things in this chart. Feel free to let me know if you notice anything good.

But wait, there's more!

Now that we have these vote matrices, we can do all manner of cool things. One of the cool things we can do is calculate the euclidean distance between the votes of each Senator and the average votes on all issues. The greater the distance, the more "radical", or extreme in their views, a senator is.

Here are all the senators, this time sorted by their distances:

xylabel_codelabelstatenamefull_namedistances
1010.551432788755709-0.0719812643128931DMAKerryKerry (D-MA)0.564834814542764
240.547215825548056-0.05932891493667951DMAMarkeyMarkey (D-MA)0.559309579320913
980.4915650454400060.008081628273507933RNJChiesaChiesa (R-NJ)0.50402702343082
1020.301028043276857-0.08450414834942661DNJLautenbergLautenberg (D-NJ)0.339392958602243
1030.0222177184793535-0.1632309407427931DMACowanCowan (D-MA)0.186545210134961
59-0.03250607389247460.1484365429243443RTXCruzCruz (R-TX)0.165896480749798
56-0.02372859741115540.14263374090773RSCScottScott (R-SC)0.158640662947416
34-0.02457517280578980.1413676633030473RKSRobertsRoberts (R-KS)0.157729205632881
5-0.00779815481582970.1421845546078293ROKInhofeInhofe (R-OK)0.157161641390102
13-0.01805581221376650.1366875678053453RWYBarrassoBarrasso (R-WY)0.152621988932664
67-0.01311226818522860.1363281170538983RWIJohnsonJohnson (R-WI)0.151926482983146
66-0.03721902416202940.1268479545362743RUTLeeLee (R-UT)0.151014401165793
63-0.004658380249896440.1341136927580713ROKCoburnCoburn (R-OK)0.150563032650489
99-0.02130285277917720.1321603472343023RKYMcConnellMcConnell (R-KY)0.149572657072359
28-0.01158604820947470.1318750576506963RWYEnziEnzi (R-WY)0.148273587508265
23-0.0192588035092910.130626389100293RIAGrassleyGrassley (R-IA)0.148018476577859
26-0.009564132822051220.1295485247178853RIDRischRisch (R-ID)0.146533324110168
93-0.01586978081120340.1289699477690563RIDCrapoCrapo (R-ID)0.146312632504688
97-0.02427139128586750.1263339551225493RKYPaulPaul (R-KY)0.146223353635669
18-0.0164137326540050.1287466869353553RTXCornynCornyn (R-TX)0.146216404936958
52-0.008257025916540790.1272435945355923RPAToomeyToomey (R-PA)0.144926057670348
80-0.01364652024051070.1255210185755893RALSessionsSessions (R-AL)0.143756581273559
11-0.01568042162970590.1172919406982393RNEFischerFischer (R-NE)0.138960750355783
43-0.008340053269006280.117182050533583RNVHellerHeller (R-NV)0.138362287865482
820.008517035485613690.111838950752833RLAVitterVitter (R-LA)0.138184156829855
61-0.005049858139846820.1161036786903173RSDThuneThune (R-SD)0.137854672693276
89-0.02226344053640730.1125319031501543RFLRubioRubio (R-FL)0.137806098357527
620.000935597401626020.1133132229117043RALShelbyShelby (R-AL)0.137064565612541
35-0.003458436197435540.1136063891642643RNCBurrBurr (R-NC)0.136557433557291
15-0.008152688087515430.1131700075585173RARBoozmanBoozman (R-AR)0.136101853780071
580.005104528593429980.1090592484498823RINCoatsCoats (R-IN)0.135690894512148
220.002736139952052890.1086424495686333RKSMoranMoran (R-KS)0.134923581714883
65-0.005003370506248330.1102904788399113ROHPortmanPortman (R-OH)0.134686530230351
530.02501013858071520.08954821614725823RTNAlexanderAlexander (R-TN)0.134450487044903
64-0.006641171701277690.106227305544573RUTHatchHatch (R-UT)0.132740120998976
40-0.01097582132178930.1034510030067343RNHAyotteAyotte (R-NH)0.131773301516089
3-0.005451570780689470.102218958246913RGAChamblissChambliss (R-GA)0.131093015114917
92-0.0119432394768267-0.1068796371743711DWAMurrayMurray (D-WA)0.130153978588983
490.009661115424208320.09144998930261063RNDHoevenHoeven (R-ND)0.129407219690427
91-0.01976308643714-0.1079590548488971DCABoxerBoxer (D-CA)0.129264357312157
290.004119186348051810.093227021049173RMSWickerWicker (R-MS)0.128684118769704
84-0.001637748271200450.09360774550733063RGAIsaksonIsakson (R-GA)0.128132382150922
770.00444529929451550.09069545175201553RTNCorkerCorker (R-TN)0.127877085086648
210.004266797904525480.09067068643200443RNEJohannsJohanns (R-NE)0.127836560969117
200.01298376636311030.0818812789832493RAZFlakeFlake (R-AZ)0.127583769293001
570.002088643065270050.08807853861310453RMSCochranCochran (R-MS)0.126766732463057
500.00310920866971810.08516226528981453RMOBluntBlunt (R-MO)0.126156637213949
16-0.02164036253366660.07199861922996113RILKirkKirk (R-IL)0.124078130072497
60-0.0168944687463986-0.1001698583314911DMAWarrenWarren (D-MA)0.124001435882044
37-0.0394437787124538-0.100501266527561DHIHironoHirono (D-HI)0.123889758913328
20.01393333919115090.06196234745167453RAZMcCainMcCain (R-AZ)0.123705285262252
78-0.0352178910737286-0.101327778434781DOHBrownBrown (D-OH)0.123412758640486
880.009660304942092330.06367418906035973RSCGrahamGraham (R-SC)0.122980572370755
100-0.0415935326847892-0.09731631632388651DMILevinLevin (D-MI)0.122403925388497
68-0.0325452232363857-0.09892429844887631DCTBlumenthalBlumenthal (D-CT)0.121121264263079
96-0.0448332633666639-0.0924216549635862IVTSandersSanders (I-VT)0.121104059523429
39-0.0430280450634182-0.09334592296544931DVTLeahyLeahy (D-VT)0.120661628709076
25-0.0353227186543595-0.09763763412300471DHISchatzSchatz (D-HI)0.120516995483394
9-0.0371022993314578-0.09684199909475061DNYSchumerSchumer (D-NY)0.120411986046654
41-0.0404919136594251-0.09430421090570991DMIStabenowStabenow (D-MI)0.120043219408707
73-0.0360776026332512-0.09661535249437091DRIReedReed (D-RI)0.119977187394233
76-0.034261765962939-0.09713060297632091DNJMenendezMenendez (D-NJ)0.119965878040237
55-0.0333199376653913-0.09722021099268881DWACantwellCantwell (D-WA)0.119905053274074
31-0.0361826736711891-0.09641106514875571DWIBaldwinBaldwin (D-WI)0.119870079027214
10-0.0331094361241657-0.09706000082564491DNYGillibrandGillibrand (D-NY)0.119775857565045
74-0.0334062122015807-0.09683462028630561DCTMurphyMurphy (D-CT)0.119651535736152
69-0.0325608835753324-0.09655798749946831DMDCardinCardin (D-MD)0.119412340011159
54-0.0248599821563081-0.0957297064341131DRIWhitehouseWhitehouse (D-RI)0.119390224208126
950.01904421805179680.0122484615417543RAKMurkowskiMurkowski (R-AK)0.119338650035987
45-0.0398873356531087-0.09332445859330051DILDurbinDurbin (D-IL)0.119264416633329
6-0.0426066858139772-0.09044428233775111DSDJohnsonJohnson (D-SD)0.119082469790906
94-0.0306370175141373-0.0957890337713271DWVRockefellerRockefeller (D-WV)0.118904362580451
4-0.0362627872805723-0.09390152402284521DMNFrankenFranken (D-MN)0.118425828858068
75-0.0386494891823663-0.09243186911194191DNMUdallUdall (D-NM)0.118366034195972
47-0.0195976054340086-0.0917884503740951DCOUdallUdall (D-CO)0.118236088253773
42-0.0273071654738363-0.09242672709318921DMDMikulskiMikulski (D-MD)0.117150656585951
79-0.0338688838171066-0.09058683780339021DCAFeinsteinFeinstein (D-CA)0.116358543323594
70-0.0362197521229333-0.08850844656049161DNVReidReid (D-NV)0.115971985332404
46-0.0381855188001876-0.0862299213491951DORMerkleyMerkley (D-OR)0.115774234530289
1-0.0348802242728007-0.08850111507224731DDECoonsCoons (D-DE)0.115670517369723
32-0.0268215357509806-0.08904322829284141DORWydenWyden (D-OR)0.115563613050844
27-0.0229079628265486-0.08694715097353951DPACaseyCasey (D-PA)0.115286524384875
19-0.0298919244925452-0.08800299932631221DMNKlobucharKlobuchar (D-MN)0.114982973280414
14-0.0364473257536752-0.08390365827336081DNHShaheenShaheen (D-NH)0.114595708701108
51-0.005620410440720870.01034682672051283RMECollinsCollins (R-ME)0.114302134456818
33-0.0122248435323734-0.07541200813227351DAKBegichBegich (D-AK)0.114243839017192
17-0.029568048844845-0.08585338412786761DFLNelsonNelson (D-FL)0.114194289812714
8-0.0356973327991686-0.08263842242973921DDECarperCarper (D-DE)0.114101444562881
36-0.0292497277663385-0.0854071015533281DNMHeinrichHeinrich (D-NM)0.114048209132161
7-0.044554280398656-0.05454062058459171DMTTesterTester (D-MT)0.113815375991399
87-0.0292309014580987-0.08284832169195471DIAHarkinHarkin (D-IA)0.113321234065552
12-0.0289778208474004-0.08249014093175211DCOBennetBennet (D-CO)0.113231028586499
86-0.0357000480753136-0.07351270653236621DVAWarnerWarner (D-VA)0.112526922656803
85-0.011292726591166-0.06251294436694251DMOMcCaskillMcCaskill (D-MO)0.112014211462848
71-0.0377614697054327-0.05373642303023281DMTBaucusBaucus (D-MT)0.111755195704732
48-0.00926246130655879-0.01888927150114171DWVManchinManchin (D-WV)0.111750853665797
90-0.0299927709252352-0.07020867997312411DVAKaineKaine (D-VA)0.111203430282439
30-0.026944901403195-0.07079990202363662IMEKingKing (I-ME)0.11110709047993
72-0.0231248134323342-0.06908277548908211DLALandrieuLandrieu (D-LA)0.111001469145908
81-0.0198635645247844-0.05924413522218491DNDHeitkampHeitkamp (D-ND)0.110305612650745
38-0.0192640058583086-0.03702424075311591DARPryorPryor (D-AR)0.110231120430787
83-0.019576243340423-0.04817753440837241DINDonnellyDonnelly (D-IN)0.109949132073788
44-0.0228120494502621-0.05012470619101971DNCHaganHagan (D-NC)0.109873667752502


We can also make a graphic of the most "extreme" senators given this distance:

senate extremity

Note that the minority party is more likely to be extreme by this distance metric, because the typical view is titled towards the party with more votes.

Using historical data

We can also calculate how polarized the parties have been by calculating how "extreme" the average member in each party was at any given time.

senate extremity

We can see how polarization has changed over time, and the average distance of each member to the typical voting pattern has shifted.

Thoughts

This analysis was interesting to do, and I hope to do more in the future. You can find all of my code here.

Some cautions:

  • I would hesitate to make any sweeping generalizations from this that are not supported by the data.
  • What you see is pretty much what you get. All of this data is publicly available, and I highly encourage you to look at it if you are interested.

Ideas:

  • It could be interesting to analyze voting patterns as compared to the text of bills. (Does senator X always vote for bills with the phrase "increase defense spending" in them?)
  • Voting patterns vs demographic shifts in the US.
  • Linking voting patterns and the rise and fall of political parties.
  • Predicting state senate votes.
  • Doing similar analysis for the House.

To leave a comment for the author, please follow the link and comment on his blog: Vik Paruchuri.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.