Introduction

For most of written history, professors have had almost sole control of the bully pulpit in higher education.  Variations on the “kids these days” theme abound. We need go no further than to Plato’s quoting of Socrates:

“The children now love luxury; they have bad manners, contempt for authority; they show disrespect for elders and love chatter in place of exercise. Children are now tyrants, not the servants of their households. They no longer rise when elders enter the room. They contradict their parents, chatter before company, gobble up dainties at the table, cross their legs, and tyrannize their teachers.”

There’s only one problem with the quote above – it’s a fake!  While it is often attributed to Plato, there is no written record to authenticate it by.  Nonetheless, it captures the spirit of professorial views of students throughout the ages and there are many other examples we can choose from.

In the 13th Century, the noted lecturer of law Odofredus bemoaned:

“All desire to know but none to pay the price.”

Well, with costs of a college education rising upwards of $250,000 per degree for elite private schools, we probably can’t make that argument any longer.  And today, students are fighting back on websites like RateMyProfessors.com. The retorts may be less elegant, but there’s no doubt that all deferential pretense is missing on the site.  Educators are being asked and even forced to defend their methodologies.  Such reviews clearly have strong influence on the way courses are taught as we will review below.

From a business standpoint, this presents a number of interesting questions.  How do you evaluate consumer feedback in a industry where the customer does not completely understand the product? Or even, how does one evaluate feedback when the consumer’s long term interests are at odds with their short-term desires? Going further, in crafting strategies for delivery, how do we weight measures of effectiveness against the consumer’s assessment of the quality of the experience?

As an example, being stuck in an airliner on the tarmac is a pretty unpleasant experience, but taking off under unsafe conditions has potentially deadly consequences.  Momentary discomfort during embarkment for the sake of safety is forgotten at the destination if the customer can be made to understand that the delay was for their safety and benefit.  However, if it is seems that the delays are due to carelessness, indifference, and profit-motive, the experience will be hardwired into the passengers perception of the airline.  In higher education, there is no event comparable to the drama of take-off from a runway, but rather a series of embarkations that culminate in mastery of a body of knowledge at the bachelor’s level.

Changing Mindsets, Fads,  and Student-Centering

A simple google search for the phrase “Learning is fun” will turn up scores of articles which link dopamine and reward centers to retention of new material and outcomes in the classroom.  It’s a seductive notion  — imagine a positive feedback  loop that reinforces learning in such a way as to make it pleasurable!  If teaching modalities could be developed that harness the power of ‘fun’, it will make us all smarter, right?  And we’ll all enjoy the experience!  Really, this would be the ultimate ‘win-win’ as

There’s a really big problem with this type of thinking.  It ignores the fact that pain is also an effective teacher and it too activates the brain’s dopamine system.  We all know the old adage about touching a hot stove, and it’s very easy to argue that lessons born from pain are highly effective and central to the development of a healthy outlook.  After all, a fear of heights, snakes, and spiders seems to be hardwired into mammals in such a way as to allow us to recognize danger without having experienced the negative outcomes.  This trait may be vital to our survival as a species.  So while it would be nicer to think that pleasure is a better instructor than pain, there really is no strong evidence either way yet.

Today’s college students have certainly been raised with the notion that learning should be fun, student-centered, accessible, and valuable.  Faculty and administration have been increasingly asked to defend the worth of a college degree, and the best estimates suggest that someone with a B.A. or B.S. will earn about $1,000,000 more in their lifetime than someone with ‘just’ a high school diploma.  There is a great deal more expectation brought to the fore now, and perhaps rightfully so given the high costs.  However, one legitimate complaint often lodged by faculty is that students are showing up in their classes less and less suited for the work.  In fact, objective measures indicate that while preparedness levels in college-bound students have held roughly constant over the last 25 years or so, there are many more students attending college today.  Thus, there are more unprepared students taking college courses now than ever before.

To explore the connection between learning and fun a little more, let’s think about some of the competitions that are currently taking place in the Winter Olympics.  The downhill event in skiing looks like an exciting and fun event to me.  I like to go fast and love mountain scenery.  However, if you took me up to the top of the slope and dropped me off, I doubt that I’d have a very good time.  I don’t know how to ski.  I don’t know what the basic rules are or how to turn.  If I went aerial, I’m positive I’d crash because I don’t know how to take off and land.   So my subjective estimation of the fun involved with skiing is directly tied to my preparedness for the event.  I’m sure that I’d have a much better time on the bunny hill until I learned how to turn and deal with excessive speed.

Thus, preparedness is a key ingredient in making learning pleasurable, to the degree that’s truly possible.  However, as we’ve seen, larger numbers of unprepared students are  attending college and creating a challenging environment  in which faculty must teach.  If we factor in the emphasis on increasing retention and reducing DWF (D, Withdrawal Fail) rates, faculty can feel as though they are caught between the proverbial rock and hard place.  Likewise, students are mortgaging a large piece of their future on the idea that the education that they get will be ‘value-added.’  There may be a tendency for them to devalue certain courses that they perceive to be ‘gatekeepers’ or ‘in their way,’ and it is not always apparent or even interesting as to why they need to take those courses.  Thus, this post seeks to examine the viewpoints on either side of measures such as are embodied by Rate My Professor and, more importantly, what can we learn from it?  Is there a way that this feedback can be effectively incorporated into curricula without diluting or damaging the content that is so vital to students’ long term interests?

Histograms for Quality (Helpfulness & Clarity) and Difficulty on Rate My Professor for 25 selected schools.

Sites Scraped for this Project

First, Rate My Professor was scraped through the use of Selenium.  Twenty-five schools were selected that embody various archetypes in higher education – the R1 research institution, the polytechnic, elite (ivy & public ivy), liberal arts colleges, and finally, regional universities.  The selection was made to ensure that broad representation of different types of schools was achieved and the astute observer will note that it is possible to be in more than one category.  These schools were cross-indexed with entries obtained through Beautiful Soup from GradeInflation.com, so that simultaneous information could be obtained about both grade inflation and student evaluations.  Finally, API calls were made to the Department of Education’s College Scorecard for information on selectivity, preparedness, revenue, and money spent on instruction.

Scatter plots for Quality and Difficulty. The left panel has points colored by ‘hotness.’ The line comes from a linear model or ‘lm’ fit in R with a correlation coefficient of -0.5.

Results and Discussion

The initial results are displayed above with histograms showing the distributions for Quality assessment by the students along with Difficulty.   The Quality rating is an amalgamation of Clarity and Helpfulness and is measured from 1 to 5 stars with increments of 0.5 whereas the Difficulty is only measured in integer steps from 1-5.  It is interesting that the Difficulty is normally distributed while the Quality appears to be bimodal or perhaps Poissonian.  There is a strong negative correlation between Quality and Difficulty with a Pearson correlation coefficient of about -0.5.

If we use a third parameter, measured as ‘hotness’ and given by the infamous  , the results are stunning.  If a professor receives the chili pepper, they are much more likely to be rated higher on the quality scale and less difficult than if they are not.  The ratio of 5:1 in the quality ratings moves from 15 for ‘hot’ professors all the way down to 2 for those who are not judged worthy of .  Furthermore, the ratio of 5:3 difficulty ratings is twice as much for ‘not-hot’ as ‘hot.’

These observations are not offered to discredit the reviews, rather to help put them in proper perspective. Faculty and students necessarily have different perspectives on quality and difficulty. Faculty tend to have a longer term view – this is the luxury of having already ‘made it.’  Students’ viewpoints are more dominated by day-to-day anxieties. Understanding both mindsets is key to utilizing the information provided by the reviews.

FTE spending and outcomes.

Next, it is interesting to examine the influence of spending per full time equivalent student on the Rate My Professor scores.  This is defined as the difference between the revenues obtained per full time student and the amount spent on them for educational purposes.  It is quite surprising that both quality and difficulty are positively correlated with FTE spending given that quality and difficulty are negatively correlated.

One interpretation of this is that teaching methodologies that lead to better results require more input from the students.  Thus, they consider it more difficult.

Conclusions and Future Work

Next, we will consider the influence of preparedness on outcomes along with grade inflation. Parameters considered will include SAT scores and admission rates along with mean GPA and quality/difficulty scores. This will complete the circle between outcomes(GPA), college ratings, and student preparedness.