An Analysis of Texas High School Academic Competition Results, Part 3 – Individuals

May 19, 2018
By

(This article was first published on r on Tony ElHabr, and kindly contributed to R-bloggers)

Let’s take a look at individual competitors in the academic UIL
competitions.

Individual Participation

The first question that comes to mind is that of participation–which
individuals have competed the most?

NOTE:

To give some context to the values for individual participants, I’ll
include the numbers for myself (“Elhabr, Anthony”) in applicable
contexts.

rnk name school city conf n
1 Jansa, Wade GARDEN CITY GARDEN CITY 1 57
2 Chen, Kevin CLEMENTS SUGAR LAND 5 56
3 Hanson, Dillon LINDSAY LINDSAY 1 53
4 Gee, John CALHOUN PORT LAVACA 4 47
5 Zhang, Mark CLEMENTS SUGAR LAND 5 47
6 Robertson, Nick BRIDGE CITY BRIDGE CITY 3 46
7 Ryan, Alex KLEIN KLEIN 5 46
8 Strelke, Nick ARGYLE ARGYLE 3 45
9 Niehues, Taylor GARDEN CITY GARDEN CITY 1 44
10 Bass, Michael SPRING HILL LONGVIEW 3 43
1722 Elhabr, Anthony CLEMENS SCHERTZ 4 13

Note: 1 # of total rows: 123,409

Although the names here may not provide much insight, the counts provide
some context regarding the limits of individual participation.

Given that counts of overall participation may not be indicative of
anything directly, it may be a better idea to break it down by
conference.

It seems that there has not been as much invdividual participation in
the 6A conference (conf 6)– which is the conference with largest high
schools (according to student body size).

I hypothesize that this phenomenon can be attributed to “pre-filtering
of talent” by these large schools. In other words, conference 6A schools
may be more likely to designate their individual competitors to compete
in only specific competitions and prevent any student who may be
capable, yet not fully prepared, from entering a competition. High
standards and expectations of aptitude are relatively common at very
large schools, even if what may be deemed “unacceptable” at such a
school would be very satisfactory at a smaller school. By comparison,
schools in all other conferences may be more willing to let individual
students compete in as many competition types as they desire, even if
they have not prepared for them whatsoever.

Such a phenomenon might be evident in lower scores (in aggregate) for
conferences where participation is greater. In fact, this is exactly
what is observed. 1 On average, conference 6A has the highest scores,
while conference 1A has the lowest.

So what about people’s scores? Who did best according to score? In order
to simplify the data, let’s look at a couple of statistics based on
score, aggregating across all scores for each individual. In particular,
let’s look at the average and sum of placing percent rank (prnk) and
of individual competitors “defeated” (n_defeat). (Note that
competitors defeated is defined as the number of scores less that that
of a given individual for a unique competition, and a unique competition
is defined as a unique combination of year, competition level, and
competition type.)

rnk name school city n prnk_mean n_defeat_mean
1 Hanson, Dillon LINDSAY LINDSAY 53 0.97 29.17
2 Chen, Kevin CLEMENTS SUGAR LAND 56 0.91 29.80
3 Jansa, Wade GARDEN CITY GARDEN CITY 57 0.89 29.86
4 Niehues, Taylor GARDEN CITY GARDEN CITY 44 0.96 31.20
5 Gee, John CALHOUN PORT LAVACA 47 0.90 25.21
6 Zhang, Mark CLEMENTS SUGAR LAND 47 0.89 29.15
7 Strelke, Nick ARGYLE ARGYLE 45 0.93 26.56
8 Robertson, Nick BRIDGE CITY BRIDGE CITY 46 0.88 25.33
9 Ryan, Alex KLEIN KLEIN 46 0.86 26.20
10 Xu, Steven DAWSON PEARLAND 43 0.88 26.81
2608 Elhabr, Anthony CLEMENS SCHERTZ 13 0.60 16.62

Note: 1 # of total rows: 117,684

Also, I think it’s interesting to look at the distribution of counts for
competitors defeated, advancement, and state competition appearances.
The heavily right skewed distribution of values gives an indication of
the difficulty of succeeding consistently.

For comparison’s sake, let’s visualize the same metrics aggregated at
the school level. Keep in mind that while the sample of students should
have larger counts for number of advancements and state competition
appearances (y-axis) for any given number of occurences (x-axis) because
there are many more students than schools, schools are more likely to
have a wider range of occurrences (x-axis) because there are less
schools in each competition (compared to the number of individuals).

To understand why this is true, let’s take an example: Say there is a
District level competition where there are 8 schools and 40 individuals
competing. It is more likely that a given school advances to the next
level of competition (as a result of having a total score that is higher
than the scores of the other 7 schools) than any single individual, who
if not from the school that advances, can only advance as a result of
having a top “n” (e.g. 3) score.

We see that the distributions are skewed towards the right here as well,
although not quite as “evenly”. This indicates that some schools tend to
perform well at a more consistent rate than individuals themselves.
Intuitively, this makes sense. It can be very difficult for individuals
alone to beat out the competition, especially if they have an “off” day.
On the other hand, schools, relying on teams of individuals, are placed
according to the sum of the top “n” (e.g. 3) of individual competitor
scores. Thus, because school scores are dependent on groups of
individuals– who will tend to perform more consistently in aggregate
than any one individual– school placings are more likely to be similar
across years, meaning that schools that are observed to do well in any
given year are more likely to do well in other years as well (relative
to individual competitors).

So it should be obvious that it is difficult to make it the highest
level of competition–State. But exactly how difficult is it? Let’s
identify those people (and their scores) who have made the State
competition level four times–which is the upper limit for a typical
high school student 2– for a given competition type.

Clearly, these individuals represent a very small subset of the total
sample. They might be considered the “elite”. Of these individuals, who
has appeared in State competitions for more than one type of
competition?

name school city conf n
Chen, Kevin CLEMENTS SUGAR LAND 5 4
Jansa, Wade GARDEN CITY GARDEN CITY 1 4
Hanson, Dillon LINDSAY LINDSAY 1 3
Strelke, Nick ARGYLE ARGYLE 3 3
Bass, Michael SPRING HILL LONGVIEW 3 2
Deaver, Matthew SILSBEE SILSBEE 3 2
Liu, Jason DAWSON PEARLAND 4 2
Ryan, Alex KLEIN KLEIN 5 2
Williams, Tyler POOLVILLE POOLVILLE 1 2
Xu, Steven DAWSON PEARLAND 4 2

Note: 1 # of total rows: 11

I would consider those individuals appearing here to be the “elite of
the elite”.

Individual Performance

Now, I want to try to answer a somewhat ambiguous question: Which
individuals were most “dominant”?

Evaluating “Dominance”

Because the term “dominance” is fairly subjective, it must be defined
explicitly. Here is my definition/methodology, along with some
explanation.

First, I assign a percent rank to individual placings in all
competitions based on score relative to other scores in that
competition. I choose to use percent rank–which is a always a value
between 0 and 1–because it inherently accounts for the wide range of
number of competitors across all competitions. (For this context, a
percent rank of 1 corresponds to the highest score in a given
competition 3, and, conversely, a value of 0 corresponds to the lowest
score.)

I should note that I evaluated some other metrics for gauging individual
success, including the total number of individuals defeated in
competitions. Percent rank based on score and number of defeats attempt
to quantify the same underlying variable, but I think percent rank is a
little more “natural” to interpret because it contextualizes number of
competitors with its unit range. By comparison, the interpretation of
number of defeats is less direct because the number of other competitors
is not accounted directly.

Then, to come up with a final set of ranks, one for each unique
competitor, based on the percent ranks for individual competitions, I
simply sum up the percent ranks for each individual.

The sum is used instead of an average 4 because rankings based on
averages –and inferences made upon them–are sensitive to individuals
who do not compete in many competitions, yet place very well in them. A
final ranking based on a summed value does not suffer from this pitfall,
although it can be sensitive to the sample size of each participant.
(i.e. An individual might participate in a high number of competitions
and under-perform relative to the average in all of them, yet their
final ranking, based on summed percent ranks, might indicate that they
are an above-average performer.)

name school conf rnk_sum_prnk rnk_mean_prnk rnk_sum_n_defeat rnk_mean_n_defeat
Hanson, Dillon LINDSAY 1 1 191 3 5205
Chen, Kevin CLEMENTS 5 2 809 2 4834
Jansa, Wade GARDEN CITY 1 3 1298 1 4827
Niehues, Taylor GARDEN CITY 1 4 200 4 3612
Gee, John CALHOUN 4 5 1114 14 10361
Zhang, Mark CLEMENTS 5 6 1272 5 5213
Strelke, Nick ARGYLE 3 7 562 12 8340
Robertson, Nick BRIDGE CITY 3 8 1598 15 10277
Ryan, Alex KLEIN 5 9 2561 8 8754
Xu, Steven DAWSON 4 10 1661 16 8160
Elhabr, Anthony CLEMENS 4 2330 30385 2934 34266

Note: 1 # of total rows: 123,337

Some of the same individuals from the participation-based ranking of
competitors also appear among the top of the ranks by my evaluation of
domination. This is somewhat expected due to the nature of my
methodology (in particular, my choice to use a sum instead of an average
or some other statistic). Certainly some additional statistical analysis
could be done here to investigate other methods of quantifying dominance
(beyond my analysis). 5 Because there is no well-agreed upon metric or
method anywhere for quantifying dominance for this particular topic,
it’s difficult to really judge the findings here.

In any matter, the difference between the ranks based on the average and
the sum of percent ranks is not all too great–I found that the
correlation between the two is anywhere between ~0.75 and ~0.95 when
using either score percent rank or number of defeats as the metric for
ranking.

rowname rnk_sum_prnk rnk_mean_prnk rnk_sum_n_defeat rnk_mean_n_defeat
rnk_sum_prnk NA 0.8192 0.9488 0.7375
rnk_mean_prnk 0.8192 NA 0.7840 0.8829
rnk_sum_n_defeat 0.9488 0.7840 NA 0.8449
rnk_mean_n_defeat 0.7375 0.8829 0.8449 NA

Evaluating “Carrying”

Another way of identifying superb ability is to compare the scores of
individuals with those of their teammates. Individuals with high scores
relative to their teammates might be said to have “carried” their
teammates. Although this kind of evaluation is dependent on the skill of
each team (independent of the competition setting), I think that it is
another interesting way of evaluating skill.

I was hoping that I might see myself appearing among the most dominant
by this measure of skill, but it does not say anything necessarily bad
about myself that I don’t. I competed with other individuals who I
considered to be very knowledgeable and who often scored better than me.
Also, from the opposite point of view, I don’t think I was a poor
performer who relied upon teammates to boost the team’s overall score.
This is why the data points corresponding to me show up in the middle of
the pack in the previous visual.

“Improvement”

I think something else that would be interesting to look at is personal
“improvement” between years. Theoretically, if we assume that
individuals improve their academic ability and competition skills every
year, then we should see individual scores for a given competition type
and level increase from one year to next. I would be very surprised to
find that this is not true.

To evaluate improvement, we can simply reduce the whole data set to just
those who have competed in the same competition type and level in more
than one year and check whether or not their scores increased or
decreased from one year to the next. 6 Actually, in order to account
for variance in competition difficulty across years, it’s better to use
the percent rank of the individual’s placing (based on score) rather
than score itself.

improve n
FALSE 23,022
TRUE 35,305

Note: 1 # of total rows: 2

So it is true that individual scores–actually, percent rank of
placings–do tend to improve as the individual ages. But is this trend
statistically significant? That’s easy enough to answer–we can simply
perform a binomial
test
–where the null
hypothesis
is that the
distribution of “TRUE” and “FALSE” regarding improvement is truly a 50 %
– 50 % split. If the p-value of
the test is below a threshold value–let’s say 0.05–then we can deny
the null hypothesis and say that there is a non-trivial trend of
individual improvement.

metric value
estimate 0.6053
p.value 0.0000
conf.low 0.6013
conf.high 0.6093

In fact, this is exactly what is observed.

Now, let’s reduce the set to just those who have appeared in
competitions at a given competition type four times (for the sake of
visualization) and plot the scores across years for each individual.

It is evident visually that people do have a tendency to improve over
time.

Wrap-up

I’ll leave the discussion of individuals at that, although there is much
more that could be explored.


  1. See the previous post.
    ^
  2. The assumption here is that each student takes four years to complete high school.
    ^
  3. A unique competition is defined as one having a unique year, competition type, and competition level.
    ^
  4. The average may also be considered a valid means of aggregating the values for each individual.
    ^
  5. I’ll leave further analysis for another person and/or time.
    ^
  6. I don’t think it is relevant to require that the scores be in consecutive years, so I don’t enforce that criteria.
    ^

To leave a comment for the author, please follow the link and comment on their blog: r on Tony ElHabr.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)