Nerd Alert: Do Level and Format Matter?

A couple weeks ago I examined whether stadium 'mattered'. Which, true to analysis form, the answer is "kind of" but more prompted the question "define mattered".  But when I was working on that project, I started looking at the other aspects of the data that Emma graciously provided. The scores she pulled included all international levels, from one to three (now two to four...) stars, and both long and short formats.

Comparison by Level
I compared the different levels with one another, because I was curious if the performance was different as competitors moved up the levels. Was the dressage harder? The show jumping more complicated? Inquiring minds want to know.

I did some regression, made some graphs and got distracted by the many shades of purple that were available to me. It's a bit harder to see in these little whiskers, but the data I looked at showed statistically that there was no difference between mean scores at each level for stadium, however dressage and cross country did have a difference. Horse and rider pairs did better at the lower levels. As competitors rose up the levels, so did their scores get worse. Except in stadium.

For dressage, I had also looked at tests at the variances of scores. What I found was that not only did the mean score vary by level, but the variance of scores. Pairs riding at the one star level had more closely grouped dressage scores, while competitors at the two and three star level had dressage scores spread over a wider range.

Comparison by Format
I think 'long' versus 'short' format is a bit of a misnomer since the loss of the classically formatted events with a true endurance day. Is there even a difference between today's long and short format events? Is there a point to keep CCI-L and S as two separate items?

The answer is, surprisingly, kind of yes.

And also kind of no.

Unsurprisingly, dressage was not statistically different between formats. The dressage tests are the same, the order doesn't change for dressage so the results are the same. Sure. Cross country did show a difference between long and short format, with mean scores statistically different between the two. 

A little disappointingly, but par for the course so far, stadium touted the same mean scores for both long and short format. I was anticipating seeing a difference as two very different types of horses are competing stadium. Long format horses have already been out on cross country and should be more tired. I would expect long format horses to pull more rails or take more time, but there isn't evidence for it.

So what do you think? Did everything come out as expected? Do you think the mean scores should be the same ascending up the levels or different? I have some hypotheses on why but I'd love to discuss. Did anything else surprise you?

*Data background and caveats: the data was pulled from EventEntries.com. Live scoring for 25 international events in 2018 in the US were used. The size of each class was not standardized which could effect how many are in top ten/non top ten for each class. The sample size for the long format higher level events is not credible and more data should be collected. These analyses were conducted only on finishers, and no account was taken for eliminations at this time. I'm working on a way to work in eliminations. Any time I post anything about analytics I am 150% open to feedback, ideas and collaboration. Thanks again Emma for collecting and scrubbing the numbers used in this analysis!