who does it best? east vs. west coast dressage

A couple of weeks ago I was catching up on The Eventing Radio Show, listening to old episodes from last year. Joe Meyer, who I love as a host and a rider, made the throwaway comment that he wouldn’t mind competing out here on the west coast because everyone does so well in dressage out here. It was meant as a joke, but it piqued my interest anyway. I have definitely complained with my west coast friends about the “easy” dressage judges out east, and I’m sure people in Texas are laughing at both coasts. But who is right?

This, it turns out, is an easy enough question to answer. So I took to Startbox and Event Entries and scraped for data on dressage scores. Then I fired up R Studio and went on a big fat fishing expedition (research slang for exploring a dataset for relationships instead of testing specific hypotheses).

The short answer? There’s no difference in average dressage score between the East and West coasts for FEI events.

Data collection & other details

There are hundreds and hundreds of events held across the US every year, and usually a few hundred people at each of those events. So to save my sanity when scraping the data, I stuck to rated events that offered the FEI levels. A few USEA-only events slipped through the cracks because I’d already opened the event and at that point, it was easier to just copy that data. I tried to get an even representation of events across the year and across the country, but of course there are more events on the East coast than anywhere else.

I ended up with a little more than 5,500 rider records before I called it quits on copying and pasting and reformatting.

For each rider record, I included the division, venue, date, state, area. I included columns that allowed me to pool similar divisions that aren’t exactly the same (CIC 2* and CCI 2*, or training and training 3-day). I also included whether the level in question was USEA or FEI rated.

I didn’t include any multi-level effects for for rider or dressage judge. Laziness was not the only reason for this — by ignoring the influence of the judge (for now), I could (kindof) see if the effects across different areas had to do with riders or judges. If the same 5 judges worked at every event across the US, then we’d expect their scores to be very reliable, and differences from coast to coast would have to do with rider differences.  Of course there’s many more than 5 judges and these effects wouldn’t be so obvious, but you can see what I’m getting at.

(Obviously if you have questions or quibbles, get in touch.)

Fun fact!  

Mean dressage score in the US? 34.756. Standard deviation is 5.19.

This is the density histogram of dressage scores across all levels. You can see that though the mean is 34.756, there is a peak in scores after that — right around a nice even 35.

So there you have it. That’s the average dressage score at (rated) events across the US.

If you’re trending below a 35 you should feel chuffed as you’re doing better than most! When you score below 30, you’re doing better than about 85% of the country. Below a 25, and you’re doing better than ~97%. Down below 20? You’re the 1%!!

(Non-eventers reading this, remember that lower dressage scores are better in our world.)

The questions

Mostly, I was interested in exploring the differences between the coasts, USEA areas, and states to see if dressage scores varied significantly from place to place. And for the most part, the differences weren’t stark or necessarily significant.

At the FEI levels, there are no significant differences in dressage scores across major geographical areas of the us — east coast and west coast, the south, and the “mid”dle. If you’re not familiar with the model outputs, the important columns here are the Estimate and the asterisks. The intercept estimate represents the average dressage score of the east coast, and the estimates below are how much the other coasts differ from the east coast dressage average. So other areas of the country do have slightly higher dressage scores on average than the east coast, but not significantly so. (I’ll get to the stargazing in a second.)

You can see this reflected in the density histogram at the top of the post. There’s a lot of overlap between the dressage scores of east and west coasts at the FEI levels.

But how do the USEA levels stack up when you compare things from coast to coast?

Well, things aren’t quite so tidy. Let’s do some stargazing (those asterisks are typically thought of as good things in stats land)!

For events only sanctioned by the USEA, there east coast has a significantly lower average dressage score than any other area.  Riders in the middle of the country (basically Montana) are scoring nearly 2 points more, on average, than riders out east. People in the south get about 1.4 points more, and out west we get a measly 0.7 points more.

It’s important to note here that the “points” I’m talking about are percentage points, not raw points on tests.

Let’s break this relationship down a little more by state, shall we?

In this case, the intercept state is California. And what we see here is kinda neat! West coast states seem to line up (ish) in terms of scoring, which makes sense because they would probably pull from a very similar pool of judges. The second row in the table is Canada (from Bromont’s results).

But start comparing to the east coast, and we start to see some differences! Florida and Maryland in particular appear to be preeeetttyy generous with those dressage scores! Riders in those states score 1.36 and 2.55 points better on average, respectively. On the other hand, Montana is out there hammering their riders with dressage scores an average of 1.17 points worse than in California.

Because I’m California-centric, I plotted the distribution of scores in the lowest-scoring state (Maryland) vs. California, to show what a significant difference here looks like. It’s not a HUGE split between the curves, but you can see that there are quite a few more riders in Maryland in the sub-30 zone than you see in California.

So what does it all mean?

There could be lots of reasons that I found significant differences between the dressage scores of different states and areas. For one, I didn’t apply a single correction for multiple tests to this data set, and I explored tons and tons of potential relationships. Statistically speaking, one of them was bound to come up significant.

Could it be that riders out east are just better than riders in other states? Ummmm. I mean sure, this is one possible answer. But given the variance between states up and down the eastern seaboard, I’m not sure this hypothesis holds up.

It’s also possible that judges on the west coast and in Montana are much harsher and stricter than on the east coast. I’ve heard people say that it’s hard to “make it” in dressage in California because there’s such strong competition down here in the form of Steffen Peters and Hilda Gurney. Perhaps the presence of Olympic-level riders makes judges more strict? If so, one would expect a similar effect in Florida. And… well, the data doesn’t quite hold up to that either (but also isn’t designed to answer that question).

It seems like there might also be some hyper-local effects on the east coast, since it’s so densely populated out there. This might be due to the fact that judges out east don’t need to have a very wide travel radius in order to judge plenty of shows. If those judges tend to score a little better, then that would create pockets of shows where scores are a little more generous.

Anyway, there’s lots of potential reasons for these trends. I just enjoyed looking at them! What I can pretty confidently say is that for 2018 at least, riders in California were not getting preferential treatment from dressage judges (erhrrm, Joe!).

21 thoughts on “who does it best? east vs. west coast dressage

  1. Since scores are so subjective depending on who’s judging, I think it would be more valid to track the judges themselves rather than just the scores and the location. A good chunk of the judges at our recognized shows are from the east coast, with some from this area and the occasional one from the west coast. The judge I rode under for one of the weeks of Chatt last summer is one that I’ve ridden under here in Texas before. I’ve also ridden under the same judge as some of my friends in VA/MD. I’d be interested to see how their scores compare between locations. Scores themselves don’t tell me much, considering the same test could score a 30 under one judge and a 35 under another.

    Like

    • Certainly, though this is nearly impossible with an evening dataset since judges names aren’t usually published with online scores. If I compared the same riders under different judges, then you’d be able to see if judges have different scales. If I compared the same judges at different venues, then you’d be able to see if there actually are regional differences in dressage quality. Obviously there are much more precise ways to answer the question, depending on what your question is.

      Like

  2. nice analysis!! and definitely opens the door to a whole new set of questions and avenues for further investigation.

    for instance, if you were getting paid and/or had access to the full usdf and usea databases of riders / horses / events, i’d be curious to know details on the number of high level pros/trainers based in each of those areas too. does Access to high level professionals correlate to competition scores?

    another potential avenue: i would hypothesize that the frequency of shows per location (ie: Opportunity) is correlated to scores too. in maryland i could go to a rated show almost every weekend if i wanted. for me, experience is directly related to my nerves or lack thereof, and my ability to perform to my best. furthermore: with so many options on the calendar, i can pick and choose show dates to optimally align with my horse’s conditioning and training program. vs in montana, for instance, where maybe there are fewer events and something like Rebecca looms large on the calendar, whether you’re ready for it or not.

    in other words, by suggesting the above two avenues for further analysis, my thoughts are that the average scores by region have less to do with “judge generosity” in those areas and more to do with other locale attributes like: Access and Opportunity.

    anyway, long story short, nice work! this is fascinating and a nice introduction to parsing the data!

    Like

    • SO many good points here! This analysis really was quick and dirty (I mean, data collection didn’t feel quick lol). I’ve always wondered if usdf and usea track judge scores and regional trends. If for no other reason than curiousity! But in actuality, it could be very valuable for understanding judge reliability and concept drift.

      Like

      • i believe usdf does in fact track judges scores (for instance, you can look judges up on center line scores and see their average scores by level, i believe). not sure about usea but my impression is that the entire industry is growing in the direction of utilizing data to improve the sport (see: equiratings). judges also, i believe, are required to participate in continuing education to keep their licenses active but i’m not sure what exactly that entails.

        the regional aspect i think matters more at the lower levels than the higher, bc there are fewer qualified and actively licensed judges at the highest levels and those judges are recruited and will travel.

        it really is an interesting idea tho. when i volunteered for the east coast FEH championships, there was discussion among the judges and usea officials about which regions have the best quality young horses. that was a unique situation bc the east, west and central championships were judged by the same two individuals, so you CAN actually compare scores across the classes to see how all the horses would have ranked if they were combined into a single national class. i don’t really know of any other regional examples that are quite so clear tho.

        Like

  3. I love this… the data nerd in me is so happy hahaha. I wonder if Maryland vs. California is skewed simply due to the number of events available. Someone in Maryland might get to compete 2x a month all season with short drives to competitions… vs. someone in California who might not have as many opportunities. And, as we all know, repetitions in the sandbox can totally make a few points of difference.

    Liked by 1 person

  4. This is really cool. Thanks for doing all the work. I never really thought of the west coast as being easier scoring in dressage. I have heard that the east coast has it easier in jump heights on XC, which is something I’m interested in seeing if it’s true in person now that I am out here. I’m with Emma on wondering how much opportunity is a factor. CA has the whole events cannot be back to back in even the same half of the state (north vs south) let alone back to back at a specific location like other zones allow (Coconino, Loch Moy, +more). But it’s good to know the scores are all pretty similar anyway.

    Liked by 1 person

  5. Ooohh my data nerd heart is singing! I think Emma’s points are really good re: access/opportunity. I’d also wonder about things like weather (ie. a lot more covered/indoors on the East coast means you can ride no matter the weather) and surrounding community (Maryland is very strong eventing wise, but are things “harsher” so to speak in California because there’s more dressage-centric community there?) Are more people on the East coast riding horses who are eventing-specific trained vs west coast where there may be more who came into it from other disciplines?

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s