who does it best? east vs. west coast dressage

A couple of weeks ago I was catching up on The Eventing Radio Show, listening to old episodes from last year. Joe Meyer, who I love as a host and a rider, made the throwaway comment that he wouldn’t mind competing out here on the west coast because everyone does so well in dressage out here. It was meant as a joke, but it piqued my interest anyway. I have definitely complained with my west coast friends about the “easy” dressage judges out east, and I’m sure people in Texas are laughing at both coasts. But who is right?

This, it turns out, is an easy enough question to answer. So I took to Startbox and Event Entries and scraped for data on dressage scores. Then I fired up R Studio and went on a big fat fishing expedition (research slang for exploring a dataset for relationships instead of testing specific hypotheses).

The short answer? There’s no difference in average dressage score between the East and West coasts for FEI events.

Data collection & other details

There are hundreds and hundreds of events held across the US every year, and usually a few hundred people at each of those events. So to save my sanity when scraping the data, I stuck to rated events that offered the FEI levels. A few USEA-only events slipped through the cracks because I’d already opened the event and at that point, it was easier to just copy that data. I tried to get an even representation of events across the year and across the country, but of course there are more events on the East coast than anywhere else.

I ended up with a little more than 5,500 rider records before I called it quits on copying and pasting and reformatting.

For each rider record, I included the division, venue, date, state, area. I included columns that allowed me to pool similar divisions that aren’t exactly the same (CIC 2* and CCI 2*, or training and training 3-day). I also included whether the level in question was USEA or FEI rated.

I didn’t include any multi-level effects for for rider or dressage judge. Laziness was not the only reason for this — by ignoring the influence of the judge (for now), I could (kindof) see if the effects across different areas had to do with riders or judges. If the same 5 judges worked at every event across the US, then we’d expect their scores to be very reliable, and differences from coast to coast would have to do with rider differences.  Of course there’s many more than 5 judges and these effects wouldn’t be so obvious, but you can see what I’m getting at.

(Obviously if you have questions or quibbles, get in touch.)

Fun fact!  

Mean dressage score in the US? 34.756. Standard deviation is 5.19.

This is the density histogram of dressage scores across all levels. You can see that though the mean is 34.756, there is a peak in scores after that — right around a nice even 35.

So there you have it. That’s the average dressage score at (rated) events across the US.

If you’re trending below a 35 you should feel chuffed as you’re doing better than most! When you score below 30, you’re doing better than about 85% of the country. Below a 25, and you’re doing better than ~97%. Down below 20? You’re the 1%!!

(Non-eventers reading this, remember that lower dressage scores are better in our world.)

The questions

Mostly, I was interested in exploring the differences between the coasts, USEA areas, and states to see if dressage scores varied significantly from place to place. And for the most part, the differences weren’t stark or necessarily significant.

At the FEI levels, there are no significant differences in dressage scores across major geographical areas of the us — east coast and west coast, the south, and the “mid”dle. If you’re not familiar with the model outputs, the important columns here are the Estimate and the asterisks. The intercept estimate represents the average dressage score of the east coast, and the estimates below are how much the other coasts differ from the east coast dressage average. So other areas of the country do have slightly higher dressage scores on average than the east coast, but not significantly so. (I’ll get to the stargazing in a second.)

You can see this reflected in the density histogram at the top of the post. There’s a lot of overlap between the dressage scores of east and west coasts at the FEI levels.

But how do the USEA levels stack up when you compare things from coast to coast?

Well, things aren’t quite so tidy. Let’s do some stargazing (those asterisks are typically thought of as good things in stats land)!

For events only sanctioned by the USEA, there east coast has a significantly lower average dressage score than any other area.  Riders in the middle of the country (basically Montana) are scoring nearly 2 points more, on average, than riders out east. People in the south get about 1.4 points more, and out west we get a measly 0.7 points more.

It’s important to note here that the “points” I’m talking about are percentage points, not raw points on tests.

Let’s break this relationship down a little more by state, shall we?

In this case, the intercept state is California. And what we see here is kinda neat! West coast states seem to line up (ish) in terms of scoring, which makes sense because they would probably pull from a very similar pool of judges. The second row in the table is Canada (from Bromont’s results).

But start comparing to the east coast, and we start to see some differences! Florida and Maryland in particular appear to be preeeetttyy generous with those dressage scores! Riders in those states score 1.36 and 2.55 points better on average, respectively. On the other hand, Montana is out there hammering their riders with dressage scores an average of 1.17 points worse than in California.

Because I’m California-centric, I plotted the distribution of scores in the lowest-scoring state (Maryland) vs. California, to show what a significant difference here looks like. It’s not a HUGE split between the curves, but you can see that there are quite a few more riders in Maryland in the sub-30 zone than you see in California.

So what does it all mean?

There could be lots of reasons that I found significant differences between the dressage scores of different states and areas. For one, I didn’t apply a single correction for multiple tests to this data set, and I explored tons and tons of potential relationships. Statistically speaking, one of them was bound to come up significant.

Could it be that riders out east are just better than riders in other states? Ummmm. I mean sure, this is one possible answer. But given the variance between states up and down the eastern seaboard, I’m not sure this hypothesis holds up.

It’s also possible that judges on the west coast and in Montana are much harsher and stricter than on the east coast. I’ve heard people say that it’s hard to “make it” in dressage in California because there’s such strong competition down here in the form of Steffen Peters and Hilda Gurney. Perhaps the presence of Olympic-level riders makes judges more strict? If so, one would expect a similar effect in Florida. And… well, the data doesn’t quite hold up to that either (but also isn’t designed to answer that question).

It seems like there might also be some hyper-local effects on the east coast, since it’s so densely populated out there. This might be due to the fact that judges out east don’t need to have a very wide travel radius in order to judge plenty of shows. If those judges tend to score a little better, then that would create pockets of shows where scores are a little more generous.

Anyway, there’s lots of potential reasons for these trends. I just enjoyed looking at them! What I can pretty confidently say is that for 2018 at least, riders in California were not getting preferential treatment from dressage judges (erhrrm, Joe!).

adventures in the annals of equine research

I’ve got a super exciting thing happening soon, which I’ll be able to announce next week. This super exciting thing has me delving into equine research journals. And, people, there is gold in there.

GOLD!!!!

also gold: stalking this handsome creature!!

Nugget the First, the Journal of Equine Veterinary Medicine publishes some of its articles open-access. This means that the authors pay a lot more money in order to make those manuscripts freely available to the public. I’ve got institutional access at work, so I don’t know exactly which ones are free and which ones aren’t (this is a hint). But there is a lot of good stuff on there, and tons of it is bound to be free.

Nugget the Second:

yes, please tell me everything about this mysterious sporthorse stallion “hindquarter movement”

Nugget three: I found this super neat site, Equestology: Sport Horse Science. They write reviews of peer-reviewed manuscripts, and have some amazingly fascinating stuff on there!  For example, a discussion of horse facial expressions during lameness. Or an in-depth analysis of how rider position differs between beginner and advanced dressage riders. I mean, on the one hand I’m super sad that my dream job of translating horsey science for the horsey public has already been taken.  On the other, what an amazing resource!! I can’t wait to read more.

Three gold nuggets is more than I’ve found in quite a while! And now I dive back in to learn more about hindquarter movements of —  I mean, equine science in general.

hippo v. lipless hippo!!

this is an advertisement

I saw a post that piqued my interest on Facebook this morning.  It was a Horse and Hound article about how breastplates can negatively affect the way horses jump.  Clearly it did more than just pique, it’s ruffled my feathers enough to break me out of my blogging ennui.  Thank you, bad science!!  You’re just what I needed.

well, that and this adorable mug

So, let’s take a look at this article by Horse and Hound. I’m not going to blame H&H too much for this reporting, since they aren’t science reporters and are really only able to work with the information they are given.  I have a sneaking suspicion that this is actually  sponsored article, and if they didn’t disclose that information it is unethical.  However, H&H is bound to know that, and so this particular article is probably not specifically sponsored — more likely, Fairfax gives H&H some large sum of money for general advertising and this article is being played off as a general reporting piece.  You will notice that the first suggested link after the article is about a girth “scientifically proven” (ever scientist’s favourite two words) to improve the way horses go.

The TL;DR of this situation: this is not science, this is an advertisement.

There are so many glaring red flags in this “study” that I can hardly list them.

There is no link to a peer reviewed journal article, or any data, figures, or any other “scientific” measures of difference. The article does point out that the horses took off “closer” (no measurement) and landed “closer” (no measurement) to the fences, thus increasing the flexion of and strain on their hocks and other joints (no measurement).

The “researchers” (Fairfax) even provided a handy-dandy little image that demonstrates how different the arc of the horse is.  No matter than in the “better” image the horse has already started to take the forward part of the landing stride with his front feet and that is where they measured his “landing” point from, and in the “bad” image the horse is pictured at a different part of the landing phase, and his landing point is measured from the foot that is further back. Plus we all know that every horse jumps every fence the exact same every time, and nothing but equipment ever influences this — not rider balance, approach, speed, or general attitude on the day!!


the exact same. every time.

All’s fair in marketing and “research”, right?

And what about those oh-so-critical study numbers that people are always reporting.  Things like sample sizep-value, effect size, or even the dastardly value of measurement?  “Significance” (their scare quotes, not mine) is all well and good, but if the effect size is less than 1%, who gives a shit?!  Wow, excellent, I can improve my horse’s bascule by less than 1% by spending $350 on your special piece of equipment.  Talk about promoting a quick fix.

There is that upper-level rider’s testimony. He says his horses jump so much free-er in front, and he can feel it.  But really, humans are biased and fickle things, and just because we think something is true doesn’t mean it is true.  Especially not subjective, un-measurable things like the feeling of a free-er jump that might be influenced by free product or small piles of gold coins.

It’s probably just me, but damn am I sick and tired of the lack of science that goes on in the equine industry! We all want to do the best for our horses, and I get that.  But a little bit of testing, common sense, and critical thinking goes a LONG WAY with this stuff.  Fortunately, many readers of this article already figured that out.

And finally, one additional pet peeve: If it’s real science, nobody is ever going to use the phrase “scientifically proven” to talk about it.  Scientists don’t use those words because science is always changing and adapting. 

Image result for snakes on a plan gif

wordy wednesday: behavior, cortisol, and welfare in horses

I’ve always wanted to put my research background to good use on this (or any) blog; writing in-depth but accessible articles knitting together research and results from multiple sources to bring a thorough and complete view to some complicated topics in equine science.  There is a crazy (though small) world of equine science out there: studies that explore everything from the effect of specific drugs on equine osteoarthritis to the interrelation of saddle slip and back shape on lameness in the ridden horse*.

Weirdly, I don’t seem to find time for that on the reg.


too busy doing this

Every once in a while I do come across a neat article about something equine-science related that I want to bring to you guys.  Frosting on the cherry is that this particular article is open access, so you can all read it if you want to!  And, in my opinion, it’s actually a fairly well-written and understandable study — perhaps because it tackles a fairly accessible topic that doesn’t require large amounts of jargon or a lifetime of studying some very specific mumbo-jumbo at 100x magnification to understand.

Low plasma cortisol and fecal cortisol metabolite measures as indicators of compromised welfare in domestic horses (Equus caballus)

Jodi Pawluski , Patrick Jego, Séverine Henry, Anaelle Bruchet, Rupert Palme, Caroline Coste, Martine Hausberger

Read the full text for yourself here.

I was originally attracted to this study because the write-up of it claimed there was some evidence in there that high cortisol levels in horses might correlate with a positive type of stress (excitement), as opposed to negative stress (poor welfare, having the snot beaten out of them, etc.).  It doesn’t quite show that, but it does cement some interesting and important findings.

  1. Horse behavior can give us good insights to their welfare (duh, but read on a bit): horses that had an ears-back posture more than 50% of the time had lowered cortisol^ levels associated with hypothalamic-pituitary-adrenal (HPA) axis down-regulation (fancy way of saying their hormone pathways were messed up), which matched other studies. This means that both the behavior/posturing and testing of cortisol levels can be useful to equine welfare scientists in the future (within reason, of course).
  2. Riding horses don’t think their lives are total shit, even when being ridden by amateurs.

In my opinion, one of the major strengths of this study is that the researchers used riding school horses in an active program (in France). Often, study subjects are kept in near hermetically-sealed conditions, in an attempt to control all external variables.  So an “inexperienced” person riding a horse in some studies is really someone who isn’t a legitimate professional, though can still course 3’6″.  While this is great for control — you know the rider isn’t likely to hurt the horse and can do exactly what you want them to do — it just doesn’t exhibit a lot of external validity — most horses don’t live their lives being ridden only by people capable of coursing 3’6″.  For the most part, being flopped around on by rank amateurs is a lot more like a horse’s experience.


floppy reality

So, what did the researchers actually find?  (In reverse order, because I’m trying to be confusing). #2 — horses did not experience a significant rise or fall in plasma cortisol or fecal glucocorticoid metabolites (FGMs).  This means that being ridden in a school program by kids who bop and pull and bounce and kick is not a significant positive or negative stressor for these horses.  Not enough to cause an acute or long-term rise or drop in FGMs or plasma cortisol. (Many caveats exist of course: these horses have been established in the lesson program for a while, meaning they didn’t flunk out so they must not hate it anyway; also, these horses had pretty astonishingly high levels of chiropractic issues — so many collinear factors here. You be the judge.)

And finding #1 — When we see horses with withdrawn behavior (in this study: ears back during feeding time, but in other studies: non-responsive behavior, facing a wall, reclusive in stall) we can pretty reasonably question whether or not their welfare might be compromised.  Just because a horse is a bit cranky doesn’t mean they have poor welfare, but it might be worth looking into.  The authors also looked into physiological measures (anemia and chiropractic issues), and there is definitely a feedback loop between psychological health, physical health, and welfare.  So we might consider that a horse who demonstrates a change in behavior, from generally perky to generally withdrawn or low-affect, might be experiencing something physical as well.

Interesting to note is that the researchers didn’t report anything about stereotypic behaviors like cribbing, weaving, etc.  I have done a lot of reading (in both horses and other species) about how stereotypic behavior might indicate welfare, and the literature is vast and, ultimately, equivocal: sometimes it means bad, sometimes it means nothing.

So there we have it. I really encourage you to read the full article if you have time, and tell me what you took from it!  We can start a little blogger journal club!

poor welfare or drugged?

* I have full access to these so please drop me a line if you’d like to know more.

^ Cortisol is often referred to as a “stress hormone”, but it really does and indicates so much more than that.  Cortisol is upregulated any time glycogen is turned into glucose to provide easily accessible energy to the muscles.  So obviously, this could be associated with both awesome exercise (like sex! or just running, I guess) and un-awesome exercise (running the fuck away from a lion). I will actually just direct you to the paper for more on this, since I feel that the authors did a really excellent job of explaining some of the complexity surrounding cortisol in the introduction.

Official stuff:

Article Source: Low plasma cortisol and fecal cortisol metabolite measures as indicators of compromised welfare in domestic horses (Equus caballus) 
Pawluski J, Jego P, Henry S, Bruchet A, Palme R, et al. (2017) Low plasma cortisol and fecal cortisol metabolite measures as indicators of compromised welfare in domestic horses (Equus caballus). PLOS ONE 12(9): e0182257. https://doi.org/10.1371/journal.pone.0182257