General Equestrian/Horse Toggle

The Trouble With “Evidence”

There are countless hoof products and practices—both verifiable and unverifiable—out there that the author says have become more popular due to salesmanship rather than evidence.

Photo: Erica Larson

“Evidence-based” is recent buzzphrase that is present in every aspect of veterinary and human medicine. After 45 years as a farrier and 32 years in surgical practice, I find the phrase overused and pretty useless—especially when people act as though it’s something new. Upon what else would you base a choice of treatment or care besides evidence?

“Traditional” farrier practice has been evidence-based insofar as the results had to justify the treatment or shoeing used. When horses were important for waging war, growing crops, moving commerce, and as major economic assets, if they were lame or couldn’t work properly, we changed the shoeing practice to improve the situation. The difference now is that we have more sophisticated measuring tools and statistical techniques to try to figure out what is “true” and works.

Most statistical techniques come down to a simple determination of “p,” or probability value: What is the chance that if we compare these two groups that look different they are actually the same? Observational studies (for instance, degrees of palmar angle change—the angle the wings of the coffin bone make with the ground—versus deep digital flexor tendon tension) use statistical information to see if the results are reproducible. I check my experiment against yours. But the old joke about “lying with statistics” sometimes holds true, either because the individual applied the wrong statistical test or because the data collected was inaccurate or, worse yet, falsified.

Research in human medicine is better-funded and of higher quality than farrier and veterinary research, yet it is still full of problems. Especially in orthopedics, a good result is measured in years of function, as is the case for the horse. A lot of “advances” and new devices that we at first thought might be big improvements have later turned out to be nothing, or they’ve caused problems worse than what they were designed to solve.

In the process of peer-reviewing scientific work, people with expertise in the area concerned (and hopefully without bias) look at evidence to see if it measures up. The philosopher Lactantius said, “The first point of wisdom is to discern that which is false; the second, to know that which is true.” If we occupy ourselves with ideas that have no basis in fact, we no longer have any time or energy to work out the truth.

There are a ton of totally unverifiable hoof-related gimmicks out there that have become popular due to salesmanship and been difficult to refute despite solid evidence against them because they achieve cult status. Many individuals and product manufacturers have advocated shoeing and foot care methods with claims of great success. The supporting evidence used to “prove” the proposed value of these techniques was scientifically invalid.

The other elephant in the horse trailer is the same as one we see in surgery: The “average” surgeon is … average; the “average” farrier is … average. If you try to compare the long-term results of therapies and treatments performed by surgeons, farriers, or veterinarians, the skill issue gets in the way because a treatment or shoeing technique that works for one person might not work for another who is not skilled enough in their vocation to make it work. The results of treatment are necessarily filtered through the skill of the person applying it. And the fact that a treatment or operation might be difficult to perform doesn’t mean that it’s wrong. It just requires more skill to be successful.

So Dr. Renate Weller (in the Evidence-Based Farriery article) is right. This research is hard to do, expensive, and underfunded. Good studies are extremely rare, but the average horse owner can’t tell the difference between reputable and questionable studies.

Too often, the secret to our success is that the horse is so tolerant and athletic that it can overcome almost any of the devices and ploys that we inflict on it.

Michael Miller, MD, CJF FWCF, is a practicing orthopedic surgeon, an AFA Certified Journeyman Farrier, and one of four Fellow of the Worshipful Company of Farriers based in North America.

Originally published in the January 2015 issue of The Horse: Your Guide To Equine Health Care.

Leave a Reply