Monday, January 2, 2017

The positive reinforcement pony

Unlike the dog training world, positive reinforcement isn't seen as much with horses.  Pats on the neck and verbal rewards are pretty much it and it's haphazard, forgotten in the heat of the moment when working on a concept.  Bits, spurs, and crops are mainstream, but giving your horse a cookie while training under saddle is seen as weird.  Leaping off of my horse and loosening his girth to mark a break through?  What on earth am I doing?  I have a lot of experts that think I'm ruining my horse when they see me whip out cookies in the midst of a training session.  You're spoiling him!  Your horse is going to get rude and nippy!  He won't work for you if you don't have a treat for him!  He won't respect you!


Ladies, gentlemen, and those that choose to not use those labels, I am here to tell you what it is actually like to have a horse that is a positive reinforcement pony.

First, some definitions:
Positive reinforcement = adding something positive as a reward, like a cookie
Negative reinforcement = taking away something negative as a reward, like releasing pressure on the bit, but this can also happen when a horse learns to evade something (and we wonder why horses learn to stop, its very rewarding to avoid the effort if the effort is seen as a negative)

Positive punishment = adding something negative as a punishment, such as a smack with a crop
Negative punishment = taking away something positive as a punishment, such as me turning away when Theo's being pushy

Your horse will get rude and pushy!

No more than any other horse.  Any horse can learn to be rude and pushy when they think a treat is coming.  Having the process of earning and receiving a cookie being formalized can actually help with pushy behavior.  Theo is not allowed to push for a treat.  He begs when he thinks there is a treat coming and he's not working (whickers, pricks his ears, arches his neck, acts cute), but he is not allowed to try to take one unless it's offered.  He is not allowed to reach for a hand that's not offered or step into a person's space.  It's important that he not get pushy or nippy.  Trainer A is very appreciative.  At the end of a lesson, she usually shares an apple or Kind bar with him.  He will arch his neck and beg, but he won't touch her, crowd her, or try to take something from her unless it's deliberately offered.  It keeps fingers and toes safe.  Theo is more polite with treats than many horses I've met because he's been taught how he has to behave in order to get one.

He won't work if you don't have a treat!

First, I always have a treat.  Always.  I keep them in my pocket at shows, when trailering, when I bring him in from the field.  No matter what situation, there is a cookie in my pocket.  And all of my breeches have pockets.  It's just how I do things.  It helps with a lot of situations, even ones with horses that aren't mine.  More than once I've had someone ask me for a cookie while loading a horse in a trailer or handling a situation like clipping.  I always have something in my pocket.  Theo assumes that I have a a cookie at all times.  It's been very helpful at shows, since he assumes he could get a reward at any time.  Lots of horses get 'ring smart' and realize they won't get disciplined in front of the judge.  I sure don't want to use my whip while in competition.  It's not a good impression for the judge.  But judges, in my experience, think handing a treat to your horse is cute or totally not worth noting.  He's not 'ring smart' because I have rewarded him in the ring at competitions.  He associates the judge's booth with cookies because I always have them just in case he does something amazing like go past without broncing. 

Second, he works regardless of me handing him a cookie.  He's on a random reward schedule, so if he doesn't get one, he assumes he needs to do more to get his cookie.  He doesn't have xray vision, he doesn't know if a treat is in my pocket.  He assumes I have one available, and if he does something good, he'll get a reward.  Random reward schedule is very important for this.  It keeps him trying even if he doesn't get a cookie immediately for any given action.  There's a very specific verbal marker that represents a cookie being delivered.  So long as I don't use that verbal marker when I don't actually have a reward, he will work with pets and cooing until it's time for his treat.  Pets and cooing serve as information for him, he's on the right track.  At the end of a string of correctly executed maneuvers, he'll get a treat.  He's just not sure where the end of that string of maneuvers is.

He's working for the cookie, not you!

One, check your ego at the door.  Who cares?  Horses need more of a reward than some mythical bond with a strange, bipedal omnivore that doesn't speak their language and controls every part of their lives.  Your horse is probably not as in love with you as you think.  Your approval is not what he lies awake at night and wishes for.  Two, I'm the one providing the cookie.  He's begging me, not the cookie.  It's a solid, concrete reward.  Petting and verbal rewards are still a huge part of working with him.  I coo to him, talk to him, scratch along his crest, and give him quick pats with a release all the time.  I use cookies to very specifically mark a behavior when I'm developing it or reward him for overall effort.

I have a degree in psychology.  I taught rats to do a lot of things using nothing but operant conditioning as part of my college work.  I have a fair bit of training for this skill between dog training and rat work.  Timing is everything.  When I first introduce something difficult to Theo, I'll shape his behavior.  He gives me an honest try, I mark the behavior with a specific word ('good boy'), followed by a cookie.  I do this a couple times to get him rolling.  Then I expect more from him to get the same reward.  For shoulder in, he got marked for the first offer of shoulder fore to start.  Then I asked for more angle before I'd give him a reward.  Then I weaned him off of the reward, asking for several reps to get a reward.  Now it's old hat and he just does the movement.  He'll get a reward if he really blows my socks off, like doing it at the canter, but for the most part he understands and gives me the behavior without needing a cookie.  He gets a pat on the neck, a verbal reward, and we keep working.

He won't respect you!

Make no mistake, Theo respects me.  Just because I formally mark and reward behaviors doesn't mean that's the only thing I use.  He's a thousand pound animal.  He tries to push me or bite me, I carry a dressage whip for a reason.  However, because of his personality, after he's been sharply corrected, I will give him a chance to earn a reward very quickly.  He gets rude during a hand walk.  I snap the chain, back him up, and give him a verbal correction.  Then I turn, act like nothing happened, and find something to play the 'touch it' game.  Lesson learned, but I also end the fight.  My horse respects me.  I'm going to guess this myth comes from the natural horsemanship world.  Yes, I've seen a lot of horses with Parelli levels that have no respect for humans.  This isn't natural horsemanship.  It's conditioning, pure and simple.

There's a lot more myths and variations, including concerns about what I'm doing to my bits and bridles.  But these are the ones I hear the most.

It's proven that positive reinforcement is more effective than positive punishment when learning.  Positive punishment is associated with increased stress in animals (and people).  If I have an option to use a technique that will lower stress, increase motivation, and increase the likelihood of the behavior being offered, I'll take it.

I don't recommend every single horse and rider pair jump into positive reinforcement with both feet.  It takes some study for the rider to really get it.  It takes careful timing and practice.  It also takes practice to manage the cookies, especially with gloves.  You have to consider your wardrobe in terms of pocket space.  My poor washer and dryer have processed a lot of forgotten horse cookies.  I don't use a treat pouch since that gives him something to visually cue on for the presence of cookies.  Cookies should be a source of mystery, magically appearing in the rider's hand from hammerspace or some other dimension.

Some horses get obsessed with the reward.  Same as dogs, some can't work with something that they want so badly.  It becomes a distraction rather than a help.  But I think positive reinforcement has a place with horses and is often overlooked or dismissed when it should be a common tool in the toolbox.  Theo is highly food motivated and used to have a terrible work ethic.  Really, what was in it for him?  He got kicked either way and work was hard.  He hated work.  Adding something he loved improved his opinion.  He now associates work with rewards to be earned, rewards that are more motivating than the random pets of someone that's also making him do the nonsense.  I could have tried kicking him more, getting bigger spurs, used the whip, any number of negative reinforcements, but we're all happier with the positive reinforcements. 

In the average ride, Theo gets about three to five rewards.  Despite the jokes, I'm not a pez dispenser.  The number sky rockets when we're adding something new that needs heavy reinforcement (turn on the haunches for the first couple weeks, teaching him 'touch it', right now it's his canter to walk transition and flying changes), but he typically gets one when I mount (he never moves a foot, even when I'm mounting on trails or at hunter paces), one after warm up, one after his hard work is done, and one when I dismount.  Two of those cookies are to reinforce his behavior when I'm mounting and dismounting.  It has saved my bacon several times that he stands completely still for both maneuvers no matter where we are, waiting for his cookie.  When we're jumping, he'll get one when he's completed a full course or when he's on break between sets with the grid. 

Dog training manuals give excellent advice on positive reinforcement, for anyone that wants to consider adding a couple of sugar cubes into their regime.  Horses, as a rule, I don't consider to be candidates for strictly positive reinforcement training because they're so big and sometimes, just sometimes, I need immediate obedience.  Like when standing on my foot.  Horses also do an awful lot of communicating in the herd with shoving and biting.  But the equestrian culture leans too far toward negative reinforcement.  I think horses need a reward to work toward, something more concrete than our approval.  The most powerful reward I have is to vault out of the saddle and loosen the girth.  That should say something in regards to how horses view their work.

I do see dressage riders presenting their horses with a sugar cube at the end of a test and it makes me feel less odd.  Maybe we'll find a spot for equestrian culture that's not so dependent on negative reinforcement or positive punishment.  I can hope so.  Some horses just need a bit more positive motivation than others.

9 comments:

  1. I love positive reinforcement! It helped me so much when Stinker was totally panicked under saddle. I would literally shove a treat in his mouth because he would totally shut down and it helped him to understand that he wasn't in trouble. To each their own, but I firmly believe in always having treats available and having well established ground rules.

    ReplyDelete
  2. 100% agree with your thoughts on treats. Judicious use of cookies has been effective for me as well. I consider the cookie to be - if well timed - like a translational shortcut. I don't speak horse and he doesn't speak human, but when I ask for a behavior and he gives it - the reward confirms that fact instantly.

    Keep up the good work. I've really enjoyed following your progress. Best wishes to you and Theo for the New Year. :D

    ReplyDelete
  3. Yup - sugar cubes are always in my pocket!

    ReplyDelete
  4. I found this very interesting- I've been thinking of doing this with Carmen and you have given me encouragement.

    ReplyDelete
  5. At my first farm we were always required to wear fanny packs filled with treats (the barn provided fanny packs right along with helmets for lesson kiddos haha) and we gave treats to horses after the completion of every task. They certainly seemed to appreciate it!

    Anyway tho. I've used a lot of training techniques focused around negative reinforcement with horses, especially with regard to yielding to pressure. Particularly around the trailer. Tho I've stalled out a little with my current horse and am waiting for more help to further develop the right responses.... And in the meantime am using a feed bucket to get him loading. Which. Now that I think about it. Probably falls into the positive reinforcement category lol. Go figure.

    ReplyDelete
  6. 1. I completely agree. Positive reinforcement can really get the horse excited to learn and overcome frustration.
    2. THANK YOU SO MUCH for my blogger secret santa gifts! I love all of them! I couldn't find your email so I'm thanking you here. Hope you and Theo had an amazing Christmas!

    ReplyDelete
  7. I do a lot of formal clicker training with my horses and discuss this very subject with my students often: I want them aware whether they're using negative or positive reinforcement or if they're punishing a behavior. I also like to get them to think through ways of training the same behavior using different training methods... but I do love reading posts like this. Makes me very happy.

    ReplyDelete
  8. Great reminder. Given that I drive more than ride my current horse there is a lot more negative reinforcement than positive in our work. It's hard to treat from the carriage :) I used a lot more formal operant conditioning and positive reinforcement with my last horse. I need to spend some time with my mare, she loves the game of +r. I use lower value treats with her than with some other horses as she will loose her manners if it's too exciting.

    ReplyDelete
  9. There is so much good info in here! I do think a lot more positive reinforcement makes a difference, but as you mentioned, not all horses can handle a food reward. In general though, a horse who has good behavior positively rewarded and confirmed seems to learn faster and be happier to learn things than a horse who is only punished for the bad things. Positive reinforcement with treats is something I use a lot on the ground, but my guy can get a little fixated, so I've been hesitant to do it under saddle for fear of exactly what your cartoon depicts. Lol. I'll have to think more on how to work it in.

    ReplyDelete