Animal Training & System Theory
One of the things that readers have raised concerns about over the years I’ve been running this blog is that training - any sort of animal training, even with positive techniques - is manipulative and problematic. I want to talk a little bit today about how the building blocks of training techniques (like reinforcement and punishment and the different operant conditioning quadrants) are actually fundamental parts of how organisms learn to exist in their environment successfully. Because, at the end of the day, what training is really doing is giving us a way to communicate effectively with other organisms.
Let’s recap the basic vocabulary of operant conditioning really fast:
Positive (+): adding a stimulus to a situation
Negative (-): removing a stimulus from a situation
Reinforcement ®: make a behavior more likely to happen again
Punishment (P): make a behavior less likely to happen again
So when we’re looking at the four ways you can influence an animal’s behavior, we’ve got:
Positive Reinforcement (R+): adding a stimulus to make a behavior more likely to happen again
Negative Reinforcement (R-): removing a stimulus to make a behavior more likely to happen again
Positive Punishment (P+): adding a stimulus to make a behavior less likely to happen again
Negative Punishment (P-): removing a stimulus to make a behavior less likely to happen again
Every living organism is constantly learning from interactions with its environment - effectively, they are constantly being trained by the natural world for how to best survive in it. This includes humans. The world around us trains us from the moment we’re born. The operant conditioning language gives us a way to easily categorize how the experiences we have shape how we choose to act going forward. For instance:
If you touch a hot stove, you get hurt. This functions as positive punishment in the situation, as the stimulus of heat is added to your hand, and serves to (hopefully) discourage you from touching hot stoves again.
If you’re hungry and you look in a bush (or in a modern allegory, you open the fridge), you find berries/snacks. This functions as positive reinforcement in the situation, as the stimulus of the tasty things is added to the situation, and you’re now more likely to check that source of food again the next time you want food.
If you’re too exuberant and energetic in trying to approach an animal, the animal will probably back away or leave. This functions as negative punishment in the situation, because the stimulus of the animal is removed from the situation, and you will be less inclined to try approaching an animal in that manner again.
Not only does the environment around us train us to interact with it successfully, but we as humans also train each other constantly. Every time you give someone feedback on their behavior, either verbally or with your reactions and body language, you are effectively training them on how you prefer to be interacted with. If someone makes a crude joke and you walk out, that’s negative punishment. If you give them harsh verbal feedback about why that sort of thing isn’t okay (and they care about that sort of thing), that’s positive punishment. In both cases, you’re doing something to decrease the probability of them making that joke again - it’s just that one way, you remove your presence and attention as a stimulus, and for the other, you add the stimulus of feedback they don’t like. Similarly, laughing at someone’s joke reinforces the behavior of telling the joke, as they’re getting a positive stimulus in response.
I want to talk a little about the theory behind why this type of learning/training type of communication with the world around us and the individuals in it is so crucial for any type of organism. To do so, I want to quote from a paper that looks at understanding animal behavior from the perspective of systems theory (Semyonova, 2002). The text refers specifically to dogs, but it is applicable to any living organism. Some of it is a little dense and technical, but please bear with me - I promise it’ll be worth it.
Let’s first look at what talking about a dog as “a system” means.
“When we look at a dog, what we are really observing is a creature that is a discrete and complex living system in itself, composed of many smaller systems (e.g., cells and their cellular organs, tissues, organs such as brain and heart). All of these smaller systems have an effect on the behavior of the whole that we are observing under the name “dog”. (…) A domestic dog is a complex, self-organizing, structurally plastic system.”
Fairly straightforward: dogs (or any animal) are simultaneously an individual animal and also an entity made up of lots of little individual things that keep them running. The"goal” of the systems that make up an animal (at the most basic level) is to find and maintain a state that keeps it alive and healthy. However, animals also are constantly immersed in and interacting with the systems around them.
“At the same time, this system we call “dog” is situated in an external environment. Events in the external world can trigger changes inside the dog as a system. The dog will try to restore some kind of internal equilibrium, but as it does this it will have to take the outside world into account. External factors may limit the choices a dog has.
So, a dog can be influenced by the external system as a) a discrete individual animal and b) as a set of systems that have to react to change around them. It’s like that video most people watched in middle-school about the powers of ten that zoomed out from a single DNA strand to the entire solar system - as you zoom out, you see how things that look like single stand-alone things are part of something bigger, and then that those bigger things are just pieces of something even larger. Systems that stand alone at one level are part of bigger systems. Here’s the really important part for understanding why this is important to animal behavior:
“Interaction with the external world is not a one way street: the dog’s behavior will, in many cases, trigger a change in the outside world: output returns to the dog as input. As a dog seeks internal balance, it will often simultaneously have to control how its output affects the external world and find ways not to disturb an equilibrium in parts of that external world. Thus, behavior is not a result of static traits, but of a complex interaction of internal and external variables and processes, and of several levels of organization changing and having to be managed all at the same time.
In other words, in studying the organism called “dog”, we are focusing on only one level of organization in a multi-level system, while the dog deals with all those levels at once. All these levels of organization affect and are constantly being affected by all other levels(:)
(cellular<-> organic <-> dog <-> social system <-> habitat).
While changes at one level neither cause nor specify the changes at another level, they constitute perturbation on other levels; the system may, at any level, seek a new attractor to accommodate the change. A description of any one level of a system (e.g., the dog) must try to find and include at least the relevant perturbations that originate at other levels (e.g., a social system) as the organism attempts to juggle optimization on various levels simultaneously.”
In plain language, what that’s saying is that when animals (or people!) do anything, they’re trying to maintain system balance not only internally but in the world around them - and that the feedback they get, at both levels, will continue to shape their next actions. So when we talk about an animal’s behavior, we have to make sure we’re taking all the different systems an animal is part of and how they’re relevant into account. For example, dog that gets hurt is now going to be juggling both a shifting internal system (the physiological response to pain, and at a deeper level, the cellular response to those pain signals) as well as having to deal with / resolve whatever in the external system around them caused that pain.
It’s exactly the same with people - if you do something in a social setting that gets you feedback, you’re going to have to juggle both your internal reaction to that feedback (maybe being angered by it, which causes all sorts of changes in the body and brain) and how you handle it in that moment (maybe apologizing because it de-escalates the situation). However, the physiological response you have to being angered may affect the manner in which you apologize, which in turn will change how the person you’re interacting with receives your apology, and their behavioral feedback in that moment will then affect your internal system again.
Why does this matter with regards to animal training? It’s because there is generally not a single “best” way for the system of an animal to maintain equilibrium - it will generally have multiple “good options” that can work. The paper uses the term “fitness landscape” to help visualize this.
“Each dog has an internal fitness landscape, a sort of map of all the possible system states, some of which provide more opportunity for success than others with respect to various parameters and system functions. This fitness landscape shows the rating of each option in terms of some attribute or achievement that pertains to the system’s optimal condition according to its own internal criteria. Higher hills represent preferred states and lower hills the less preferred states. An organism or any other system will migrate between fitness hills, trying to optimize its position on this fitness landscape.” (…) ”Moving between these optima (or fitness hills) involves the balancing of many variables and is relevant to a current niche rather than to any imposed static function or criterion. There may be several more or less equally preferable [fitness hills] available at any one time, again depending on the combination of conditions prevailing at that moment. The system itself constrains the options available and chooses between them referring only to internal factors; fitness is determined with respect to internal criteria.”
Now, it’s really important to note the bit here that “picking” a fitness hill isn’t a conscious choice on an animal’s part when we’re talking about internal systems - the biological systems within a dog adapt to input without regard to how the dog “wants” them to. However, external systems do exert pressure and influence on how internal systems operate, because the system that is “dog” has to operate within the system that is “the environment”.
“The environment becomes a factor in selecting which of the attractors available to the system will be chosen at a given moment, although the choices are still made according to internal rules and constraints.”
What plays a big role in how external environments become a factor in behavioral decisions? Learning! A successful animal isn’t going to let all that feedback from external systems only affect it once - it’s going to remember it and use it to modify how the internal system responds to similar stimuli in the future.
“On the level of the organism, the fitness landscape and the migration across it will be affected by the dog’s history. This history will play a role in the animal’s mapping of the heights of various fitness hills, and in determining the choice whether or not to attempt an adaptive walk to a higher peak, the tactics an animal will use to achieve this migration, and the routes it may choose from. If it does decide to migrate, a dog must have a plan. A plan is any attempt to predetermine the trajectory of an adaptive walk across the fitness landscape. This involves predictions about the animal’s own actions, about the actions of others in the system, as well as how these actions will affect the landscape and the system as a whole. Lack of precise knowledge of starting conditions, input and/or other agents in the system can be responsible for lack of precision in predictions. New information may effect the plan and the trajectory, as well as changing the fitness landscape. Beliefs (in behavioristic terms: learned associations, conclusions about prevailing contingencies, superstitious learning) can also effect strategy. (Beckerman 1991, passim) Thus, a dog’s movement within its state space is a result of learning as an ongoing production process.”
The more an animal knows about the variables in it’s environment, the better it can “pick” the fitness hill that will be the most successful in any given situation. The less it knows, the more likely it is to make a less-optimal choice. And what sorts of knowledge does this paper bring up as things that can help an animal navigate it’s internal fitness landscape successfully? Predictions about the actions of other entities in the system and how they affect the animals and learned associations!
And this is where, finally, the way the surrounding world “trains” individuals that are interacting with it and animals-as-systems-theory intersect. A system of “dog” that is hungry and has tried to hunt a porcupine and come away with a mouth full of quills - in response to which the internal system within the dog undergoes drastic changes - is probably going to learn from that experience; the experience of interacting with the porcupine-containing “environment” system in a way that causes pain will discourage the “dog” system from attempting to deal with hunger in the “internal-to-dog” system by biting a porcupine in the future.
When we’re talking about animals in human care, what we’re looking at is a situation where we control most (not all, but most) of the external system that they’re interacting with. There are absolutely still birds and weather and delivery trucks and scary objects that fall off of counters and squirrels that run through the backyard - but we control where the animals live, where / when / what they eat, when they have access to new areas, what other individuals they meet, what they can and can’t do, etc. This means that the systems of “animals in human care” - we’ll just keep using dogs as the example - will be most successful when they understand and can effectively predict the workings of the things within external system that we control.
They will learn predict how the system around them works naturally, from the feedback they get from interacting with things in it: getting positive attention (which gives their internal system a dopamine rush) when they interact with a person will encourage them to interact with the person more; barking at a delivery van to self-soothe (because the sound of the vehicle approaching often is coupled with new stimuli and/or a startle experience when the doorbell rings, which makes the internal state of the dog “on edge”) and finding it calming will encourage the barking behavior to occur every time that truck is heard.
However, the biggest thing our dogs need to learn to be able to predict is the behavior of another complex system, “the human”, and that system is even more obscure, because doesn’t even use the same communication signals as other “dog”-like systems and it’s frequently not even consistent in the feedback it gives! We, as humans, are highly complex systems because of that whole “language and society and culture” thing, and we’re often the hardest thing an animal has to co-exist with for them to learn to predict.
When you train an animal, you’re not just using a stimulus to make an animal do or not do something again. You’re actually tapping into the way organisms learn to interact with their environments, but you’re hacking it so that the communication between two distinct systems (you and the dog) is super-effective. Your dog could probably learn that it’s a more adaptive strategy to not jump on people (and instead sit at their feet for pets) through trial and error and lots of getting yelled at and walked away from… or you could figure out what feedback makes the “internal-to-dog” system likely to view a behavior as successful and adaptive for the situation and then give the dog that feedback super clearly and consistently.
Now, we’re talking about this because folk were concerned that any type of training - even basic “don’t tackle Grandma” manners work - wasn’t ethical because it was manipulating animals to act against their own interests. Well, remember, the actual choice of what “fitness hill” to utilize an animal makes is entirely dependent on internal-system criteria. You’re not going to be able to force an animal to do something that isn’t in line with the needs of the internal system (think about trying to work with a dog that’s overly excited or stressed - no feedback you give in the external system will be powerful enough to override the functional needs of the dog in that situation). As training should always be based on voluntary engagement, this results in a dynamic where animals engage with their trainers repeatedly specifically because they find that the information conveyed during training is beneficial to the internal system as well as improving their functionality within the external system.
The world around any individual is training them constantly, and their internal system is learning from it and adapting to it. When we train animals in our care, we’re effectively cutting through the extraneous noise of inconsistent feedback and helping them learn the best way to be successful in a given scenario as efficiently as possible. Whether what specific things you’re choosing to train an animal to do is ethical or not is an entirely different question - but the basic act of training is simply utilizing precise, effective communication to transmit information about coexistence between the systems of “animal” and “trainer”.
