Sep 13, 2020 16 min read blog

Feedback Loops and Controllers

Control is critical to modern life; humanity has harnessed science and engineering to control so many things. From climate control in buildings to cruise control in automobiles and process control in industrial manufacturing, control techniques are everywhere and make many aspects of our life easier and better. But control techniques are not only applicable to engineering problems, in this article I hope to convince you that we can find analogs to the concepts and techniques used for engineering problems in every day decision making. Furthermore, we can use these techniques to improve ourselves, make better decisions, and achieve our goals.

The anatomy of feedback and control

In order to effectively design any system, it is crucial to understand how other examples of similar systems work, starting from scratch is rarely a good idea and control systems are no exception. In my experience a generalized control system has 4 reoccurring phases. (1) determine what the desired state is, (2) observe the current state, (3) decide how to make the current state into the desired state, (4) make a change, (5) repeat. There are many ways of breaking down any system and there will be more or fewer stages depending on the system in question. However, most aspects of most control systems will fit into these 4 phases, so let’s investigate each one a little further.

Note: "The System" is the object or environment being controlled by the controller

Phase 1: The Ideal State

Successfully making effective change to a system is impossible without first defining what success is, the control system must know what it is striving for. For example, what is the desired temperature of the room? What is the desired speed of the vehicle? Without knowing what the "set point" or "end goal" is the system will probably not make it happen.

Phase 2: The Current State

In order to know what to change the controller generally must know what the current state is. Is the room hotter or colder than desired? Is the vehicle speed higher or lower than the driver wants? In engineering examples, the process variable is usually measured through transducers (sensors) like temperature or speed sensors which constitute feedback to the controller to let it know the current state.

Phase 3: What Changes Should be Made

Now that the controller knows the current state and the desired state it will need to decide: what should it do to make the current state closer to the desired state? If the room is too cold, then the heater should be turned on. If the car is going too fast the throttle should be reduced. Not only is it crucial to know what direction to move the system (hotter, colder, faster, slower) but also by how much. For example, if a car is currently travelling at 100 km/h and the desired speed is 50 km/hr a larger change is required than if the current speed was 55 km/hr

Phase 4: Make the Change

A control system without a way to make any changes is just wishing for the system to meet its desired state. The controller must make a change in the system, typically by means of an actuator. For example, a furnace is the actuator for a heating system and a servo might move the throttle on a vehicle. The system will change in response to the actuator. The furnace will heat up the room and the throttle will slow down the car.

Phase 5: Repeat

A critical aspect of control systems is that they are constantly re-evaluating their environment and striving to meet the current goal. After the action is performed the controller immediately looks for new inputs and makes another change. This is the real power of controllers; they constantly adjust their output command based on the latest inputs and can therefore react to new changes in the system.

Control Systems in Engineering

At the risk of being too technical I would like to introduce control systems as engineers see them. I'll keep it simple so bear with me on this one. But first some definitions

The system: The environment, object, or process (not including the controller). When talking about control "the system" has one or more variables that have a desired value. For example, the system of a car has a variable of speed that we want to control.
Process variable: A measured parameter of the system that we desire to set at a desired value. For example: the car's speed.
Controller: A device that determines an action to perform on the system to bring the process variable closer to the desired value. In this example, the cruise controller.

Transfer Functions

Engineers like math, so they made transfer functions... Ok, not quite true, transfer functions are useful tools that are used to solve controls problems and are traditionally defined with mathematical concepts. Understanding them is critical to knowing how to make an effective control system.
Mathematically, the transfer function of a system is defined as the response of said system to an "impulse" input, or what happens when a single pulse goes through the system. In the Laplace domain it is defined as the outputs divided by the inputs. In non-geek speak it is what happens when some special, known input goes through the system. The important thing to remember about transfer functions is that every system has some sort of transfer function, even if we can't fully measure or define it. Given a certain set of inputs it will (or at least should) give a certain output. The more we understand about a system's transfer function the better we will be able to control it.

Open Loop Control

The simplest controllers are what engineers call "Open Loop", meaning that they do not receive feedback from the system. Without feedback these controllers must make an educated guess as to what the system is doing and give the actuator an output command hoping that the system will react properly. Open-loop control is used when precise control is not important to successful completion of the controller’s mission. A typical application is a clothes drier, where the controller will turn on the heating element to a specific power, say "low" or "high" and hope that the clothes dry properly. In this case if the heating element failed the controller would have no way of knowing, and the clothes would come out damp.

Closed Loop Control

In contrast, "Closed-loop" controllers get feedback from the system that they are controlling, they have knowledge about the current state of the system and can observe if it is getting closer or farther from the desired state. For example, a cruise controller senses how fast the car is currently travelling, and therefore can compensate for changes in the road condition. When driving up a hill the cruise controller will increase throttle to maintain the set speed without requiring input from the driver. The cruise controller does this by sensing that the vehicle is slowing down and determining that more power is required from the engine to maintain speed.

System Dynamics

If the system were static, that is its state never changed, it would not need a controller. All systems that we attempt to control are dynamic and change over time. Hopefully the system dynamics are based on some rules or equations that we can model or at least guess at. The nature of every system is different and that is why different kinds of controllers are needed. Additionally, almost every real-world system is affected by disturbances; that is some forces unknown to the controller designer that will tend to move the system away from the desired state.
In engineering we try to model the system dynamic as closely as we can, but often can't predict exactly how it will behave to every input. If we could know exactly how the system would behave in every situation, we would just use a closed-loop controller. Unfortunately, the world isn't so simple, and designing systems that can deal with disturbances as they arise is critical to building a useful control system.

Loop Repetition

It is critical that he controller keep calculating, and it's critical that each cycle of the feedback loop occur in a timely manner. How useful would cruise control be if it only updated every 10 seconds? The car could slow down significantly by the time the controller realizes, only to apply maximum throttle and boost the vehicle way over the speed limit. Short update times are critical to ensuring the controller does not miss any disturbances before they become too big to control.

How Engineers Design Control Systems

The job of a controls engineer is to determine the most suitable controller which will result in the desired change in a system. Since the system is changing the controller must also change in response, and it is the engineer who must program the rules or logic into the controller. Here is how it all comes together: First, the designer will determine the desired result, does the system need to dry clothes or keep a car moving at a constant speed? Next the system to be controlled is investigated and the designer attempts to determine how the system will respond to different control inputs. Sometimes parameters of the system need to be measured in order to be controlled, so a sensor or transducer is chosen to report the system state back to the controller. The engineer will then make a good guess as to what a functional controller might be, what it's logic or transfer function should be. Finally, the designer will model and test the entire system repeatedly. Simulation, testing, and tuning are by far the most important part of the controller design process. Engineers will spend days or even weeks perfecting control systems to make them work flawlessly, tuning all of the different parameters just right so that the controller can react perfectly to any changes in the system.

What can go wrong?

The best controllers will move the system exactly to the desired state in the shortest possible time (critically damped). Some other controllers will get to the final solution but will overshoot and oscillate around the desired state until the desired state is reached (underdamped). Another possibility is never reaching the desired state at all, these controllers don't apply enough force to get the job done (overdamped). The worst controllers will cause havoc in the system by applying the exact wrong inputs at the wrong time (undamped), resulting the exact opposite to the designers intended consequences. For example, if a cruise controller commands acceleration when the vehicle speed is already above the desired speed the car will soon go out of control. It is the job of the designer to get the controller to be useful in a given context and ensure the controller works for all of its intended use cases. For more on damping see Damping Ratio - Wikipedia.

Summary

Open loop control is only good for very simple problems, for a controller to be useful at controlling dynamic systems it needs to know the state of the system, it needs to have feedback.
Closed loop control is much better at controlling dynamic systems, but it is more expensive and complicated to implement. Tuning dynamic controllers can be difficult.
All parts of the controller and system have dynamics. Everything changes over time and the state of the system and the controller are constantly shifting.
Transfer functions define the output of a system given a certain set of inputs. Every system has a transfer function whether it is known or not.
Control loops need to happen fast in order to control a system properly, the speed of the loop depends on the system being controlled.
Delays in information coming into the controller can cause huge problems. Inputs to the controller should be as close to real-time as possible.
Bad controllers can be worse than no controller at all and can cause massive problems by applying the wrong inputs to the system.

"Why do I care?" I hear you say... "I'm not designing control systems"
And you would be right, as much as designing control systems is useful for engineering problems it does not seemingly apply to the everyday experiences of most people. However, each of these concepts can be directly applied to helping us determine a better way to make decisions and affect change in our environments. It's all about loops.

John Boyd and the OODA Loop

John Boyd was a fighter pilot, when working in the military he developed the OODA loop to improve his tactical advantage. Boyd's OODA loop is a feedback loop which goes like this: Observe, Orient, Decide and Act. Observe means to take in data from the environment. Orientation means to process that raw data into meaningful information which describes the current state of the environment. Decide means to determine what change needs to be made to obtain a tactical advantage. Finally, Act means to perform a change which affects the system.

The OODA loop was the direct result of Boyd’s experience trying to get an edge on his adversaries when flying sorties. His thesis is that if one can Observe the battlefield, process the information correctly, decide the best course of action, and perform that action consistently, and, the critical part: faster than their adversary, they will win. Of course, there are other factors, but in general the pilot who can quickly react to an unfolding situation and use it to his advantage will win the battle. Boyd's model is iterative, in that the pilot must constantly be going through the steps of the OODA loop in order to gain and maintain the upper hand. The pilot must also correctly perform every stage of the loop, a mistake in any stage will not produce the upper hand he needs to win.

Observe

In order to get the upper hand on an opponent the fighter pilot must first be able to maintain situational awareness and pay attention to the right things. The pilot must not only know his speed, altitude, position, ect. But he must also be able to observe the enemy. Without obtaining the basic data on the state of his environment the fighter pilot will not be able to act properly.

Orient

At its most basic form the orientation phase asks, "what does the data mean?" The pilot must be able to correctly analyze the information he perceives and determine its meaning in the context of his situation. For example, if the opponent is attacking from a higher altitude it means the pilot is at an energy disadvantage and must act accordingly to fend off the attack. Having an accurate mental model of the battlefield is critical for the pilot to be able to quickly assess the situation and decide on a course of action.

Decide

Once the pilot thinks he has an idea of what is going on he must decide, will he turn left or right, up or down. The decision is also greatly influenced by the pilots’ mental model, his experience will act as a guide to aid in selecting the best action for a given situation.

Act

Of course, it's not much good knowing what one should do if one never does it, the pilot must actually move his controls to make his airplane perform the maneuver that he decided on, which will actually put his aircraft in a tactically superior position.

Boyd's thesis is that, as an individual acting in an environment with one or more adversaries one must not only perform each phase of the loop but perform better and faster than the opponent in order to gain and maintain a tactical advantage. By making better decisions faster we can "get inside their loop" as Boyd says. Additionally, part of getting inside the adversary's loop is to interfere with their OODA process. If we can use an optical illusion to affect their perception, make a move that is completely outside of their expectations, or limit their actions then we can gain the upper hand.

What this means to an individual

Now it's time to draw some inferences.

Observation is like the sensors, in order to make the best decisions one must pay attention to the right things, collect the proper data. Garbage-in Garbage-out as they say. Without actively seeking the information needed to decide, one greatly reduces the chances of making the best decision. We must remain conscious about the information that we are obtaining. Working in the real-world means making decisions under uncertainty, we must be aware that we will not and can not know everything before an action is required. We must be cautious about the information that we are using to decide: Is it complete? Is there too much information clouding our judgement? Do we trust the source of our information?

Orientation and Decision is like the controller. Without the proper rule set the controller will not make the right commands to create the appropriate changes to the system. As decision makers we first understand the state of our environment, and then we decide what we are going to do about it. These two phases are greatly affected by our mental models of the world. It is critical that we constantly strive to better our understanding of the environment that we are acting in so that we can make sense of our observations in this broader context. We must also strive to make the best decision for a given context, this is most often done by learning about similar decisions that others have made, and reflecting on the results of our own decisions. Improving one’s orientation and decision-making skills are akin to an engineer tuning his controller. The more effort that is put in the better the controller will be, and this will lead to better outcomes.

Action is the final step: one must execute the decision, just like the actuator of a control system. The only thing that will change the state of our environment in our favor is putting in work, the right work, the work that the decision demands. There is no point in putting in the effort of making the best decision possible if we are not going to execute on it. Action is the most important step in upgrading our mental models because every time we take an action we are, or should be, learning. Each time the world responds to our actions we get a piece of feedback. We should be observing this response and comparing it to our expectation. Only by doing this will we constantly be learning, upgrading our mental models, tuning our controllers, and getting better at making decisions.

OODA loop in organizations

John Boyd also realized that the OODA loop is applicable to so much more than the individual fighter pilot. A powerful application of the OODA model is to organizations. OODA loops are constantly occurring at every level in every organization, weather the participants are aware or not. At every level in which decisions are made the OODA process is occurring in some way, shape or form. The quality of the final decision is dictated by the work put in at each step. Therefore, it is important to consciously improve each step of the OODA process in order to make better decisions at all levels of the organization. Loops can be analyzed at three different organizational levels: individual (already discussed), tactical, and strategic.

Broadly speaking the Tactical OODA loop can be though of as the steps needed to carry out the strategy, or "what happens on a daily basis". These loops are typically smaller, run faster, and are usually internal to the teams or departments of an organization. For example, an accounting department will need to answer questions like "where do we store our files" and "what is our process for billing our customers". The resulting decisions will drive the work of individual contributors on the team, who in turn are running in their own OODA loops to bring the tactical goals to life.

Strategic OODA loops are used to define the overall mission, goals, and direction of the organization. Problems on this level include "how should our business model change in response to evolving global markets" and "what products do we need to bring to market to stay ahead of our competitors". Strategic OODA loops will take the longest to complete and are the responsibility of the upper levels of management in an organization. Since these loops typically affect the work of so many people, and the timeline for each phase is so long they are the most expensive if done incorrectly. Great care must be taken when making strategic decisions as it will affect every aspect of the organization’s future.

Almost every action undertaken by an organization is the direct result of an individual contributor and their OODA loop. The actions of individual contributors are the result of decisions at a tactical level, and decisions at a tactical level are the direct result of mandates created at a strategic level. Communication down the chain is the only way to act on a decision made at a strategic level. Without great communication and buy-in from team members within the organization strategic objectives will not affect tactical decisions, tactical decisions will not affect the actions of individual contributors, and the strategic objectives will not be completed.

How can Organizations use the OODA model?

Evaluate and improve the OODA process. Ensure all aspects of the OODA model are considered when making decisions at any level of the organization.
Ensure smaller loops are being properly influenced by larger ones. Individual actions should come from tactical direction which is the result of strategic objectives.
Improve the speed of all loops, actively look for feedback and make organizational changes as quickly as possible to gain an advantage over competing organizations.
Empower individual contributors by giving them insight into the strategic objectives and tactical direction, let front-line workers figure out how to solve the front-line problems with the guidance of the higher-level objectives.
Seek feedback to determine if actions are leading to desired outcomes. If not, observe the problem, evaluate what went wrong, update the decision and make changes as quickly as possible.

What can we do today?

Everything happens in a loop, recognize the loops that you use to make your decisions. Collect feedback to evaluate your decisions and constantly strive to reach your full potential.
Upgrade your mental model. The decisions you make are based on your preconceived notions of how the world works. You will not make good decisions if you do not understand the world around you.
Move fast and break things. If you aren't taking action and making mistakes then you are not learning, without feedback your mental model will stay stagnant and you won't be improving.
Pay attention to what you pay attention to. You control what you observe; so observe the right things. You must have an appreciation for the uncertainty in information as well as information that you might be missing.
Speed up your OODA loop, but make sure all of the steps are getting the attention that they deserve.
Recognize and utilize different levels of OODA loops. Strategic loops should be used to set and update your goals, tactical loops should determine what needs to happen next, and implementation loops should be used when actually solving problems.