AI in Robotics: Programmed Behavior vs Machine Learning (and When You Need Each)

A robot has two ways to decide what to do. An engineer writes the decision rule, or the robot infers it from data. The first approach is programmed behavior: PID loops, finite state machines, behavior trees, and model-based planners. The second is machine learning (ML): neural network policies, learned perception, and reinforcement learning (RL). Production systems rarely pick one. They split responsibilities across both, and the split point determines whether the robot is safe, debuggable, and fast enough.

This guide defines both paradigms mathematically, compares them in tables, and shows working ROS2 code for each. It ends with a decision framework and a hybrid architecture that most deployed robots converge on.

Quick Takeaways

• Programmed behavior wins where the physics is known, the safety case must be provable, and loop rates exceed 500 Hz.
• Machine learning wins where the environment is unstructured, the input is high-dimensional (pixels, point clouds), or the model is too costly to write by hand.
• Hybrid stacks use learned perception feeding deterministic planning and control. This is the dominant pattern in warehouse AMRs, surgical robots, and autonomous vehicles.
• Sim-to-real transfer is the main cost of ML control. Budget for domain randomization and a safety-filter layer.

Defining the Two Paradigms

Programmed Behavior

Programmed behavior encodes the designer’s knowledge directly. The designer models the plant, derives a control law, and proves or tests its properties. Common building blocks:

• PID controllers for joint position, velocity, and current loops.
• Finite state machines (FSMs) for mode logic (idle, grasp, retreat).
• Behavior trees for modular, reactive task execution.
• Model-based planners such as A*, RRT*, MPC, and inverse kinematics solvers.

Machine Learning

ML replaces hand-written rules with a learned function π(a | s) that maps observations to actions or to intermediate representations. Three modes matter in robotics:

• Supervised learning for perception: object detection (YOLO), semantic segmentation, depth estimation.
• Imitation learning for manipulation: behavior cloning from teleoperated demonstrations.
• Reinforcement learning for locomotion and dexterity: policies trained with PPO or SAC in simulation.

The Math Behind Each Approach

Classical Control: PID

The PID law computes the control effort u(t) from the tracking error e(t) = r(t) – y(t):

u(t) = Kp * e(t) + Ki * ∫ e(τ) dτ + Kd * de(t)/dt

In discrete form at sample period Ts:

u[k] = Kp*e[k] + Ki*Ts*Σ e[i] + Kd*(e[k] - e[k-1])/Ts

Every term has a physical meaning. Kp sets stiffness, Ki removes steady-state offset, and Kd adds damping. You can compute stability margins from the plant transfer function, for example a gain margin above 6 dB and a phase margin above 45 degrees.

Model-Based Kinematics

For a 2-link planar arm with link lengths L1, L2, the inverse kinematics is closed-form:

cos(q2) = (x² + y² - L1² - L2²) / (2 * L1 * L2)
q2 = atan2(±sqrt(1 - cos²(q2)), cos(q2))
q1 = atan2(y, x) - atan2(L2*sin(q2), L1 + L2*cos(q2))

The solution is exact, runs in microseconds, and has known singularities. No training data is required.

Reinforcement Learning

RL maximizes the expected discounted return of a policy π_θ:

J(θ) = E_{τ~π_θ} [ Σ_{t=0}^{T} γ^t * r(s_t, a_t) ]

The policy gradient used by PPO clips the update ratio to keep training stable:

L_CLIP(θ) = E[ min( r_t(θ)*A_t, clip(r_t(θ), 1-ε, 1+ε)*A_t ) ]
where r_t(θ) = π_θ(a_t|s_t) / π_θold(a_t|s_t)

The reward function r(s, a) replaces the explicit control law. That is the strength and the risk: the policy optimizes the reward you wrote, not the behavior you intended.

Implementation: A PID Velocity Controller in ROS2

This rclpy node closes a velocity loop for a BLDC motor at 100 Hz. Anti-windup clamps the integral term.

import rclpy
from rclpy.node import Node
from std_msgs.msg import Float64

class PidVelocityNode(Node):
    def __init__(self):
        super().__init__('pid_velocity_node')
        # Declare tunable gains as ROS2 parameters (changeable at runtime)
        self.declare_parameter('kp', 0.8)
        self.declare_parameter('ki', 2.0)
        self.declare_parameter('kd', 0.01)
        self.declare_parameter('u_max', 24.0)   # Supply limit in volts

        self.setpoint = 0.0       # Target velocity, rad/s
        self.measured = 0.0       # Encoder-derived velocity, rad/s
        self.integral = 0.0
        self.prev_error = 0.0
        self.dt = 0.01            # 100 Hz loop

        self.create_subscription(Float64, 'cmd_vel_target', self.on_target, 10)
        self.create_subscription(Float64, 'motor_velocity', self.on_measured, 10)
        self.pub = self.create_publisher(Float64, 'motor_voltage', 10)
        self.create_timer(self.dt, self.update)

    def on_target(self, msg):
        self.setpoint = msg.data

    def on_measured(self, msg):
        self.measured = msg.data

    def update(self):
        kp = self.get_parameter('kp').value
        ki = self.get_parameter('ki').value
        kd = self.get_parameter('kd').value
        u_max = self.get_parameter('u_max').value

        error = self.setpoint - self.measured
        self.integral += error * self.dt
        # Anti-windup: clamp the integral so Ki*integral never exceeds u_max
        if ki > 0.0:
            self.integral = max(-u_max/ki, min(u_max/ki, self.integral))
        derivative = (error - self.prev_error) / self.dt
        self.prev_error = error

        u = kp*error + ki*self.integral + kd*derivative
        u = max(-u_max, min(u_max, u))   # Saturate to the 24V rail

        out = Float64()
        out.data = u
        self.pub.publish(out)

def main():
    rclpy.init()
    rclpy.spin(PidVelocityNode())

if __name__ == '__main__':
    main()

In production, run this loop in C++ with ros2_control for deterministic timing. Python timers jitter by several milliseconds under load.

Implementation: A Learned Policy as a ROS2 Node

The inference side of ML control is simple. This node loads a TorchScript policy exported from simulation and runs it at 50 Hz.

import rclpy, torch
from rclpy.node import Node
from sensor_msgs.msg import JointState
from std_msgs.msg import Float64MultiArray

class PolicyNode(Node):
    def __init__(self):
        super().__init__('policy_node')
        # TorchScript export: no Python training code needed at runtime
        self.policy = torch.jit.load('/opt/models/walker_ppo.pt').eval()
        self.obs = None
        self.create_subscription(JointState, 'joint_states', self.on_state, 10)
        self.pub = self.create_publisher(Float64MultiArray, 'joint_targets', 10)
        self.create_timer(0.02, self.infer)   # 50 Hz

    def on_state(self, msg):
        # Observation vector: joint positions + velocities (normalized as in training)
        self.obs = torch.tensor(list(msg.position) + list(msg.velocity),
                                dtype=torch.float32)

    def infer(self):
        if self.obs is None:
            return
        with torch.no_grad():
            action = self.policy(self.obs.unsqueeze(0)).squeeze(0)
        # Clip to joint limits: a minimal safety filter on the network output
        action = torch.clamp(action, -1.0, 1.0)
        out = Float64MultiArray()
        out.data = action.tolist()
        self.pub.publish(out)

def main():
    rclpy.init()
    rclpy.spin(PolicyNode())

if __name__ == '__main__':
    main()

The observation normalization must match training exactly. A mismatch is the most common cause of a policy that works in simulation and collapses on hardware.

Head-to-Head: Where Each Approach Fits

Sensor Stack and Compute Requirements

Failure Modes and Safety

Programmed controllers fail in ways you can enumerate. Saturation, windup, singularities, and unmodeled friction all show up in analysis or bench testing.

ML policies fail in ways you cannot enumerate. A network can output a confident, dangerous action on an input outside its training distribution. Three mitigations apply in practice:

1. Safety filters: Wrap the policy with a control barrier function (CBF) or simple limit checks. The filter overrides unsafe actions.
2. Runtime monitors: Track input novelty (e.g., Mahalanobis distance to training data) and fall back to a classical controller when it spikes.
3. Rate and torque limits: Enforce hard limits in firmware, below the network layer.

Standards such as ISO 10218 and ISO 13849 assume deterministic safety functions. Place any ML component outside the safety-rated path.

Practical Workflow: Hybrid Mobile Manipulator

A warehouse AMR with an arm illustrates the split. The architecture:

1. Learned perception: A detector (running on a GPU at 30 Hz) outputs object poses from an RGB-D camera.
2. Programmed estimation: An EKF fuses IMU, wheel odometry, and LiDAR localization into a pose estimate at 100 Hz.
3. Programmed planning: Nav2 plans the base path. MoveIt 2 plans arm trajectories using inverse kinematics.
4. Programmed task logic: A behavior tree sequences navigate, detect, grasp, place, and handles retries.
5. Programmed low-level control: ros2_control runs PID and impedance loops at 1 kHz.
6. Optional learned grasping: A grasp network proposes candidates. A deterministic collision checker validates them before execution.

A simplified behavior tree for the pick task, in BehaviorTree.CPP XML:

<root BTCPP_format="4">
  <BehaviorTree ID="PickTask">
    <Sequence>
      <Action ID="NavigateToShelf" goal="{shelf_pose}"/>
      <RetryUntilSuccessful num_attempts="3">
        <Action ID="DetectObject" pose="{object_pose}"/>  <!-- ML node -->
      </RetryUntilSuccessful>
      <Action ID="PlanGrasp" object="{object_pose}"/>     <!-- MoveIt 2 -->
      <Action ID="ExecuteGrasp"/>                         <!-- Impedance control -->
      <Action ID="NavigateToDropoff"/>
    </Sequence>
  </BehaviorTree>
</root>

The ML node produces a pose. Everything downstream is deterministic and auditable. When a pick fails, engineers can read the tree log and know which stage broke.

Sim-to-Real: The Hidden Cost of ML Control

Training an RL policy takes hours in simulation (Isaac Sim, MuJoCo, Gazebo). Deploying it takes weeks. The reality gap comes from unmodeled actuator dynamics, sensor noise, latency, and friction. Reduce it with:

• Domain randomization: Vary mass (±20%), friction (0.4-1.2), motor strength, and sensor noise during training.
• Actuator networks: Learn the real motor response from logged data and include it in simulation.
• Latency modeling: Add 10-30 ms of observation delay to match the real stack.
• System identification: Measure the real robot and fit the simulator parameters.

Programmed control has a smaller gap. A well-identified plant model transfers directly to hardware, which is why model-based control remains the default for industrial arms.

Decision Framework

Answer these questions in order:

1. Is the system safety-critical or certified? Use programmed behavior for the safety path.
2. Is the plant model known and accurate? Use classical control or MPC.
3. Is the input high-dimensional and unstructured (images, point clouds)? Use ML perception.
4. Is the dynamics model too hard to derive (contact, deformables, legged terrain)? Consider RL or imitation learning, with a safety filter.
5. Do you need explainable failures for maintenance or regulation? Keep decision logic in behavior trees or FSMs.
6. Do you have data or a trustworthy simulator? If not, ML is not yet an option.

FAQ

Is machine learning replacing classical control in robotics?

No. Classical control still runs the inner loops of nearly every robot, including those with ML policies. ML expands what robots can perceive and handle, while PID, MPC, and impedance control keep actuation stable and verifiable.

When should I use reinforcement learning instead of PID?

Use RL when the dynamics are too complex to model, such as legged locomotion on uneven terrain or in-hand manipulation. Use PID when the plant is well-understood and linear enough, which covers most motor, joint, and velocity loops.

Can a robot use both programmed behavior and machine learning?

Yes, and most production robots do. A common pattern is ML for perception, deterministic planners for motion, and PID or impedance control for actuation, with a safety layer that can override learned outputs.

Why do RL policies fail on real robots after working in simulation?

The cause is the sim-to-real gap: unmodeled friction, actuator lag, sensor noise, and communication latency. Domain randomization, learned actuator models, and latency injection during training reduce the gap.