Learning Quadrupedal Robot Locomotion for Narrow Pipe Inspection

In plain words

Pipes are hard to inspect. They are narrow, dark and often slippery, and a wheeled or tracked crawler does not fit every bend. This paper asks whether a four-legged robot, the kind that walks like a dog, can be taught by trial and error to walk through a narrow pipe on its own.

The robot learns in a physics simulator. The simulated pipes are built to be harder than the real ones: the walls are slippery, the entrance is raised, the robot is pushed at random, and some runs add small blocks inside the pipe. The training is done in three stages. The robot first learns to walk in a wide pipe, then in a narrow one, and finally in a narrow one with obstacles. The control program also sees a rough map of the pipe floor and ceiling around its body, which is used only while training. The reward favours moving forward at the commanded speed, using little energy, avoiding walls and staying near the middle of the pipe.

The authors then tested the learned walking on real pipes made of PVC. Those tests succeeded without obstacles, but the real success rate was much lower than in simulation. The method is a first step and the paper says so.

TL;DR

• A reinforcement-learning policy (PPO) controls the 12 joints of a Unitree Go2 to cross a cylindrical pipe §II-A §III-A §III-D.
• Training uses a privileged 296-value vector: an 11 by 12 height scan of the floor and of the ceiling, base velocity, and simulation parameters eq. 3 tab. I.
• The reward has seven terms: linear and yaw velocity tracking, three energy penalties, contact, and centreline distance §II-C tab. II.
• Author-reported: 100% success in simulation for 0.3 m and 0.2 m pipes without obstacles at stage 3 and 93.75% with obstacles at 0.2 m tab. IV. On three real pipes (radius 0.19 to 0.24 m) the robot crossed without obstacles in all trials. With obstacles it reached 46.5% on one pipe tab. V.
• The paper is short, the real-world trial counts are small, and several details needed to reproduce it are missing. See Pitfalls.

Problem

Narrow pipes restrict the motion of conventional inspection robots, and the authors argue that model-based locomotion controllers struggle in such confined, irregular terrain §I. Standard learned locomotion methods are said to rely on terrain information that is hard to obtain inside a pipe, and on reward designs aimed at open terrain, not at keeping away from walls §I. The paper does not measure these claims against other methods.

The task is modelled as a Markov decision process with joint-position commands as the action §II-B1. The paper does not state the control frequency or how the actions are scaled.

Method

The approach has three parts: a simulated pipe terrain, an observation design with a privileged height scan, and a reward function fig. 1. A two-stage setup is described in the caption of fig. 1. In the first stage a policy is trained with the privileged scan. In the second stage a policy that takes a depth camera image is trained to imitate it. The text itself does not describe the second stage. See Pitfalls.

Terrain. A sub-terrain is an 18 m by 4 m rectangle. Its first 2 m by 4 m are a platform where the robot starts; the rest is a pipe §II-A1. Each pipe is a cylinder whose axis runs along the x axis. Its floor height is a circular profile in y eq. 1 §II-A2. Training uses a grid of such sub-terrains: 10 rows for difficulty levels and 40 columns fig. 2.

Curriculum. The paper uses a threshold τ = v_x · T, where v_x is forward speed and T is the episode length §II-A1. A robot that goes farther than 0.8 τ moves up one level. A robot that goes shorter than 0.4 τ moves down one level. A higher level means a smaller pipe radius §II-A1. The paper gives no value for T and no limit on the number of levels.

Obstacles. Between 0 and 4 obstacles are placed on the inner pipe surface. Each is 0.1 m to 0.3 m high, 0.2 m to 0.5 m long and 0.1 m wide §II-A3.

Sim-to-real modifications. The pipe walls get a low friction (static 0.2, dynamic 0.1). The pipe entrance is raised by 0.1 m. A random external force is applied to the base during training §II-A4. The magnitude of the force is not stated.

Observation. The proprioceptive vector has these parts: base angular velocity, roll and pitch from the IMU, gravity, commanded forward velocity, joint positions and velocities, the last action, and the privileged vector eq. 2 tab. I. The paper gives the total as 49 values eq. 2 (see Pitfalls).

The privileged vector has four parts eq. 3 tab. I:

• downward height scan, 11 by 12 = 132 points, sampled around the robot;
• upward height scan, 11 by 12 = 132 points, measuring the distance from the robot base to a virtual ceiling at the same grid positions §II-B2;
• base linear velocity, 3 values;
• environment vector, 29 values: a mass parameter (4), a friction parameter (1) and motor strength (24) eq. 3.

The privileged vector is compressed by an encoder. The paper names a feedforward network only as an example §II-B2. The encoder's size is not stated.

Reward. The reward is a sum of the following terms. The weights are in [tab. II].

• Linear velocity eq. 4 rewards the component of the robot's velocity along the target velocity, capped at the command magnitude, with a small constant C = 1e-5 in the denominator §II-C1. The heading idea is taken from [cheng2024extreme].
• Yaw velocity eq. 5 rewards yaw-rate tracking through an exponential of the absolute error §II-C1.
• Energy eq. 6 eq. 7 eq. 8 penalises torque squared, change in torque, and joint acceleration. The paper cites [rudin2022learning] for these forms §II-C2.
• Collision eq. 9 penalises each contact whose force exceeds a threshold. The threshold value and the list of penalised bodies are not stated §II-C3.
• Centreline eq. 10 penalises the mean distance from points on the robot base to the pipe centreline §II-C4 fig. 4.

# Pipe floor height along the lateral axis  [eq. 1] [§II-A2]
# Input:  lateral position y, pipe radius r, centre y0, centre height z0
# Output: floor height z

def pipe_height(y, r, y0, z0):
    # Circular cross-section; the profile does not depend on x  [§II-A2]
    return sqrt(r**2 - (y - y0)**2) + z0

# Terrain curriculum  [§II-A1]
# Input:  current level, distance travelled in one episode (m), speed v_x, T
# Output: new level

def update_level(level, distance, v_x, T):
    # Threshold tau = v_x * T  [§II-A1]
    threshold = v_x * T
    if distance > 0.8 * threshold:
        # Harder: the pipe radius shrinks at the next level  [§II-A1]
        return level + 1
    if distance < 0.4 * threshold:
        # Easier  [§II-A1]
        return level - 1
    return level

# Reward for one control step  [§II-C] [tab. II]
# Input:  robot state, contact forces, joint torques
# Output: scalar reward

def reward(state):
    # Linear velocity tracking with heading term  [eq. 4]
    r_lin = w_lin * min(dot(v_target, v_current), c_vel) / (c_vel + C)
    # Yaw-rate tracking  [eq. 5]
    r_ang = w_ang * exp(-abs(w_cmd - w_yaw))
    # Torque, torque-change and acceleration penalties  [eq. 6] to [eq. 8]
    r_energy = -(
        w_tau * sum(tau ** 2)
        + w_dtau * sum((tau - tau_prev) ** 2)
        + w_acc * sum(((dq - dq_prev) / dt) ** 2)
    )
    # Contact penalty: count of contacts above the force threshold  [eq. 9]
    r_collision = -w_col * count(norm(f_contact) > threshold)
    # Mean distance of base sample points to the centreline  [eq. 10]
    r_centerline = -w_cen * mean(distance_to_centreline)
    return r_lin + r_ang + r_energy + r_collision + r_centerline

Hyperparameters

Reward weights, as printed tab. II:

Results

Author-reported. Simulation (Table IV) uses 16 test runs per row. Real-world trials (Table V) use an unstated number of runs. The paper compares only stage-to-stage results. It reports no comparison to other methods.

Simulation success and average completion time tab. IV:

Real-world results tab. V. Pipe types A, B and C are the three real pipes. The number of trials is not stated.

The paper states that the real success rate is much lower than in simulation with the same policies, and it attributes the gap to pipe swaying, the slippery inner surface and noisy sensors §III-D. No simulation number for the real pipe sizes is given.

Headline numbers, as stated in the text: the policy crosses three real pipes (radii 0.242 m, 0.217 m and 0.192 m) without obstacles, at average speeds of 0.176, 0.150 and 0.111 m/s §III-D.

Paper vs code

No official code. The paper has no code link, no project page and no code availability statement, and the LaTeX source contains no URL to a repository. A GitHub repository search for "bidirectional scandots", "quadruped pipe inspection", "Weibang Bai quadruped" and "pipe crossing go2" returned no results. Source: LaTeX only (no PDF was fetched); no official code was used or cited.

Pitfalls

• Observation size. The proprioception vector is written as R^49 eq. 2, but its listed parts sum to 3 + 2 + 3 + 3 + 12 + 12 + 12 + 296 = 343 values. Either the 49 is a typo, or the privileged vector is not meant to be concatenated in full. The paper does not resolve this.
• Privileged encoder. The scan is said to pass through an encoder "such as a feedforward neural network" §II-B2. Its architecture, output size and training are not stated.
• Two-stage training. The caption of fig. 1 describes a second stage that distils the policy into one that takes a depth camera. The text does not describe the distillation loss, data, duration or student architecture. The depth normalisation is also not described beyond crop and clip §III-D.
• Upward scan. The paper says the ceiling distance is measured from the robot base to a virtual ceiling §II-B2. Sign convention, units and whether values are relative to the base or to the world are not stated.
• Reward with c_vel and C. The linear-velocity term divides by c_vel + C with C = 1e-5 §II-C1. The paper does not say what happens when the commanded speed is zero, or whether velocities are projected onto a plane.
• Undefined sums. The energy terms sum over i = 1 to n eq. 6 eq. 7 eq. 8. n is not defined: it could be time steps or joints. The text defines τ_i as the torque at the i-th time step eq. 6, which suggests time steps, but the paper does not say so.
• Contact penalty. The force threshold, and which bodies are penalised, are not stated eq. 9 §II-C3.
• Centreline sample count. The number of base points n in eq. 10 is not stated §II-C4 fig. 4.
• Curriculum. T is not given, so the threshold τ cannot be computed. The number of levels, the starting level and whether the level can drop below stage 1 are not stated. The text says the radius shrinks with level, but gives only per-stage ranges in tab. III.
• Stage 1 tested off its range. Table IV tests stage 1 on 0.2 m pipes, but stage 1's training radius range is [0.3, 0.5] m tab. III. This is an out-of-distribution test. The paper does not discuss this choice.
• Friction. Two friction settings appear: the reduced wall friction (0.2 and 0.1) and a randomised friction range [0.6, 2] §II-A4 §III-A. The text does not say which surface each applies to, or whether they are used together.
• Disturbance. The random external force has no stated magnitude, direction or frequency §II-A4.
• Equation numbers. The paper's numbers were counted from the LaTeX source, since no PDF was fetched. Ten numbered equations are counted: eq. 1 to eq. 10. Verify them against the compiled PDF before quoting them. Section numbers follow the IEEE Roman-numeral convention of the class file and should be checked the same way.
• Real-trial counts. The number of real trials is not stated, so the 46.5% traversal rate cannot be checked. The text does not explain how it was measured.
• Real and sim not matched. Table IV uses 0.2 m and 0.3 m pipes of 18 m length. Table V uses 0.192 m to 0.242 m pipes of 2 m and 3 m length. The two tables cannot be compared directly.
• Completion time. The start and end points for measuring completion time are not stated.
• Action interface. The paper does not state the control rate, action scaling, PD gains, or episode length.
• Compute. Training uses 6144 parallel robots on one PC and takes about 24 hours across the three stages §III-B. The paper gives no latency or on-board compute cost for the deployed policy.
• Abstract wording. The "privileged visual information" in the abstract is the height scan, not an image. The depth camera is only used at the deployment stage described in fig. 1.

Lineage

• Builds on: the simulator and terrain approach of Legged Gym and [rudin2022learning] (terrain, energy and collision terms) and the heading reward of [cheng2024extreme]. The base algorithm is PPO. Other works cited for context include [kumar2021rma], [hoeller2024anymal] and [zhuang2023robot] §I. None of these has a card in this wiki, so nothing from their content is used here.
• Compared against: no external method. The paper compares the three training stages with each other tab. IV.
• Superseded by: not stated. The paper proposes LiDAR as a future sensor §IV.