Autonomous Navigation of 4WIS4WID Agricultural Field Mobile Robot using Deep Reinforcement Learning

In plain words

Farm robots have to drive between rows of crops without squashing plants, and they often have to turn in tight spaces. This robot has a steering motor and a drive motor on each of its four wheels. That lets it drive sideways, spin on the spot, or turn like a car, depending on how the wheels are pointed.

The authors teach the robot with trial and error in a simulator. A camera looks at the crop rows and finds the line of each row. The learning agent gets that line, its own position, the goal and a list of waypoints, and it picks how fast to turn the robot. Each episode starts from a new random place and a slightly changed row layout, so the agent does not learn one fixed route. The paper compares four learning algorithms and finds one of them works best. All of the work is in simulation; the robot was not built or tested in a real field.

TL;DR

• Deep reinforcement learning drives a four-wheel-steering, four-wheel-drive robot along crop rows and between waypoints in a Gazebo field simulation §III-C, §IV.
• The agent picks only the turn rate; the controller picks symmetric, zero-turn or lateral steering from the waypoint direction §III-C, tab. I.
• Of PPO, SAC, TD3 and A2C, PPO gives the shortest episodes and the highest reward in the paper's runs §IV-B, fig. 9, fig. 10.
• Author-reported: 351 of 420 randomised trials reach the goal (83.57%) tab. IV.
• Simulation only, with no real-robot results §V.

Problem

Crop rows are regular, but the robot must avoid tender plants and handle outdoor conditions §I. Earlier work either uses a map, mostly targets skid-steered or differential-drive robots, or starts the robot in a field corner §II. The authors say that DRL for complex robot configurations and three-dimensional environments has not received much attention §II. The goal is one learned controller that tracks rows and moves between rows without a separate controller per steering mode §V.

Method

Section numbers follow IEEE style: Roman numerals for sections (§III) and letters for subsections (§III-A). Equation numbers are the printed numbers (1)–(16); the reward equations are not numbered.

Robot model §III-A, fig. 1, fig. 2. The robot is rigid and has eight joints: four revolute steering joints limited to ±90° and four continuous wheel joints §III-A. Pitch, roll and heave are ignored; the state is linear velocities vx, vy and angular velocity ω §III-A. The track t is the distance between the front wheels, the wheelbase l the distance between front and rear wheels, both equal in this model; R is the wheel radius. Wheels sit at (l/2, t/2) from the centre §III-A. A no-slip assumption removes vy for symmetric and zero-turn steering §III-A.

Wheel speeds for symmetric four-wheel steering eq. 1–4. Each wheel's linear speed is the signed magnitude of its velocity vector; the wheel angular speed is that speed divided by R §III-A. Wheels 1 and 4 use +ωt/2 inside the square root, wheels 2 and 3 use −ωt/2 eq. 1–4.

Steering angles eq. 5–8. Each steering angle is an arctangent of ωl over (2vx ± ωt). Angles θ1 and θ4 have equal magnitude and opposite sign, and so do θ2 and θ3 eq. 5–8. The paper says these satisfy the kinematic steering condition eq. 9, with the right-hand side equal to 2 because the track equals the wheelbase §III-A.

Zero-turn steering eq. 10–14. Setting vx = 0 gives equal wheel speeds of magnitude ω·√(l²+t²)/(2R) and steering angles of ±45° in this geometry eq. 10–14.

Lateral steering eq. 15–16. All four wheels steer to 90°, and all four wheel speeds are set to ±ω eq. 15–16, so the robot moves sideways.

# Wheel speeds and steering angles, symmetric 4WS  [eq. 1-8]
# Inputs: vx (m/s), omega (rad/s); t track, l wheelbase, R radius (m)
# Outputs: wheel_speeds (4,) rad/s; steer_angles (4,) rad

def symmetric_4ws(vx, omega, t, l, R):
    # Sign of the body speed, shared by all four wheels  [eq. 1-4]
    sign = signum(vx)
    # Half-track and half-base terms of the wheel velocity  [eq. 1-4]
    half_track = omega * t / 2
    half_base = omega * l / 2
    # Wheels 1 and 4 add half_track, wheels 2 and 3 subtract it  [eq. 1-4]
    speed_1 = sign * sqrt((vx + half_track)**2 + half_base**2)
    speed_2 = sign * sqrt((vx - half_track)**2 + half_base**2)
    speed_3 = speed_2
    speed_4 = speed_1
    # Angular speed is linear speed divided by wheel radius  [eq. 1-4]
    speeds = (speed_1, speed_2, speed_3, speed_4)
    wheel_speeds = [speed / R for speed in speeds]
    # Steering angles; the paper gives plain arctan, no quadrant rule  [eq. 5-8]
    angle_1 = atan(omega * l / (omega * t + 2 * vx))
    angle_2 = atan(omega * l / (-omega * t + 2 * vx))
    # Wheels 3 and 4 mirror wheels 2 and 1  [eq. 7-8]
    steer_angles = [angle_1, angle_2, -angle_2, -angle_1]
    return wheel_speeds, steer_angles

Crop row detection §III-B, fig. 3. Cameras are at the front and rear. The rear camera means the robot does not turn when it enters a new row §III-B. The pipeline crops the image, masks it, converts it to grey, applies a threshold and edge detection, and then a probabilistic Hough line transform §III-B, fig. 3(b). The green masking makes the method independent of crop type §III-B. The track error is the absolute difference between the x coordinate of the camera centre and the x coordinate of the detected line centre §III-B, fig. 3(c).

Training environment §III-C, fig. 4. At the start of each episode, the robot and the goal are placed at random. The robot starts between rows or in non-crop areas; the goal is inside a crop section §III-C. Crop rows get a small random change in orientation each episode, for domain randomisation §III-C. There are five rows with eight plants each, modelled as potato plants on a mud-like terrain §III-C.

Waypoints §III-C, fig. 3(d). Waypoints are placed along the shortest route. Lateral moves inside the crop are restricted, so the distance measure is the Manhattan distance to the nearest waypoint, not the Euclidean distance §III-C.

Observations §III-C. The agent observes the robot pose, the goal position, the track errors from both cameras, its previous actions, the nearest waypoint position and its Manhattan distance, and the joint commands §III-C. All observations are normalised before they reach the agent §III-C.

Actions §III-C, tab. I. The agent outputs one continuous value per step, which sets ω. The steering mode is chosen from the robot's orientation relative to the nearest waypoint: a horizontal orientation selects lateral steering, a vertical one selects symmetric steering, and a large orientation error selects zero-turn steering §III-C. The thresholds are not stated. In symmetric steering, vx is fixed at ±3.0 m/s, with the sign set by whether the waypoint is ahead or behind §III-C, tab. I. In zero-turn and lateral steering, vx = 0 and ω is scaled by 3 tab. I.

# Map the policy output to body velocities  [§III-C, tab. I]
# Inputs: a in [-1, 1] (policy output), mode, waypoint_ahead (bool)
# Outputs: vx (m/s), omega (rad/s)

def action_to_command(a, mode, waypoint_ahead):
    # Symmetric steering: fixed speed, sign follows the waypoint side;
    # omega is the unscaled policy output  [tab. I]
    if mode == "symmetric":
        vx = 3.0 if waypoint_ahead else -3.0
        omega = a
    # Zero-turn and lateral steering: vx is 0, omega is scaled by 3  [tab. I]
    else:
        vx = 0.0
        omega = 3.0 * a
    return vx, omega

Rewards §III-C. The per-step reward is the sum of six terms: distance to the nearest waypoint, camera alignment, waypoint bonus, goal term, control smoothness and time penalty §III-C. The distance term is minus the Manhattan distance to the nearest waypoint. The camera term gives +1 if either camera's track error is at most 40. The waypoint term gives 100 divided by the number of waypoints in the episode when the robot is within 0.3 m of the waypoint. The goal term gives +100 within 0.3 m of the goal and −100 on timeout or when the Manhattan distance reaches 1.5 m. The smoothness term is minus the absolute change in ω between steps. The time penalty is −1 per step §III-C.

# Per-step reward, multi-row setup  [§III-C, Rewards]
# Inputs: md_waypoint (m), dist_goal (m), err_cam1, err_cam2 (track errors)
#         omega_prev, omega (unscaled policy outputs), n_steps, n_wp
# Output: scalar reward

def step_reward(
    md_waypoint,
    dist_goal,
    err_cam1,
    err_cam2,
    omega_prev,
    omega,
    n_steps,
    n_wp,
):
    # Minus the Manhattan distance to the nearest waypoint  [§III-C]
    r_dist = -md_waypoint
    # +1 if either camera's track error is at most 40  [§III-C]
    r_cam = 1.0 if min(err_cam1, err_cam2) <= 40 else 0.0
    # Waypoint bonus, shared over the waypoints of the episode  [§III-C]
    r_wp = 100.0 / n_wp if md_waypoint <= 0.3 else 0.0
    # Goal term; the paper gives no zero branch, so 0 is assumed  [§III-C]
    if dist_goal <= 0.3:
        r_goal = 100.0
    elif n_steps >= 500 or md_waypoint >= 1.5:
        r_goal = -100.0
    else:
        r_goal = 0.0
    # Smoothness on unscaled omega, not the rescaled command  [§III-C]
    r_ctrl = -abs(omega_prev - omega)
    # Fixed penalty per step  [§III-C]
    r_time = -1.0
    return r_dist + r_cam + r_wp + r_goal + r_ctrl + r_time

Termination §III-C. An episode ends when the Manhattan distance to the nearest waypoint exceeds 1.5 m or the goal is within 0.3 m. It is truncated after 500 steps §III-C.

Learning algorithms and software §III-D, fig. 5, fig. 6. The simulator is Gazebo, the RL interface is OpenAI ROS, and the algorithms come from Stable Baselines3: SAC, TD3, A2C and PPO §III-D. Fig. 6 shows how the policy's action is applied to the simulated robot fig. 6.

Hyperparameters

PPO values come from the table in the paper. The environment constants come from the text and table I. Everything else is "not stated".

Results

Author-reported. Setup: simulated fields in Gazebo; no real-robot tests §IV, §V.

Other numbers the text states: the average PPO episode is below 100 steps, and the average reward stays above 180 over the 1-million-step run §IV-B. Crop row tracking with a single row is shown as a path plot and a time plot fig. 7, fig. 8. A trained model was tested qualitatively on slightly larger fields with basil and maize plants §IV-B, fig. 12; no numbers are given for those tests.

Paper vs code

No official code.

The LaTeX source contains no URL to code or a project page, and the paper names no repository. The searches were: a GitHub repository search for "4WIS4WID" (six results, none owned by the authors or their lab), a repository search for "crop row navigation reinforcement gazebo" (no results), a repository search for the author names (no results), and a user search for "Bishakh" (the lab account Bishakh has one unrelated repository).

Pitfalls

• Unspecified choices. The observation normalisation method and the steering-mode thresholds are not given. Implementers must choose their own values and document them.
• Signum at vx = 0. Equations 1–4 use signum(vx). With vx = 0, the sign is zero and every wheel speed is zero. Equation 10 drops the sign and gives a magnitude instead, so the paper does not say which sign to use for zero-turn wheel speeds eq. 1–4, eq. 10.
• Arctangent branch. Equations 5–8 use arctan without a stated quadrant rule. The denominator 2vx ± ωt can be zero for some ω and vx, and the paper does not say how to handle it eq. 5–8.
• Kinematic check. The paper states that the symmetric configuration satisfies the steering condition eq. 9, but the calculation is not shown; implementers should verify it numerically.
• Distance reward. The text describes the reward as movement toward the nearest waypoint, but the formula is minus the current Manhattan distance, an absolute quantity, not a change §III-C.
• Reward normalisation. The text says the rewards are normalised to −1 to +1 (and the smoothness term to −1 to 0), but no normalisation method is given. The formula constants (+100, −100, +1, −1) are used as written in the rewards section §III-C.
• Goal term has no zero case. The goal term lists +100, −100 and −100 cases but no default. This card assumes 0 otherwise. The paper does not say which case wins when two apply at once. The termination limit is "greater than 1.5 m", while the penalty applies at "1.5 m or more" §III-C.
• Single-row reward differs. The single-row experiment uses +1 when the camera error is less than 40, −1 per step, and +10 at the goal. It does not mention the distance, waypoint, smoothness or −100 terms §IV-A. Do not assume the multi-row reward was used there.
• Action scale versus smoothness. The zero-turn and lateral commands are scaled by 3 tab. I, but the smoothness term uses the unscaled ω. The paper says so on purpose, to keep motion smooth §III-C.
• Lateral steering at the joint limit. Lateral steering sets the wheels to 90°, which is the edge of the ±90° steering range §III-A, eq. 15.
• Camera and image parameters. The units of the track error are not stated. The value 120 is described as half the image width §IV-A, which implies pixels. The cropped image size, the Hough parameters and the threshold values are not given.
• Waypoint versus goal distances. Waypoints use the Manhattan distance; the goal distance is described as "distance", with no metric stated §III-C.
• Success definition. The paper reports 351 successful trials of 420 tab. IV but does not define success in one sentence. The termination rules §III-C suggest success is reaching the goal, but this is inferred.
• Manhattan distance. The paper reads the mean Manhattan distance (4.93 m) as a planned distance and the travelled distance (4.59 m) as a result. Its "conservative estimate" conclusion is the authors' own reading §IV-B.
• Single-run and reporting gaps. The number of random seeds is not stated. Whether the single-row figures are one run or an average over 500 is not stated tab. II. The start distance of 9 m is not tied to a metric §IV-A.
• Comparison with PD control. The comparison §IV-C, tab. V pits a learned controller against a PD controller on a different robot type. The PD gains, number of runs and the metric used for "distance" are not stated. The 0.68 m difference is the gap between the two reported distances, not a separate measurement tab. V.
• Algorithm comparison is qualitative. The statements about SAC, TD3 and A2C rest on the reward and episode-length plots fig. 9, fig. 10. No numbers are given for them.
• Generalisation claims. The tests on larger fields and on basil and maize fig. 12 are visual only. The paper calls the approach "generic", but no metrics support that claim.
• Simulation only. Sim-to-real transfer is listed as future work §V. Friction, sensor noise and actuator delay are not described.

Lineage

• Builds on: standard DRL algorithms as implemented in Stable Baselines3 §III-D; no earlier method from this group is named.
• Compared against: SAC, TD3, A2C within the paper §IV-B; a PD controller on a skid-steer robot §IV-C.
• Superseded by: not known from this wiki.