Autonomous Mail Delivery Robot

A self-driving robot that delivers mail to the correct mailbox number.

Autonomous Mail Delivery Robot

Overview & Objective

Modify the Donkey Car framework and physical car to autonomously deliver personalized mail to multiple different destinations based on number. Ideally, the Donkey Car model will be trained to do autonomous laps while hugging the right side of the road. Upon recognizing a mailbox number using OpenCV, the car will stop, queue the right package to be delivered, push the package off the side of the vehicle and continue on to the next delivery.

Must Haves:

  • Deliver personalized mail to or directly in front of all destinations. Personalized means that the address (in this case number) determines what mail is delivered.
  • Train the car to do laps in autonomous mode and then deliver mail when near a driveway by using its mechanical on board systems.

Nice to Haves:

  • User controls where the mail gets delivered. Assigns numbers/symbols to individual mail on top of the car.
  • Train the car to hug the right side of the road in autonomous mode.
  • Car can recognize and associate QR codes with correct package instead of numbers.

The Team

  • Andrew Ma, MAE
  • Bijan Ardalan, MAE
  • Lucas Hwang, ECE
Autonomous Mail Delivery Robot Team

Technologies Used

Key Features

  • Autonomous lane following using Donkey Car framework.
  • Visual recognition of numbered signs using OpenCV.
  • Targeted package delivery based on recognized number.
  • Mechanical package selection and ejection system using servos and rack/pinion.
  • User configuration for package-to-number mapping.
  • Ability to resume autonomous driving after delivery.

Mechanical Design

Acrylic Mounting Plate

Designed early on, this 1/4" acrylic plate allows standard size motors to be mounted in the center, feeding down through the middle gap of the baseplate. It includes various screw holes, zip tie slots, and larger slots for cable management, providing a stiff platform for the delivery system.

Base Plate Render Base Plate Drawing

Camera Mounting

Originally angled down 60 degrees for line tracking, the camera angle was decreased to 30 degrees to capture numbers on signs alongside the road. The camera is mounted 195mm above the base plate, with an optional ±10 degree extender piece for fine-tuning.

Final Camera Mount Exploded Camera Mount
Camera Mount Drawing

Jetson Case

A 3D printed case (Thingiverse link) was used to house the Jetson Nano.

Jetson Nano Case

Mail Delivery Mechanism

A double rack and pinion design with two linear sliders provides two degrees of freedom. One system selects the package, and the other pushes it off the car. Racks and pinions were designed in SolidWorks and laser-cut from acrylic. Standoffs accommodate different heights, and Velcro secures packages during transit.

Annotated Mail Delivery Mechanism

Choosing The Motors

Servo motors were chosen for precise package selection due to their internal control systems, avoiding the need for external sensors. Initial position servos had limited rotation, unsuitable for the rack and pinion. We switched to continuous rotation servos. Two servos are used, one for each rack and pinion.

Sign Design

Neon pink construction paper with black marker numbers proved optimal, minimizing glare compared to other materials like folders. The camera's blue tint made pink stand out best. 3D printed stands with cardboard backing allowed easy swapping and rotation.

Sign Glare Example Sign Glare Edged Example
Orange Sign Example Pink Sign Example

Electronic Design

Robocar Wiring Schematic

Wiring Schematic

Software Design

Software Overview

The software comprises two main parts: motor control (`myMotor.py`, based on `actuator.py`) and OpenCV number recognition. `myMotor.py` handles PWM board interfacing and motor speed control.

Servo Motors Control

A modified `actuator.py` (`myMotor.py`) was created and added as a DonkeyCar part. It uses the Adafruit PCA9685 library to set PWM signals. Pulse time constants were manually determined to control slider movement per package. Default motor speeds and PWM stop/clockwise/counter-clockwise values were configured in `myconfig.py`.

Neural Network Training

Standard Donkeycar framework was used for training (~12,000 records indoors due to weather). It was crucial to train with the signs present on the course to ensure the model learned to handle the visual change. Introducing signs after training could confuse the model.

Number Recognition Methodology

Seven-segment number recognition was used, involving several steps:

  1. Color Filtering: An RGB mask filters for the neon pink sign color. The image is cropped to the right half.
    Original Camera Input Color Filtered Mask
  2. Contour Recognition: OpenCV finds the contour of the pink paper (display contour). The image is cropped to this contour. Then, the digit contour (black marker) is found within the display contour, and the image is cropped again.
    Display Contour Digit Contour
  3. Seven Segment Recognition: The final digit image is split into 7 segments. If a segment is >50% filled with non-black pixels, it's considered 'on'. A dictionary lookup maps the 'on' segments to the recognized digit.
    Seven Segment Example 1 Seven Segment Example 2

Number Recognition Code Snippets

(Code snippets illustrating the process were included in the original document but are omitted here for brevity. Key steps involved dictionary lookups, RGB masking, contour finding via `cv2.findContours`, image cropping, and pixel analysis per segment.)

Code Snippet 1 Code Snippet 2 Code Snippet 3 Code Snippet 4 Code Snippet 5 Code Snippet 6 Code Snippet 7

Switching between Delivery and Driving

The OpenCV script was integrated into the `update_parts` method of Donkeycar. On each update cycle, an image is processed. If no number is recognized, driving continues. If a number is recognized (and it's different from the previously recognized one to prevent repeated stops), the car's throttle is set to zero, the motor delivery sequence runs, and then throttle control is returned to the autonomous driving model.

Code Snippet 8 - Integration Logic

Lessons Learned

Useful Knowledge

  • Native Camera Aspect Ratio (16:9): Using the native resolution prevents distortion, crucial for accurate number recognition, especially at the edges of the view.
  • Continuous vs. Position Servos: Understand the difference and choose based on design needs (angle control vs. speed control). Continuous rotation was needed for our rack/pinion.
  • Weight/Space Management: Plan component layout considering size and weight early. Overcrowding and excess weight caused sagging and wheel interference, requiring stronger springs and reorganization.

Challenges and Solutions

  • Random stops during outdoor training: Faulty power switch susceptible to vibrations was replaced.
  • SSH connection issues: Failed internal wifi adapter required reverting to a slower USB wifi dongle, impacting workflow.
  • Jetson Nano 3.3V line broken: Accidental short circuit damaged the 3.3V output. Switched to using the 5V output pin (within PWM board tolerance) to power the board.
  • Faulty Power Converter: Broken DC-DC converter supplied incorrect voltage (6V instead of 5V) to Jetson, preventing boot-up. Replacing the converter resolved the issue.

Final Prototype & Results

Results

All primary objectives were met: autonomous track circling and specific package delivery triggered by OpenCV number recognition. Continuous rotation servos were successfully interfaced and controlled via the Jetson Nano. The system correctly stopped at different sign locations and resumed driving. User specification of package-to-number mapping was also implemented.

The prototype successfully drove autonomously, recognized signs for '3' and '4', stopped, delivered the corresponding package, and resumed driving. This worked in both local angle and full autonomous modes.

Future Improvements/Ideas

Delivery Mechanism:

  • Implement control system with limit switches for more robust positioning than timed pulses.
  • Decrease weight of mechanical components.
  • Add driving routine for precise alignment with driveways/mailboxes.
  • Replace pusher with a pick-and-place arm for precision and gentler handling.
  • Use TOF sensors for better distance measurement and alignment.

Number Recognition:

  • Utilize Jetson GPU for a neural network-based recognition model (potentially faster and more robust).
  • Test and calibrate for different lighting conditions, possibly using multiple RGB masks.

References