Robotics engineering internship · Jul–Oct 2026
AstairoPets
Robot hardware and multi-model perception
AstairoPets is a pet companion robot that brings sensing, interaction and feeding into one platform. My work covered the robot’s electrical system architecture and the vision software that turns video into structured observations of the home, people and pets.
Technologies: RK3566, ESP32-S3, Python, PyTorch, YOLO, RTMPose, GRU, ONNX, RKNN
Hardware system and connections
I designed the system around an RK3566 host for perception and interaction, with an ESP32-S3 handling local sensing and actuation. The architecture covers wheel drive, encoders, pan-tilt servos, feeding, an IR-CUT camera, radar, odor sensing, audio and a display.
Four revisions of component selection and connection documentation translated that architecture into schematics, pin assignments and wiring tables. Separate 5 V system and actuator branches support the modules, while a regulated 12 V branch supplies the host. INA260 provides whole-system voltage and current monitoring.
The fourth revision added odor sensing without disturbing the audio interface. I reassigned the display reset pin to release UART9 and specified bidirectional 3.3 V/5 V level translation for the odor module.
A unified perception pipeline
The vision pipeline combines a 17-class household/pet-supply detector, 20-keypoint cat/dog pose, RTMPose human pose and causal pet tracking. Each branch owns a distinct output type, avoiding duplicate public detections of people, cats and dogs.
Raw predictions remain available for inspection. A separate tracked stream carries stable IDs, smoothed coordinates and explicit observed, carried or missing states. Downstream behavior models consume that consistent track history rather than unrelated frame-level predictions.
Behavior decisions across models
Static classification provides a quick estimate of a cat’s current state. RGB-GRU and Pose-GRU branches use aligned two-second windows to add motion history. The integrated temporal path uses fixed-average fusion; a learned quality gate remains an offline comparison.
Eating decisions also consider nearby dish evidence. Causal smoothing and hysteresis limit abrupt label changes. A confidence-based policy selects a primary behavior from static and temporal predictions while preserving both source scores and the selection reason.
Model selection and edge delivery
I compared six static classification architectures and selected MobileNet-V3-Large for its accuracy and compute tradeoff. On the 289-image development-test subset after exact-image duplicate removal, it reached 82.70% accuracy and 85.87% Macro-F1.
For RK3566 deployment, I prepared FP16 RKNN and ONNX assets and versioned Linux releases with dependency locks, integrity checks and rollback support. Server conversion acceptance on 104 real inputs produced the same seven-class top-1 prediction as ONNX for every input.
| Architecture | Parameters | Development accuracy |
|---|---|---|
| ResNet-18 | 11.18 M | 78.55% |
| EfficientNet-B0 | 4.02 M | 83.04% |
| MobileNet-V3-Large | 4.21 M | 82.70% |