Brian Zheng

Projects

A selection of my recent work and engineering projects

End-to-End Autonomous Drone

Autonomous drone delivery system with real-time object detection and AI-driven flight control.

C++ Python ROS2 TensorFlow YOLOv4 AirSim PX4 ArduPilot

Problem: Create an autonomous delivery system capable of navigating complex campus environments and accurately delivering payloads.

Approach: Developed custom object detection models using YOLOv4-tiny and integrated them into a real-time drone control pipeline via ROS2 and MAVSDK.

Result: Successfully simulated and deployed flight missions in AirSim and PX4 environments with end-to-end autonomy from navigation to payload release with onboard AI inference.

Real-Time Expression Detector

Personalized facial expression recognition system using deep learning and computer vision.

PyTorch MediaPipe GStreamer OpenGL GRU

Problem: Build a real-time, personalized facial expression recognition system that runs efficiently on standard hardware.

Approach: Built a real-time inference pipeline using MediaPipe Face Mesh for landmarks and a lightweight PyTorch GRU classifier (hidden-64) with class weighting, label smoothing, and gradient clipping.

Result: Achieved 30+ FPS with webcam → landmarks → GRU → EMA smoothing pipeline, with validation macro-F1 reporting and confusion matrix analysis.

6-DOF CV Robotic Arm

Computer vision-controlled robotic arm with gesture tracking and inverse kinematics.

C++ Python MediaPipe OpenCV SOLIDWORKS

Problem: Create a robotic arm controlled by hand gestures and color tracking for intuitive human-robot interaction.

Approach: Implemented MediaPipe and OpenCV's HSV color thresholding to detect and track gestures, extracting positions using image moments. Designed arm using SOLIDWORKS with PID control and Inverse Kinematics logic.

Result: Successfully controlled a 6-DOF robotic arm and gripper through real-time gesture and color tracking.

ASCII Cube Matrix

Interactive 3D ASCII cube visualization with drag-to-rotate controls and orbital motion.

JavaScript Canvas API 3D Graphics

Problem: Create an aesthetically pleasing ASCII-based 3D visualization inspired by classic terminal graphics.

Approach: Implemented z-buffering and 3D projection algorithms to render rotating cubes using ASCII characters, with smooth drag controls and orbital motion.

Result: A fluid, interactive visualization with auto-rotation, manual control, and 3D orbital mechanics.

Fourier Series Visualization

Interactive Fourier epicycles animation with shape morphing and parametric curve generation.

JavaScript Canvas API DFT Animations

Problem: Visualize Fourier series decomposition in an intuitive and interactive way.

Approach: Developed DFT algorithm to decompose parametric curves into epicycles, with smooth transitions between different shapes.

Result: An engaging visualization showing how complex curves can be constructed from rotating circles.

Experience & Achievements

My professional journey and accomplishments

Autonomy Software Developer

Sept 2025 - Present

enVgo • Waterloo, ON

Optimized and increased the accuracy of a Segmentation Model by 19% through improving dataset accuracy and streamlining scalable data generation, while also improving inference speeds using TensorRT and CUDA on Jetson architectures
Created model-agnostic scripts in dockerized containers to evaluate metrics of segmentation models by computing mIOU, precision, and F1 scores
Developed real-time birdseye view through homography transformations using GStreamer/OpenGL to stitch 4 fisheye camera feeds, and GLSL shaders with per-camera undistortion maps
Implemented Structure-from-Motion (SFM) model using synchronized camera feeds to estimate vessel pose and reconstruct the 3D dock environment, allowing localization and motion planning for autonomous docking
Developed radar CAN/ROS drivers, and applied RL and EKFs to filter noise and extract tracked point clouds

Director of Eve Autonomy and Perception Software

May 2025 - Present

WATonomous • University of Waterloo

Leading a team of 40+ members in the research and development of an autonomous Kia EV, incorporating AI/machine learning, sensor fusion, and complex path planning algorithms
Developing ROS2 wrappers for LiDAR ground segmentation and CV multi-object tracking algorithms like ByteTrack
Developed a multi-modal 3D object detection and tracking pipeline by fusing LiDAR and camera data, RANSAC Model Floor Segmentation, and class-parameterized DBSCAN clustering of batched YOLOv8 detections
Enhanced object detection pipeline performance by optimizing the YOLOv8 Model using TensorRT and CUDA GPU acceleration, maximizing hardware resources and inference speeds
Implemented ROS2 Drivers and launch files for LiDAR and Cameras and calibrated camera intrinsics
Implemented a hybrid A* and BFS search algorithm for local path planning and a Pure Pursuit and PID controller for path tracking

Robotic Systems Engineer

Jan 2025 - May 2025

Developed a multi-modal 3D object detection and tracking pipeline by fusing LiDAR and camera data, RANSAC Model Floor Segmentation, and class-parameterized DBSCAN clustering of batched YOLOv8 detections
Enhanced object detection pipeline performance by optimizing the YOLOv8 Model using TensorRT and CUDA GPU acceleration, maximizing hardware resources and inference speeds
Implemented ROS2 Drivers and launch files for LiDAR and Cameras and calibrated camera intrinsics
Implemented a hybrid A* and BFS search algorithm for local path planning and a Pure Pursuit and PID controller for path tracking

Education & Certifications

Bachelor of Applied Science (BASc)

Mechatronics, Robotics, and Automation Engineering

University of Waterloo

Aug 2024 - May 2029

Certified SOLIDWORKS Associate (CSWA)

Issued by Dassault Systèmes

View Certificate

Jan 2025

Skills & Tech Stack

Technologies and tools I work with

Programming Languages

C/C++

Python

Java

JavaScript

HTML/CSS

AI/ML & Computer Vision

PyTorch

TensorRT

OpenCV

CUDA

MediaPipe

Robotics & Autonomy

ROS2

GStreamer

OpenGL

WebRTC

ArduPilot/PX4

Tools & Platforms

Docker

Git

SOLIDWORKS

AWS EC2

Linux

CVAT

About Me

I'm a Mechatronics Engineering student at the University of Waterloo, specializing in autonomy software development and perception systems. Currently, I'm working as an Autonomy Software Developer at enVgo, where I optimize AI models and develop real-time computer vision pipelines for autonomous vessels.

At WATonomous, I lead a team of 40+ members developing an autonomous Kia EV, focusing on sensor fusion, 3D object detection, and path planning. My work spans the full autonomy stack—from low-level sensor drivers to high-level AI inference and control algorithms.

I'm passionate about building intelligent systems that bridge the gap between perception, planning, and control. Whether it's optimizing YOLOv8 models with TensorRT, implementing Structure-from-Motion for 3D reconstruction, or developing real-time multi-camera stitching with GStreamer, I love tackling challenging problems in robotics and computer vision.

Blog & Writing

Thoughts on projects, engineering, and technology

January 2025

Building a Morse Code Robot with LEGO EV3

A deep dive into creating an autonomous robot that communicates using Morse code, covering hardware design, sensor integration, and algorithmic implementation.

5 min read

December 2024

Improving LiDAR–Camera Fusion in ROS 2

Exploring techniques for better sensor fusion between LiDAR and camera data in ROS 2, including calibration methods and real-time processing optimizations.

8 min read

Get In Touch

Let's connect! I'm always open to discussing new projects, opportunities, or just having a conversation.

Autonomy Software Developer & Mechatronics Engineer