Post-doctoral Scholar · University of Central Florida

Muhammad Shahbaz, Ph.D.

I build high-fidelity digital twins of real intersections so that 3D perception models — trained entirely in simulation — work the moment they're deployed on a real LiDAR. Closing the Sim2Real gap is my dissertation, my postdoc, and most of my GitHub.

Focus Sim2Real Learning 3D Computer Vision Digital Twins Roadside LiDAR Sensor Fusion
Stack PyTorch CARLA Unreal / Unity Open3D ROS C++ / Python
01

A little about me

I'm a researcher who fell in love with computer science through game dev as a teenager, kept the obsession through three degrees, and now spend my days at the intersection of simulation, perception, and the real world.

My Ph.D. (UCF, 2025) — From Failure to Fidelity: Enabling Scalable Sim2Real LiDAR Perception — argued that the gap between simulated and real LiDAR is mostly a data problem, not a model problem. The work has since produced an open-source framework, three public datasets, a JOSS paper, and a pending patent.

Outside the lab I build things for fun: a VR narrative game for Quest 3, a voice-to-keyboard dictation tool I actually use every day, a personal AI assistant with an Android frontend. Eight years of programming, four languages I'm fluent in, and one weakness for over-engineering side projects.

02

Research focus

One thesis, four threads. They all share a question: how do we get a perception model to behave the same way on real sensor data that it does in a simulator?

F1

High-Fidelity Digital Twins

Statistically aligned replicas of real intersections built in Unreal & Matlab RoadRunner. Geometry, materials, traffic flow, and sensor placement are all measured from the source — not guessed. The output is synthetic LiDAR that fools downstream detectors.

  • Provisional patent: Statistically Aligned LiDAR Digital Twin Pipeline
  • Three public synthetic datasets on Harvard Dataverse
F2

Sim2Real LiDAR Perception

Training 3D object detectors entirely in simulation and benchmarking them on real roadside LiDAR. In several configurations the sim-trained detector now beats models trained on real data — the upside of perfect labels.

  • Two T-ITS-class manuscripts under review
  • CenterPoint & LiDAR-camera fusion baselines
F3

Self-Supervised Roadside Detection

DINOSTAR trains object detectors on raw point clouds with no human labels — using statistically-modeled teachers built from clustering, background filtering, and bounding-box fitting. It matches human-annotated baselines.

  • arXiv:2501.17076 · submitted to ACM SIGSPATIAL 2025
F4

Open Tooling for the Field

LiGuard is a GUI-powered Python framework that lets researchers compose point-cloud pipelines without rewriting the framework. Published in JOSS, used by other labs, kept alive through three major versions.

  • JOSS 2025 · MIT licensed · on PyPI
  • pip install LiGuard
04

Featured work

The projects I'd actually want to talk to you about. Research first, then tools, then the side projects that betray my game-dev origins.

Research framework Python · Open3D · OpenCV

LiGuard

A GUI-driven Python framework for LiDAR (and paired image) data. Researchers build dynamic pipelines by toggling components in a config — no need to touch the framework's source. KITTI / OpenPCDet / SUSTechPoints readers built in; custom processes drop into a single algo/ directory.

  • JOSSpeer-reviewed
  • v2.x3 major revisions
  • PyPIpip install LiGuard
  • MITlicense
Dissertation core Unreal · CARLA · CenterPoint

UrbanTwin

High-fidelity synthetic replicas of three real roadside LiDAR datasets (LUMPI, V2X-Real, TUMTraf-I). Built by aligning geometry, traffic, and sensor models to the source recordings. Lets detectors train on perfect labels and still generalize. Published as a dataset paper in IEEE OJITS 2026.

  • 3public datasets
  • 1journal accepted
  • 2under review (T-ITS)
  • 1provisional patent
Self-supervised

DINOSTAR

Train 3D object detectors on roadside LiDAR with zero human labels. Statistical teachers — clustering + background filtering + box fitting — produce noisy supervision; a student detector trained on the union matches fully-supervised baselines.

Award · 2026

CVPR DriveX Challenge — 3rd Prize, Track 1

The Sim2Real LiDAR challenge at the 6th DriveX Workshop, IEEE/CVF CVPR 2026 (Denver). Submission built on the UrbanTwin pipeline. Co-author with Shaurya Agarwal.

Personal AI Kotlin · FastAPI · Gemini

CBr — Cloud Brain

A device-agnostic personal assistant. Android frontend (Jetpack Compose) talks to a FastAPI backend that orchestrates Gemini Flash, spawns workers (Thinking, Image-gen, Tool-Creator), and routes replies through a typed STM protocol with SDUI rendering.

Tool I use daily faster-whisper · CUDA

DictateTyper

A Windows tray app that turns my voice into keystrokes in any app. Live streaming transcription with LocalAgreement-2 commits — words appear while I'm still speaking, no flicker. GPU-accelerated, per-user settings, PyInstaller-packable.

Solo dev · VR Unity · Meta Quest 3

A Beautiful Story

A narrative VR game for Quest 3. Two protagonists, one mountain apartment, an AI-driven character with a bounded narrative and open-ended conversation. The fun side of "what happens when LLMs drive game NPCs."

Web tool React · FastAPI · WebSockets

LiGuard-Web (STRIDE)

A visual node-graph runtime for data pipelines — think ComfyUI for point clouds. React Flow canvas, streaming execution over WebSockets, parallel branch execution with proper error isolation, and a pluggable node registry.

More on GitHub — 39 public repos and counting.

05

Publications

Selected. Full list on Google Scholar.

Journal · Published

  1. 2026IEEE Open Journal of ITS

    UrbanTwin: Synthetic Roadside LiDAR Datasets

    M. Shahbaz, S. Agarwal · vol. 7, pp. 353–364

    DOI

  2. 2025Journal of Open Source Software

    LiGuard: A GUI-powered Python Framework for Processing Point-Cloud Data

    M. Shahbaz, S. Agarwal

    DOI Code

Journal · Under Review

  1. 2025IEEE T-IV · 2nd round

    LiGuard: A Streamlined Open-Source Framework for Rapid & Interactive Lidar Research

    M. Shahbaz, S. Agarwal

  2. 2025IEEE T-ITS

    Bridging Sim2Real Learning Gaps through High-Fidelity Digital Twins for LiDAR-based Perception Tasks

    M. Shahbaz, S. Agarwal

  3. 2025IEEE T-ITS

    SpatioStar: A Framework for Modeling High-Fidelity Digital Twins for Sim2Real LiDAR-based ITS

    M. Shahbaz, S. Agarwal

  4. 2025ACM SIGSPATIAL

    DINOSTAR: Deep Iterative Neural Object Detector Self-Supervised Training for Roadside LiDAR Applications

    M. Shahbaz, S. Agarwal

    arXiv:2501.17076

Conference

  1. 2022IBCAST · IEEE

    Autonomous Drone Swarm Navigation in Complex Environments

    S. Qamar, M. Qamar, M. Shahbaz, M. Arif Arshad, N. Saher Shah, A. Khan

    DOI 2 citations

  2. 2020RAEE & CS · IEEE

    Autonomous Navigation of Swarms in 3D Environments Using Deep Reinforcement Learning

    M. Shahbaz, A. Khan

    DOI Code 4 citations

Talks & Presentations

  1. 2024TRB Annual

    Translating LiDAR-based Object Detection from On-Road to Roadside: An Evaluatory Study

  2. 2023FL AV Summit

    Self-Supervising Object Detection for Roadside LiDAR Applications

  3. 2022FL AV Summit

    Using LiDAR for Real-Time Pedestrian Safety

06

Recognition & releases

Awards

  • 2026
    3rd Prize, Track 1 — CVPR 2026 DriveX Challenge IEEE/CVF CVPR 2026, Denver · Sim2Real LiDAR challenge
  • 2021
    ORCGS Doctoral Fellowship University of Central Florida
  • 2018
    IT Endowment Scholarship PIEAS, Islamabad
  • 2018
    1st Position, Final Year Project Competition PMAS AAUR — Augmented Reality Carrom Board
  • 2016
    1st Regional, Microsoft Imagine Cup Pakistan

Datasets

Patent

  • Statistically Aligned LiDAR Digital Twin Pipeline Provisional, UCF Ref. 2085.237.PR · 2026
07

Experience & education

  1. 2025 — now

    Post-doctoral Scholar

    University of Central Florida · Dept. of CECE · Urbanity Lab

    Scaling Sim2Real methodologies for LiDAR perception. Two T-ITS-class manuscripts under review, one provisional patent, a CVPR challenge prize, and a fourth-version push of LiGuard.

  2. 2022 — 2025

    Graduate Research Assistant

    UCF · CECE · FDOT-funded projects

    Led PULSE (Pedestrian Upfront LiDAR-based Safety on Edge), worked on the Smart Work Zone project. 3D perception with LiDAR–camera fusion, sensor rigs (Ouster, Hesai, Flir), and the data infrastructure to feed them.

  3. 2021 — 2025

    Ph.D., Civil Engineering (Intelligent Transportation Systems)

    University of Central Florida · CGPA 3.71 / 4.0

    Dissertation: From Failure to Fidelity: Enabling Scalable Sim2Real LiDAR Perception. ORCGS Doctoral Fellowship.

  4. 2018 — 2020

    M.S., Computer Science

    Pakistan Institute of Engineering & Applied Sciences (PIEAS) · CGPA 3.78 / 4.0

    QS-ranked #1 university in Pakistan. Thesis on deep reinforcement learning for autonomous drone swarms; first-author IEEE paper at RAEE&CS.

  5. 2018 — 2019

    Freelance Software Developer

    Fiverr · Level 1 Seller

    Led a smart-home build: lighting, thermostats, switches, curtains, a smart doorbell with dual audio/video. Raspberry Pi + Arduino Nano + cross-platform control software.

  6. 2016 — 2018

    Indie Game & App Developer

    Personal · Google Play

    Shipped Moneypad (personal finance), Blow (survival game), and 3D Banta Balls (online multiplayer take on the classic Pakistani street game). 1st in Regional Microsoft Imagine Cup, 2016.

  7. 2014 — 2018

    B.S., Computer Science

    PMAS Arid Agriculture University, Rawalpindi · CGPA 3.50 / 4.0

    Senior design project: Augmented Reality Carrom Board with a custom conductive-thread glove controller. 1st place, 2018 Open House competition.

08 · Get in touch

Hiring for 3D perception, autonomy, or simulation?
I'd like to hear about it.

I'm open to research scientist, applied scientist, and senior engineering roles in autonomous driving, robotics, and spatial AI. Postdoc collaborations welcome too.

Based in Orlando, Florida · open to relocation