Zebrafish swimming model
Predicting a zebrafish’s swimming from its tail movements to drive the lab’s virtual-reality experiments.
- Role
- Lead locomotion modelling for three-week-old fish
- Team
- Lin Lab, University of TorontoProf. Qian Lin
- When
- Sep 2026 – now
- Outcome
- In evaluation for the lab's VR system
- Built with
- PyTorch, DeepLabCut, napari
Where the fish swamPredicted from its tail alone
The problem
In the lab’s VR setup, a fish is held in place while a projected scene moves according to how it would swim. A model predicts that motion from its tail. The existing model has a pronounced turning bias, and its errors accumulate over a swim.
What I built
I lead locomotion modelling for three-week-old zebrafish in the lab’s planned VR paper, owning the work from raw recordings to swimming predictions. I built the data and annotation pipeline for 1.82 million frames from 20 fish, then developed and compared the motion models.
More detail
The dataset is roughly twice the recording volume used by the existing model. I built the lab’s annotation tools and retrained its pose tracker, reducing validation error on unseen fish by 22%.
What improved
Up to 49% lower motion-prediction error in recorded-tail replay.
I changed training so the models had to use their own velocity predictions, matching the feedback they would get in VR. This cut replay error by 43% for the MLP and 49% for the GRU compared with training on measured velocities.
More detail
The replay comparison used four validation fish. A separate tail-only GRU averaged under 1°/s of turning bias on fish excluded from training; the clip shows one of those fish. The models are in offline evaluation for the lab’s VR system.