Description
This is a project of estimating 2D Keypoint of break dancer using fine-tuned VIT-Pose. Then the 2D keypoints is lifted to 3D and classiffied using MotionBERT (top-rock, footwork, powermove, and freeze). We put interpolation constraint to ensure robustness in real-world scenario.
My Contribution
- Utilizing YOLOv8 for detecting the human and use MIAP dataset to finetune it.
- Finetuning the MotionBERT and VITPose data using synthetic human from Omniverse and Character Creator.
- Creating a data of dance classification using Youtube video and fine tune Motion Bert classificcation head.
- Adding interpolation and optimization using TensorRT to meet the speed requirement
- Achieve near 100% accuracy in closed environment test set
Keywords
Research
Object Detection
2D Human Keypoint Detection
3D Human Skeleton
Tools
Pytorch
VITPose
MotionBERT
Omniverse
Character Creator 4