3D Human Pose Estimation

Datasets

Human3.6M 数据集的下载与 Human3.6M pkl 文件缺失的处理方法 - 知乎

MMPose

Overview — MMPose 1.3.2 documentation

[Overview — MMPose 1.3.2 documentation](https://mmpose.readthedocs.io/en/latest/model_zoo.html#body-3d-keypoint)

  • [BACKBONE] 3d Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (Video Pose Lift + Videopose3d on H36m ⇨)

  • [BACKBONE] A Simple Yet Effective Baseline for 3d Human Pose Estimation (Image Pose Lift + Simplebaseline3d on H36m ⇨)

  • [BACKBONE] Learning Human Motion Representations: A Unified Perspective (Motionbert + Motionbert on H36m ⇨)

  • [DATASET] Human3.6m: Large Scale Datasets and Predictive Methods for 3d Human Sensing in Natural Environments (Video Pose Lift + Videopose3d on H36m ⇨, Image Pose Lift + Simplebaseline3d on H36m ⇨, Motionbert + Motionbert on H36m ⇨)

Method

VideoPose3D(CVPR 2019)

3d-pose-baseline (ICCV 2017)

MotionBERT (ICCV 2023)

UGCN (ECCV 2020)

Dev Env

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
(openmmlab-mmpose) (base) houjinliang@3080server:~/MyCVProject/bDetection/mmpose$ python --version
Python 3.8.19

(openmmlab-mmpose) (base) houjinliang@3080server:~/MyCVProject/bDetection/mmpose$ pip list
Package                Version      Editable project location
---------------------- ------------ ----------------------------------------------
addict                 2.4.0
aliyun-python-sdk-core 2.15.1
aliyun-python-sdk-kms  2.16.3
asttokens              2.0.5
attrs                  23.2.0
backcall               0.2.0
cachetools             5.3.3
certifi                2024.7.4
cffi                   1.16.0
charset-normalizer     3.3.2
chumpy                 0.70
click                  8.1.7
colorama               0.4.6
comm                   0.2.1
contourpy              1.1.1
coverage               7.5.4
crcmod                 1.7
cryptography           42.0.8
cycler                 0.12.1
Cython                 3.0.10
debugpy                1.6.7
decorator              5.1.1
exceptiongroup         1.2.1
executing              0.8.3
filelock               3.14.0
flake8                 7.1.0
fonttools              4.53.1
idna                   3.7
importlib-metadata     7.0.1
importlib_resources    6.4.0
iniconfig              2.0.0
interrogate            1.7.0
ipykernel              6.28.0
ipython                8.12.2
isort                  4.3.21
jedi                   0.18.1
jmespath               0.10.0
json-tricks            3.17.3
jupyter_client         8.6.0
jupyter_core           5.7.2
kiwisolver             1.4.5
Markdown               3.6
markdown-it-py         3.0.0
matplotlib             3.7.5
matplotlib-inline      0.1.6
mccabe                 0.7.0
mdurl                  0.1.2
✅mmcv                   2.1.0
✅mmdet                  3.3.0
✅mmengine               0.10.4
✅mmpose                 1.3.1        /mnt/houjinliang/MyCVProject/bDetection/mmpose
model-index            0.1.11
munkres                1.1.4
nest-asyncio           1.6.0
numpy                  1.24.4
nvidia-ml-py           12.535.161
nvitop                 1.3.2
opencv-python          4.10.0.84
opendatalab            0.0.10
openmim                0.3.9
openxlab               0.1.1
ordered-set            4.1.0
oss2                   2.17.0
packaging              24.1
pandas                 2.0.3
parameterized          0.9.0
parso                  0.8.3
pexpect                4.8.0
pickleshare            0.7.5
pillow                 10.4.0
pip                    24.0
platformdirs           3.10.0
pluggy                 1.5.0
prompt-toolkit         3.0.43
psutil                 5.9.0
ptyprocess             0.7.0
pure-eval              0.2.2
py                     1.11.0
pycocotools            2.0.7
pycodestyle            2.12.0
pycparser              2.22
pycryptodome           3.20.0
pyflakes               3.2.0
Pygments               2.15.1
pyparsing              3.1.2
pytest                 8.2.2
pytest-runner          6.0.1
python-dateutil        2.9.0.post0
pytz                   2023.4
PyYAML                 6.0.1
pyzmq                  25.1.2
requests               2.28.2
rich                   13.4.2
scipy                  1.10.1
setuptools             60.2.0
shapely                2.0.4
six                    1.16.0
stack-data             0.2.0
tabulate               0.9.0
termcolor              2.4.0
terminaltables         3.1.10
tomli                  2.0.1
✅torch                  1.10.1+cu113
✅torchvision            0.11.2+cu113
tornado                6.4.1
tqdm                   4.65.2
traitlets              5.14.3
typing_extensions      4.11.0
tzdata                 2024.1
urllib3                1.26.19
wcwidth                0.2.5
wheel                  0.43.0
xdoctest               1.1.5
xtcocotools            1.14.3
yapf                   0.40.2
zipp                   3.17.0

Demo

Animal Pose Estimation

2D 动物图片姿态识别推理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_8xb32-300e_coco.py \
    https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_m_8xb32-300e_coco/rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --output-root  vis_results/AnimalPoseEstimation  \
     --draw-heatmap --det-cat-id=15
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_8xb32-300e_coco.py \
    https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_m_8xb32-300e_coco/rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --output-root vis_results/AnimalPoseEstimation  \
    --save-predictions 
    --draw-heatmap --det-cat-id=15

2D 动物视频姿态识别推理

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_8xb32-300e_coco.py \
    https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_m_8xb32-300e_coco/rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input demo/resources/demo_dog.mp4 \
    --output-root vis_results/AnimalPoseEstimation \
    --draw-heatmap --det-cat-id=16

使用 Inferencer 进行 2D 动物姿态识别推理

1
2
python demo/inferencer_demo.py tests/data/ap10k \
    --pose2d animal --vis-out-dir vis_results/ap10k

Face Keypoint Estimation

2D 脸部图片关键点识别推理

1
2
3
4
5
6
7
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/yolox-s_8xb8-300e_coco-face.py \
    https://download.openmmlab.com/mmpose/mmdet_pretrained/yolo-x_8xb8-300e_coco-face_13274d7c.pth \
    configs/face_2d_keypoint/rtmpose/face6/rtmpose-m_8xb256-120e_face6-256x256.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-face6_pt-in1k_120e-256x256-72a37400_20230529.pth \
    --input tests/data/cofw/001766.jpg \
    --draw-heatmap --output-root vis_results/FaceKeypointEstimation

2D 脸部视频关键点识别推理

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/yolox-s_8xb8-300e_coco-face.py \
    https://download.openmmlab.com/mmpose/mmdet_pretrained/yolo-x_8xb8-300e_coco-face_13274d7c.pth \
    configs/face_2d_keypoint/rtmpose/face6/rtmpose-m_8xb256-120e_face6-256x256.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-face6_pt-in1k_120e-256x256-72a37400_20230529.pth \
    --input demo/resources/demo_face.mp4 \
    --output-root vis_results/FaceKeypointEstimation \
    --radius 1

使用 Inferencer 进行 2D 脸部关键点识别推理

1
2
python demo/inferencer_demo.py tests/data/wflw \
    --pose2d face --vis-out-dir vis_results/wflw --radius 1

Hand Keypoint Estimation

2D 手部图片关键点识别

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_nano_320-8xb32_hand.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmdet_nano_8xb32-300e_hand-267f9c8f.pth \
    configs/hand_2d_keypoint/rtmpose/hand5/rtmpose-m_8xb256-210e_hand5-256x256.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-hand5_pt-aic-coco_210e-256x256-74fb594_20230320.pth \
    --input tests/data/onehand10k/9.jpg \
    --output-root vis_results/HandKeypointEstimation \
    --draw-heatmap

2D 手部视频关键点识别推理

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_nano_320-8xb32_hand.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmdet_nano_8xb32-300e_hand-267f9c8f.pth \
    configs/hand_2d_keypoint/rtmpose/hand5/rtmpose-m_8xb256-210e_hand5-256x256.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-hand5_pt-aic-coco_210e-256x256-74fb594_20230320.pth \
    --input demo/resources/tests_data_nvgesture_sk_color.avi \
    --output-root vis_results/HandKeypointEstimation \
    --kpt-thr 0.1

使用 Inferencer 进行 2D 手部关键点识别推理

1
2
3
python demo/inferencer_demo.py tests/data/onehand10k \
    --pose2d hand --vis-out-dir vis_results/onehand10k \
    --bbox-thr 0.5 --kpt-thr 0.05

Human Pose Estimation

2D 人体姿态 Top-Down 图片检测

使用整张图片作为输入进行检测

1
2
3
4
5
6
python demo/image_demo.py \
    tests/data/coco/000000000785.jpg \
    configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py \
    https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth \
    --out-file vis_results/HumanPoseEstimation/2d_human_whole_pic.jpg \
    --draw-heatmap

使用 MMDet 做人体 bounding box 检测

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
    configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-420e_body8-256x192.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth \
    --input tests/data/coco/000000197388.jpg \
    --draw-heatmap \
    --output-root vis_results/HumanPoseEstimation \

2D 人体姿态 Top-Down 视频检测

我们的脚本同样支持视频作为输入，由 MMDet 完成人体检测后 MMPose 完成 Top-Down 的姿态预估

1
2
3
4
5
6
7
8
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
    configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-420e_body8-256x192.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth \
    --input tests/data/posetrack18/videos/000001_mpiinew_test/000001_mpiinew_test.mp4 \
    --output-root=vis_results/2D_human_Top-Down \
    --draw-heatmap

2D 人体姿态 Bottom-Up 图片和视频识别检测

1
2
3
4
5
6
python demo/bottomup_demo.py \
    configs/body_2d_keypoint/dekr/coco/dekr_hrnet-w32_8xb10-140e_coco-512x512.py \
    https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/dekr/coco/dekr_hrnet-w32_8xb10-140e_coco-512x512_ac7c17bf-20221228.pth \
    --input tests/data/coco/000000197388.jpg \
    --output-root=vis_results/2D_Bottom-Up \
    --save-predictions

使用 Inferencer 进行 2D 人体姿态识别检测

1
2
3
python demo/inferencer_demo.py \
    tests/data/posetrack18/videos/000001_mpiinew_test/000001_mpiinew_test.mp4 \
    --pose2d human --vis-out-dir vis_results/posetrack18

Human Whole-Body Pose Estimation

2D 人体全身姿态 Top-Down 图片识别

使用整张图片作为输入进行检测

此时输入的整张图片会被当作 bounding box 使用。

1
2
3
4
5
python demo/image_demo.py \
    tests/data/coco/000000000785.jpg \
    configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_vipnas-res50_dark-8xb64-210e_coco-wholebody-256x192.py \
    https://download.openmmlab.com/mmpose/top_down/vipnas/vipnas_res50_wholebody_256x192_dark-67c0ce35_20211112.pth \
    --out-file vis_results/HumanWhole-BodyPoseEstimation/whole_pic.jpg

使用 MMDet 进行人体 bounding box 检测

1
2
3
4
5
6
7
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
    configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_hrnet-w48_dark-8xb32-210e_coco-wholebody-384x288.py \
    https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
    --input tests/data/coco/000000196141.jpg \
    --output-root vis_results/HumanWhole-BodyPoseEstimation

2D 人体全身姿态 Top-Down 视频识别检测

1
2
3
4
5
6
7
python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
    https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
    configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_hrnet-w48_dark-8xb32-210e_coco-wholebody-384x288.py \
    https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth \
    --input https://user-images.githubusercontent.com/87690686/137440639-fb08603d-9a35-474e-b65f-46b5c06b68d6.mp4 \
    --output-root vis_results/2D_whole_body_Top-Down/

使用 Inferencer 进行 2D 人体全身姿态识别

1
2
python demo/inferencer_demo.py tests/data/crowdpose \
    --pose2d wholebody --vis-out-dir vis_results/crowdpose

Try - 3D Human Pose Demo

rtmdet, rtmpose, video_pose_lift

1
2
3
4
5
6
7
8
9
10
python demo/body3d_pose_lifter_demo.py \
demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-420e_body8-256x192.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth \
configs/body_3d_keypoint/video_pose_lift/h36m/video-pose-lift_tcn-243frm-supv-cpn-ft_8xb128-200e_h36m.py \
https://download.openmmlab.com/mmpose/body3d/videopose/videopose_h36m_243frames_fullconv_supervised_cpn_ft-88f5abbb_20210527.pth \
--input demo/resources/kk.mp4 \
--output-root  vis_results/3DHumanPoseDemo  \
--online

rtmdet, rtmpose, motionbert

1
2
3
4
5
6
7
8
9
10
python demo/body3d_pose_lifter_demo.py \
demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth \
configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-420e_body8-256x192.py \
https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth \
configs/body_3d_keypoint/motionbert/h36m/motionbert_dstformer-ft-243frm_8xb32-120e_h36m.py \
https://download.openmmlab.com/mmpose/v1/body_3d_keypoint/pose_lift/h36m/motionbert_ft_h36m-d80af323_20230531.pth \
--input demo/resources/kk.mp4 \
--output-root  vis_results/3DHumanPoseDemo \
--online

Interferece

1
2
python demo/inferencer_demo.py tests/data/coco/000000000785.jpg \
    --pose3d human3d --vis-out-dir vis_results/3DHumanPoseDemo
CV
#Python #DeepLearning #PyTorch #OpenMMLab #MMPose
3D Human Pose Estimation
https://blog.cosmicdusty.cc/post/CV/3DHumanPoseEstimation/
作者
Murphy
发布于
2024年7月25日
许可协议