RK人工智能算法最全官方示例

peter

RKNN Model Zoo基于 RKNPU SDK 工具链开发, 提供了目前主流算法的部署例程. 例程包含导出RKNN模型, 使用 Python API, CAPI 推理 RKNN 模型的流程.

• 支持 RK3562, RK3566, RK3568, RK3576, RK3588, RV1126B 平台。
• 部分支持RV1103, RV1106
• 支持 RV1109, RV1126, RK1808 平台。
  

源文地址：https://github.com/airockchip/rknn_model_zoo/tree/main

以下demo除了从对应的仓库导出模型, 也可从网盘 https://console.zbox.filez.com/l/8ufwtG (提取码: rknn) 下载模型文件.

Category	Name	Dtype	Model Download Link	Support platform
图像分类	mobilenet	FP16/INT8	mobilenetv2-12.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RV1103|RV1106 RK1808|RK3399PRO RV1109|RV1126
图像分类	resnet	FP16/INT8	resnet50-v2-7.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov5	FP16/INT8	./yolov5s_relu.onnx ./yolov5n.onnx ./yolov5s.onnx ./yolov5m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RV1103|RV1106 RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov6	FP16/INT8	./yolov6n.onnx ./yolov6s.onnx ./yolov6m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov7	FP16/INT8	./yolov7-tiny.onnx ./yolov7.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov8	FP16/INT8	./yolov8n.onnx ./yolov8s.onnx ./yolov8m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov8_obb	INT8	./yolov8n-obb.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolov10	FP16/INT8	./yolov10n.onnx ./yolov10s.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RV1103|RV1106 RK1808|RK3399PRO RV1109|RV1126
物体检测	yolo11	FP16/INT8	./yolo11n.onnx ./yolo11s.onnx ./yolo11m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RV1103|RV1106 RK1808|RK3399PRO RV1109|RV1126
物体检测	yolox	FP16/INT8	./yolox_s.onnx ./yolox_m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	ppyoloe	FP16/INT8	./ppyoloe_s.onnx ./ppyoloe_m.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
物体检测	yolo_world	FP16/INT8	./yolo_world_v2s.onnx ./clip_text.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
人体关键点	yolov8_pose	INT8	./yolov8n-pose.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
图像分割	deeplabv3	FP16/INT8	./deeplab-v3-plus-mobilenet-v2.pb	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
图像分割	yolov5_seg	FP16/INT8	./yolov5n-seg.onnx ./yolov5s-seg.onnx ./yolov5m-seg.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
图像分割	yolov8_seg	FP16/INT8	./yolov8n-seg.onnx ./yolov8s-seg.onnx ./yolov8m-seg.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
图像分割	ppseg	FP16/INT8	pp_liteseg_cityscapes.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
图像分割	mobilesam	FP16	mobilesam_encoder_tiny.onnx mobilesam_decoder.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
人脸关键点	RetinaFace	INT8	RetinaFace_mobile320.onnx RetinaFace_resnet50_320.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
车牌识别	LPRNet	FP16/INT8	./lprnet.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RV1103|RV1106 RK1808|RK3399PRO RV1109|RV1126
文字检测	PPOCR-Det	FP16/INT8	../ppocrv4_det.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
文字识别	PPOCR-Rec	FP16	../ppocrv4_rec.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
自然语言翻译	lite_transformer	FP16	lite-transformer-encoder-16.onnx lite-transformer-decoder-16.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B RK1808|RK3399PRO RV1109|RV1126
图文匹配	clip	FP16	./clip_images.onnx ./clip_text.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
语音识别	wav2vec2	FP16	wav2vec2_base_960h_20s.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
语音识别	whisper	FP16	whisper_encoder_base_20s.onnx whisper_decoder_base_20s.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
语音识别	zipformer	FP16	encoder-epoch-99-avg-1.onnx decoder-epoch-99-avg-1.onnx joiner-epoch-99-avg-1.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
语音分类	yamnet	FP16	yamnet_3s.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B
文字转语音	mms_tts	FP16	mms_tts_eng_encoder_200.onnx mms_tts_eng_decoder_200.onnx	RK3562|RK3566|RK3568|RK3576|RK3588|RV1126B

Model performance benchmark(FPS)

demo	model_name	inputs_shape	dtype	RK3566 RK3568	RK3562	RK3588 @single_core	RK3576 @single_core	RV1109	RV1126	RK1808
mobilenet	mobilenetv2-12	[1, 3, 224, 224]	INT8	180.7	281.3	450.7	467.0	212.9	322.3	170.3
resnet	resnet50-v2-7	[1, 3, 224, 224]	INT8	37.9	54.9	110.1	99.0	24.4	36.2	37.1
yolov5	yolov5s_relu	[1, 3, 640, 640]	INT8	25.5	33.2	66.1	65.0	20.2	29.2	37.2
	yolov5n	[1, 3, 640, 640]	INT8	39.7	47.4	82.5	112.7	36.3	53.2	61.2
	yolov5s	[1, 3, 640, 640]	INT8	19.3	23.6	48.4	57.5	13.6	20.0	28.2
	yolov5m	[1, 3, 640, 640]	INT8	8.6	10.8	20.9	23.7	5.8	8.5	13.3
yolov6	yolov6n	[1, 3, 640, 640]	INT8	48.8	56.4	106.4	109.1	37.8	56.8	66.8
	yolov6s	[1, 3, 640, 640]	INT8	15.2	17.3	36.4	35.0	10.8	16.3	24.1
	yolov6m	[1, 3, 640, 640]	INT8	7.2	8.6	17.8	17.4	5.6	8.3	11.5
yolov7	yolov7-tiny	[1, 3, 640, 640]	INT8	27.9	36.5	72.7	74.8	15.4	22.4	37.2
	yolov7	[1, 3, 640, 640]	INT8	4.6	5.9	11.4	13.0	3.3	4.8	7.4
yolov8	yolov8n	[1, 3, 640, 640]	INT8	34.0	40.9	73.5	90.2	24.0	35.4	42.3
	yolov8s	[1, 3, 640, 640]	INT8	15.1	18.4	38.0	40.8	8.9	13.1	19.1
	yolov8m	[1, 3, 640, 640]	INT8	6.5	8.2	16.2	16.7	3.9	5.8	9.1
yolov8_obb	yolov8n-obb	[1, 3, 640, 640]	INT8	33.9	41.3	74.0	90.2	25.1	37.3	42.8
yolov10	yolov10n	[1, 3, 640, 640]	INT8	20.7	34.1	61.2	80.2	/	/	/
	yolov10s	[1, 3, 640, 640]	INT8	10.3	16.9	33.8	39.9	/	/	/
yolo11	yolo11n	[1, 3, 640, 640]	INT8	20.6	34.0	60.0	77.9	11.7	17.0	17.6
	yolo11s	[1, 3, 640, 640]	INT8	10.2	16.7	33.0	38.2	5.0	7.3	8.4
	yolo11m	[1, 3, 640, 640]	INT8	4.6	6.5	12.7	14.6	2.8	4.0	5.1
yolox	yolox_s	[1, 3, 640, 640]	INT8	15.2	18.3	37.1	41.5	10.6	15.7	23.0
	yolox_m	[1, 3, 640, 640]	INT8	6.6	8.2	16.0	17.6	4.6	6.8	10.7
ppyoloe	ppyoloe_s	[1, 3, 640, 640]	INT8	17.1	20.0	32.5	41.3	11.2	16.4	21.1
	ppyoloe_m	[1, 3, 640, 640]	INT8	7.8	9.2	15.8	17.8	5.2	7.7	9.4
yolo_world	yolo_world_v2s	[1, 3, 640, 640]	INT8	7.4	9.6	22.1	22.3	/	/	/
	clip_text	[1, 20]	FP16	29.8	67.4	95.8	63.5	/	/	/
yolov8_pose	yolov8n-pose	[1, 3, 640, 640]	INT8	22.6	31.0	55.9	66.8	/	/	/
deeplabv3	deeplab-v3-plus-mobilenet-v2	[1, 513, 513, 1]	INT8	10.9	21.4	34.0	39.4	10.1	13.0	4.4
yolov5_seg	yolov5n-seg	[1, 3, 640, 640]	INT8	32.2	38.5	69.3	88.3	28.6	42.2	49.6
	yolov5s-seg	[1, 3, 640, 640]	INT8	15.0	18.1	36.8	41.6	9.6	14.0	22.5
	yolov5m-seg	[1, 3, 640, 640]	INT8	6.8	8.4	16.4	18.0	4.7	6.8	10.8
yolov8_seg	yolov8n-seg	[1, 3, 640, 640]	INT8	27.8	33.0	60.8	71.1	18.6	27.6	32.9
	yolov8s-seg	[1, 3, 640, 640]	INT8	11.7	14.1	28.9	30.8	6.6	9.8	14.6
	yolov8m-seg	[1, 3, 640, 640]	INT8	5.2	6.4	12.6	12.7	3.1	4.6	6.9
ppseg	ppseg_lite_1024x512	[1, 3, 512, 512]	INT8	5.9	13.9	35.7	33.6	18.4	27.1	20.9
mobilesam	mobilesam_encoder_tiny	[1, 3, 448, 448]	FP16	1.0	6.6	10.0	11.9	/	/	/
	mobilesam_decoder	[1, 1, 112, 112]	FP16	24.3	69.6	116.4	108.6	/	/	/
RetinaFace	RetinaFace_mobile320	[1, 3, 320, 320]	INT8	156.4	300.8	227.2	470.5	144.8	212.5	198.5
	RetinaFace_resnet50_320	[1, 3, 320, 320]	INT8	18.7	26.9	49.2	56.6	14.6	20.8	24.6
LPRNet	lprnet	[1, 3, 24, 94]	FP16	143.2	420.6	586.4	647.8	30.6(INT8)	47.6(INT8)	30.1(INT8)
PPOCR-Det	ppocrv4_det	[1, 3, 480, 480]	INT8	22.1	28.0	50.7	64.3	11.0	16.1	14.2
PPOCR-Rec	ppocrv4_rec	[1, 3, 48, 320]	FP16	19.5	54.3	73.9	96.8	1.0	1.6	6.7
lite_transformer	lite-transformer-encoder-16	embedding-256, token-16	FP16	337.5	725.8	867.6	784.1	22.7	35.4	98.3
	lite-transformer-decoder-16	embedding-256, token-16	FP16	142.5	252.0	343.8	272.3	48.0	65.8	109.9
clip	clip_images	[1, 3, 224, 224]	FP16	2.3	3.4	6.5	6.7	/	/	/
	clip_text	[1, 20]	FP16	29.7	66.6	96.0	63.7	/	/	/
wav2vec2	wav2vec2_base_960h_20s	20s audio	FP16	RTF 0.817	RTF 0.323	RTF 0.133	RTF 0.073	/	/	/
whisper	whisper_base_20s	20s audio	FP16	RTF 1.178	RTF 0.420	RTF 0.215	RTF 0.218	/	/	/
zipformer	zipformer-bilingual-zh-en-t	streaming audio	FP16	RTF 0.196	RTF 0.116	RTF 0.065	RTF 0.082	/	/	/
yamnet	yamnet_3s	3s audio	FP16	RTF 0.013	RTF 0.008	RTF 0.004	RTF 0.005	/	/	/
mms_tts	mms_tts_eng_200	token-200	FP16	RTF 0.311	RTF 0.138	RTF 0.069	RTF 0.069	/	/	/

• 该性能数据基于各平台的最大NPU频率进行测试
• 该性能数据指模型推理的耗时, 不包含前后处理的耗时
• /表示当前版本暂不支持
  

Demo编译说明

对于 Linux 系统的开发板:

1. ./build-linux.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
   
2.     -t : target (rk356x/rk3576/rk3588/rv1106/rv1126b/rv1126/rk1808)
   
3.     -a : arch (aarch64/armhf)
   
4.     -d : demo name
   
5.     -b : build_type(Debug/Release)
   
6.     -m : enable address sanitizer, build_type need set to Debug
   
7. Note: 'rk356x' represents rk3562/rk3566/rk3568, 'rv1106' represents rv1103/rv1106, 'rv1126' represents rv1109/rv1126，'rv1126b' is different from 'rv1126'.
   
8. 
   
9. # 以编译64位Linux RK3566的yolov5 demo为例:
   
10. ./build-linux.sh -t rk356x -a aarch64 -d yolov5

复制代码

对于 Android 系统的开发板:

1. # 对于 Android 系统的开发板, 首先需要根据实际情况, 设置安卓NDK编译工具的路径
   
2. <font color="#ff0000">export</font> ANDROID_NDK_PATH=~/opts/ndk/android-ndk-r18b
   
3. ./build-android.sh -t <target> -a <arch> -d <build_demo_name> [-b <build_type>] [-m]
   
4.     -t : target (rk356x/rk3588/rk3576)
   
5.     -a : arch (arm64-v8a/armeabi-v7a)
   
6.     -d : demo name
   
7.     -b : build_type (Debug/Release)
   
8.     -m : enable address sanitizer, build_type need set to Debug
   
9. 
   
10. # 以编译64位Android RK3566的yolov5 demo为例:
    
11. ./build-android.sh -t rk356x -a arm64-v8a -d yolov5

复制代码

版本	RKNPU2 SDK	RKNPU1 SDK
2.3.2	>=2.3.2	>=1.7.5
2.3.0	>=2.3.0	>=1.7.5
2.2.0	>=2.2.0	>=1.7.5
2.1.0	>=2.1.0	>=1.7.5
2.0.0	>=2.0.0	>=1.7.5
1.6.0	>=1.6.0	-
1.5.0	>=1.5.0	>=1.7.3

RKNPU相关资料

• RKNPU2 SDK: https://github.com/airockchip/rknn-toolkit2
• RKNPU1 SDK: https://github.com/airockchip/rknn-toolkit