针对现有的鱼类游动轨迹提取方法不能兼顾轨迹提取效率和准确率的局限性,提出了一种基于鱼类关键点识别与定位的鱼类游动轨迹提取方法。该方法在RetinaFace算法的基础上,通过改进网络结构和损失函数、优化锚框的尺寸设计、编解码鱼类关键点(头部点和形心点)、为鱼类目标的关键点添加额外的标注并制作成鱼类关键点数据集等改进策略,构建了基于关键点识别的鱼类轨迹提取模型。研究结果表明,本研究方法对鱼体关键点识别的精度很高,准确率、召回率、平均精度均值3项精度评价指标分别为97.12%、95.72%、96.42%;所提取的轨迹坐标平均相对偏差为MREx(0.065%,0.092%)、MREy(0.112%,0.011%),与鱼类的实际游动轨迹基本吻合;鱼类目标关键点的识别速度可达32帧/s,能够满足实时提取鱼类轨迹的需求。
Abstract
The existing fish trajectory extraction methods fail to balance efficiency and accuracy. This study introduces a fish trajectory extraction approach based on fish landmark recognition and location utilizing the RetinaFace algorithm. The method entails constructing a fish trajectory extraction model through enhanced network structure and loss function for landmark detection, optimizing anchor size design, and encoding and decoding fish landmarks (specifically, the head point and centroid point). Additionally, it involves supplementing landmarks of fish targets with extra labels and generating a fish key point dataset. The findings demonstrate that the proposed research method achieves high accuracy in identifying fish landmarks, with precision evaluation indices including an accuracy rate of 97.12%, a recall rate of 95.72%, and a mean average precision of 96.42%. Moreover, the average relative deviation of the extracted trajectory coordinates is MREx(0.065%,0.092%) and MREy(0.112%,0.011%), aligning closely with the actual swimming trajectory of fish. The recognition rate for landmarks of fish targets reaches 32 frames per second, which meets the real-time extraction requirements for fish trajectory recognition.
关键词
鱼类 /
鱼道监测 /
鱼类关键点检测 /
鱼类游动轨迹提取 /
RetinaFace模型
Key words
fish /
fishway monitoring /
detection of fish landmark /
fish trajectory extraction /
RetinaFace model
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 王义川, 王 煜, 林晨宇, 等. 鱼道过鱼效果监测方法述评[J]. 生态学杂志, 2019, 38(2): 586-593.(WANG Yi-chuan, WANG Yu, LIN Chen-yu, et al. A Review on Monitoring Methods for the Effectiveness of Fishway[J]. Chinese Journal of Ecology, 2019, 38(2): 586-593.(in Chinese))
[2] 谭均军, 高 柱, 戴会超, 等. 竖缝式鱼道水力特性与鱼类运动特性相关性分析[J]. 水利学报, 2017, 48(8): 924-932, 944.(TAN Jun-jun, GAO Zhu, DAI Hui-chao, et al. The Correlation Analysis between Hydraulic Characteristics of Vertical Slot Fishway and Fish Movement Characteristics[J]. Journal of Hydraulic Engineering, 2017, 48(8): 924-932, 944.(in Chinese))
[3] 柯森繁,高 柱,刘国勇,等.基于Matlab的鱼类游泳动力学分析[J].水生生物学报,2016,40(5):985-991.(KE Sen-fan,GAO Zhu,LIU Guo-yong,et al. The Analysis of Fish Swimming Dynamics Based on the Matlab[J]. Acta Hydrobiologica Sinica,2016,40(5):985-991.(in Chinese))
[4] 刘星桥, 张 弛. 基于嵌入式图像处理系统的鱼类轨迹跟踪[J]. 江苏农业科学, 2018, 46(10): 203-207.(LIU Xing-qiao, ZHANG Chi. Study on Fish Tracking Based on Embedded Image Processing System[J]. Jiangsu Agricultural Sciences, 2018, 46(10): 203-207.(in Chinese))
[5] LI X, LIU M, ZHANG S, et al. Fish Trajectory Extraction Based on Object Detection[C]//Proceedings of 2020 39th Chinese Control Conference (CCC). Shenyang, China. New York: IEEE Press, 2020: 6584-6588.
[6] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[7] 周福欢,柴鑫雨.基于YOLOv5算法对斑马鱼幼鱼的检测研究[J].智能计算机与应用,2022,12(8):129-131,135.(ZHOU Fu-huan,CHAI Xin-yu.A Study on the Detection of Zebrafish Larvae Based on YOLOv5[J]. Intelligent Computer and Applications,2022,12(8):129-131,135.(in Chinese))
[8] O’MAHONY N, CAMPBELL S, CARVALHO A, et al. Deep Learning Vs. Traditional Computer Vision[C]//Proceedings of the 2019 Computer Vision Conference (CVC). Las Vegas, USA. May 2-3, 2019: 128-144.
[9] DENG J, GUO J, ZHOU Y, et al. RetinaFace: Single-stage Dense Face Localisation in the Wild[J]. Doi: 10.48550/arXiv.1905.00641.
[10] ROBERT B F, YUN H C B, DANIELA G, et al. Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data[M]. Switzerland:Springer Publishing Company,2016.
[11] PEDERSEN M,HAURUM J B,HEIN BENGTSON S,et al.3D-ZeF:a 3D Zebrafish Tracking Benchmark Dataset[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. New York: IEEE Press,2020:2423-2433.
[12] LI X, DING L, WANG L, et al. FPGA Accelerates Deep Residual Learning for Image Recognition[C]//Proceedings of 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC). Chengdu, China. New York: IEEE Press, 2017: 837-840.
[13] NAJIBI M, SAMANGOUEI P, CHELLAPPA R, et al. SSH: Single Stage Headless Face Detector[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. New York: IEEE Press, 2017: 4885-4894.
[14] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single Shot MultiBox Detector[C]//Proceedings of 14th European Conference on Computer Vision-ECCV 2016. Amsterdam, The Netherlands, October 11-14, 2016.
[15] 江丹丹,桂福坤.基于视频图像的鱼类行为轨迹追踪[J].浙江海洋学院学报(自然科学版),2015,34(2):112-118.(JIANG Dan-dan,GUI Fu-kun.Fish Motion Trajectory Tracing Technology Using Video Images[J].Journal of Zhejiang Ocean University (Natural Science),2015,34(2):112-118.(in Chinese))
基金
国家自然科学基金项目(52179070);国家优秀青年科学基金项目(51922065)
PDF(6940 KB)
