«上一篇/Previous Article|本期目录/Table of Contents|下一篇/Next Article»

《建筑科学与工程学报》[ISSN:1673-2049/CN:61-1442/TU]

卷:

39卷

期数:

2022年04期

页码:

100-107

栏目:

出版日期:

2022-07-12

文章信息/Info

Title:

Image Dataset Synthetic Method for Construction Machinery Based on Deep Learning

文章编号:

1673-2049(2022)04-0100-08

作者:

卢昱杰^1,2,3,刘 博¹,刘金杉¹,赵宪忠¹

(1. 同济大学 建筑工程系,上海 200092; 2. 同济大学 工程结构性能演化与控制教育部重点实验室,上海 200092; 3. 同济大学 上海智能科学与技术研究院,上海 200092)

Author(s):

LU Yu-jie^1,2,3, LIU Bo¹, LIU Jin-shan¹, ZHAO Xian-zhong¹

(1. Department of Building Engineering, Tongji University, Shanghai 200092, China; 2. Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University, Shanghai 200092, China; 3. Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China)

关键词:

计算机视觉;  深度学习;  建筑施工;  施工机械;  合成数据集

Keywords:

computer vision;  deep learning;  building construction;  construction machinery;  synthetic dataset

分类号:

TU973.2

DOI:

10.19815/j.jace.2021.07146

文献标志码:

A

摘要:

工程现场环境复杂,获取包含丰富信息的图像难度大且标注成本高,造成基于计算机视觉的深度学习施工机械图像数据集构建困难。为满足快速、高质量构建建筑工程领域施工机械深度学习图像数据集,提出一种基于三维建模引擎的施工机械图像生成与自动标注方法,并以挖掘机为例构建了名为SCED(Synthesized Construction Equipment Dataset)的挖掘机数据集。首先,采用三维建模引擎UE4对目标挖掘机设备进行模型构建,然后借助UnrealCV工具对原始模型进行多角度、多区域的图像采集,使用自编写模块实现自动语义分割与掩码图像生成,并完成图像的自动标注,最终生成包含10 000张图像的数据集。与现有公开机械数据集进行了目标尺寸、数量与构建工作量的对比,并比较了构建效率与成本,最后进行了图像数据集质量与效果验证。结果表明:该构建方法综合效率更高且成本更低,构建的SCED图像数据集丰富性和泛化能力更好,针对小目标物具有更好的检测效果; 研究成果可为今后建筑施工领域深度学习图像数据集的构建提供参考依据。

Abstract:

The complex environment of construction sites, the difficulty of obtaining images containing rich information and the high cost of image annotation make it difficult to build deep learning construction machinery image datasets based on computer vision. In order to build a fast and high-quality deep learning image dataset of construction machinery in the construction field, a method of generating and automatically annotating images of construction machinery based on a 3D modeling engine was proposed, and an excavator dataset named SCED(Synthesized Construction Equipment Dataset)was constructed with an excavator as an illustration. Firstly, the target excavator equipment was modelled using the 3D modelling engine UE4, then the original model was captured from multiple angles and regions utilizing UnrealCV tool, automatic semantic segmentation and mask image generation were achieved using a self-written module, and the bounding boxes of excavator instance were automatically annotated, generating a dataset containing 10 000 images. The target size, number and construction effort were compared with existing publicly available mechanical datasets, and the construction efficiency and cost were also compared, and finally the quality and effectiveness of the image dataset were verified. The results show that the construction method is more efficient and less expensive, and the SCED image dataset is richer and better generalised, with better detection results for small targets. The research findings can provide a reference for the construction of deep learning image datasets in the field of building construction in the future.

参考文献/References:

[1] 方 明,孙腾腾,邵 桢.基于改进YOLOv2的快速安全帽佩戴情况检测[J].光学精密工程,2019,27(5):1196-1205.
FANG Ming,SUN Teng-teng,SHAO Zhen.Fast Helmet-wearing-condition Detection Based on Improved YOLOv2[J].Optics and Precision Engineering,2019,27(5):1196-1205.
[2]孟诗乔,张啸天,乔甦阳,等.基于深度学习的网格优化裂缝检测模型研究[J].建筑结构学报,2020,41(增2):404-410.
MENG Shi-qiao,ZHANG Xiao-tian,QIAO Su-yang,et al.Research on Grid Optimized Crack Detection Model Based on Deep Learning[J].Journal of Building Structures,2020,41(S2):404-410.
[3]李书进,赵 源,孔 凡,等.卷积神经网络在结构损伤诊断中的应用[J].建筑科学与工程学报,2020,37(6):29-37.
LI Shu-jin,ZHAO Yuan.KONG Fan,et al.Application of Convolutional Neural Network in Structural Damage Identification[J].Journal of Architecture and Civil Engineering,2020,37(6):29-37.
[4]黄宏伟,李庆桐.基于深度学习的盾构隧道渗漏水病害图像识别[J].岩石力学与工程学报,2017,36(12):2861-2871.
HUANG Hong-wei,LI Qing-tong.Image Recognition for Water Leakage in Shield Tunnel Based on Deep Learning[J].Chinese Journal of Rock Mechanics and Engineering,2017,36(12):2861-2871.
[5]薛亚东,李宜城.基于深度学习的盾构隧道衬砌病害识别方法[J].湖南大学学报(自然科学版),2018,45(3):100-109.
XUE Ya-dong,LI Yi-cheng.A Method of Disease Recognition for Shield Tunnel Lining Based on Deep Learning[J].Journal of Hunan University(Natural Sciences),2018,45(3):100-109.
[6]叶肖伟,张小明,倪一清,等.基于机器视觉技术的桥梁挠度测试方法[J].浙江大学学报(工学版),2014,48(5):813-819.
YE Xiao-wei,ZHANG Xiao-ming,NI Yi-qing,et al.Bridge Deflection Measurement Method Based on Machine Vision Technology[J].Journal of Zhejiang University(Engineering Science),2014,48(5):813-819.
[7]AN X H,ZHOU L,LIU Z G,et al.Dataset and Benchmark for Detecting Moving Objects in Construction Sites[J].Automation in Construction,2021,122:103482.
[8]XIAO B,KANG S C.Development of an Image Data Set of Construction Machines for Deep Learning Object Detection[J].Journal of Computing in Civil Engineering,2021,35(2):05020005.
[9]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].arXiv:1049.1556.
[10]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//IEEE.Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2014:580-587.
[11]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//IEEE.Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE,2016:770-778.
[12]LI Z,PENG C,YU G,et al.DetNet:Design Backbone for Object Detection[C]//FERRARI V,HEBERT M,SMINCHISESCU C,et al.Proceedings of the 15th European Conference.Munich:LNCS,2018:339-354.
[13]陈伟华,南鹏飞,闫孝姮,等.基于深度学习的采煤机截割轨迹预测及模型优化[J].煤炭学报,2020,45(12):4209-4215.
CHEN Wei-hua,NAN Peng-fei,YAN Xiao-heng,et al.Prediction and Model Optimization of Shearer Memory Cutting Trajectory Based on Deep Learning[J].Journal of China Coal Society,2020,45(12):4209-4215.
[14]张 繁,王 通,黄可蒙,等.基于Unity3D的改进实时红外仿真系统[J].计算机辅助设计与图形学学报,2018,30(7):1177-1186.
ZHANG Fan,WANG Tong,HUANG Ke-meng,et al.One Improved Real-time Infrared Simulation System Based on Unity3D[J].Journal of Computer-aided Design & Computer Graphics,2018,30(7):1177-1186.
[15]QIU W,ZHONG F,YI Z,et al.UnrealCV:Virtual Worlds for Computer Vision[C]//LIU Q,LIENHART R,WANG H H.Proceedings of the 25th ACM International Conference on Multimedia.New York:Association for Computing Machinery,2017:1221-1224.
[16]张素兰,郭 平,张继福,等.图像语义自动标注及其粒度分析方法[J].自动化学报,2012,38(5):688-697.
ZHANG Su-lan,GUO Ping,ZHANG Ji-fu,et al.Automatic Semantic Image Annotation with Granular Analysis Method[J].Acta Automatica Sinica,2012,38(5):688-697.
[17]高 慧,张继威,来 扬,等.深度学习的人体图像半自动标注系统[J].北京邮电大学学报,2021,44(1):104-109.
GAO Hui,ZHANG Ji-wei,LAI Yang,et al.Deep Learning Based Semi-automatic Labeling System for Human Images[J].Journal of Beijing University of Posts and Telecommunications,2021,44(1):104-109.
[18]TAN M X,PANG R M,LE Q V.EfficientDet:Scalable and Efficient Object Detection[C]//IEEE.Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:10778-10787.
[19]柯圣财,赵永威,李弼程,等.基于卷积神经网络和监督核哈希的图像检索方法[J].电子学报,2017,45(1):157-163.
KE Sheng-cai,ZHAO Yong-wei,LI Bi-cheng,et al.Image Retrieval Based on Convolutional Neural Network and Kernel-Based Supervised Hashing[J].Acta Electronica Sinica,2017,45(1):157-163.
[20]刘 洋,战荫伟.基于深度学习的小目标检测算法综述[J].计算机工程与应用,2021,57(2):37-48.
LIU Yang,ZHAN Yin-wei.Survey of Small Object Detection Algorithms Based on Deep Learning[J].Computer Engineering and Applications,2021,57(2):37-48.
[21]刘金杉,李元海,卢昱杰,等.基于深度学习的隧道衬砌表观病害模拟检测系统[J/OL].哈尔滨工业大学学报:1-10[2022-02-14].http://kns.cnki.net/kcms/detail/23.1235.T. 20220112.0853.002.html.
LIU Jin-shan,LI Yuan-hai,LU Yu-jie, et al.Tunnel Lining Surface Defect Simulation and Detection System Based on Deep Learning[J/OL].Journal of Harbin Institute of Technology:1-10[2022-02-14].http://kns.cnki.net/kcms/detail/23.1235.T.20220112.0853.002.html.
[22]卫 星,李 佳,孙 晓,等.基于混合生成对抗网络的多视角图像生成算法[J].自动化学报,2021,47(11):2623-2636.
WEI Xing,LI Jia,SUN Xiao,et al.Cross-view Image Generation via Mixture Generative Adversarial Network[J].Acta Automatica Sinica,2021,47(11):2623-2636.

相似文献/References:

[1]李书进,赵 源,孔 凡,等.卷积神经网络在结构损伤诊断中的应用[J].建筑科学与工程学报,2020,37(06):29.
　LI Shu-jin,ZHAO Yuan,KONG Fan,et al.Application of Convolutional Neural Network in Structural Damage Identification[J].Journal of Architecture and Civil Engineering,2020,37(04):29.
[2]杨 铄,许清风,王卓琳.基于卷积神经网络的结构损伤识别研究进展[J].建筑科学与工程学报,2022,39(04):38.[doi:10.19815/j.jace.2022.02043]
　YANG Shuo,XU Qing-feng,WANG Zhuo-lin.Research Progress on Structural Damage Detection Based on Convolutional Neural Networks[J].Journal of Architecture and Civil Engineering,2022,39(04):38.[doi:10.19815/j.jace.2022.02043]

备注/Memo

备注/Memo:

收稿日期:2021-07-03
基金项目:国家自然科学基金项目(52078374)
作者简介:卢昱杰(1985-),男,贵州贵阳人,教授,博士研究生导师,工学博士,E-mail:Lu6@tongji.edu.cn。
通信作者:刘金杉(1995-),男,河南洛阳人,工学硕士研究生,E-mail:supernova_ks@outlook.com。

常用功能

更新日期/Last Update: 2022-07-10