|Table of Contents|

Image Dataset Synthetic Method for Construction Machinery Based on Deep Learning(PDF)

《建筑科学与工程学报》[ISSN:1673-2049/CN:61-1442/TU]

Issue:
2022年04期
Page:
100-107
Research Field:
Publishing date:

Info

Title:
Image Dataset Synthetic Method for Construction Machinery Based on Deep Learning
Author(s):
LU Yu-jie1; 2; 3;  LIU Bo1;  LIU Jin-shan1;  ZHAO Xian-zhong1
(1. Department of Building Engineering, Tongji University, Shanghai 200092, China; 2. Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University, Shanghai 200092, China; 3. Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China)
Keywords:
computer vision;  deep learning;  building construction;  construction machinery;  synthetic dataset
PACS:
TU973.2
DOI:
10.19815/j.jace.2021.07146
Abstract:
The complex environment of construction sites, the difficulty of obtaining images containing rich information and the high cost of image annotation make it difficult to build deep learning construction machinery image datasets based on computer vision. In order to build a fast and high-quality deep learning image dataset of construction machinery in the construction field, a method of generating and automatically annotating images of construction machinery based on a 3D modeling engine was proposed, and an excavator dataset named SCED(Synthesized Construction Equipment Dataset)was constructed with an excavator as an illustration. Firstly, the target excavator equipment was modelled using the 3D modelling engine UE4, then the original model was captured from multiple angles and regions utilizing UnrealCV tool, automatic semantic segmentation and mask image generation were achieved using a self-written module, and the bounding boxes of excavator instance were automatically annotated, generating a dataset containing 10 000 images. The target size, number and construction effort were compared with existing publicly available mechanical datasets, and the construction efficiency and cost were also compared, and finally the quality and effectiveness of the image dataset were verified. The results show that the construction method is more efficient and less expensive, and the SCED image dataset is richer and better generalised, with better detection results for small targets. The research findings can provide a reference for the construction of deep learning image datasets in the field of building construction in the future.

References:

[1] 方 明,孙腾腾,邵 桢.基于改进YOLOv2的快速安全帽佩戴情况检测[J].光学精密工程,2019,27(5):1196-1205.
FANG Ming,SUN Teng-teng,SHAO Zhen.Fast Helmet-wearing-condition Detection Based on Improved YOLOv2[J].Optics and Precision Engineering,2019,27(5):1196-1205.
[2]孟诗乔,张啸天,乔甦阳,等.基于深度学习的网格优化裂缝检测模型研究[J].建筑结构学报,2020,41(增2):404-410.
MENG Shi-qiao,ZHANG Xiao-tian,QIAO Su-yang,et al.Research on Grid Optimized Crack Detection Model Based on Deep Learning[J].Journal of Building Structures,2020,41(S2):404-410.
[3]李书进,赵 源,孔 凡,等.卷积神经网络在结构损伤诊断中的应用[J].建筑科学与工程学报,2020,37(6):29-37.
LI Shu-jin,ZHAO Yuan.KONG Fan,et al.Application of Convolutional Neural Network in Structural Damage Identification[J].Journal of Architecture and Civil Engineering,2020,37(6):29-37.
[4]黄宏伟,李庆桐.基于深度学习的盾构隧道渗漏水病害图像识别[J].岩石力学与工程学报,2017,36(12):2861-2871.
HUANG Hong-wei,LI Qing-tong.Image Recognition for Water Leakage in Shield Tunnel Based on Deep Learning[J].Chinese Journal of Rock Mechanics and Engineering,2017,36(12):2861-2871.
[5]薛亚东,李宜城.基于深度学习的盾构隧道衬砌病害识别方法[J].湖南大学学报(自然科学版),2018,45(3):100-109.
XUE Ya-dong,LI Yi-cheng.A Method of Disease Recognition for Shield Tunnel Lining Based on Deep Learning[J].Journal of Hunan University(Natural Sciences),2018,45(3):100-109.
[6]叶肖伟,张小明,倪一清,等.基于机器视觉技术的桥梁挠度测试方法[J].浙江大学学报(工学版),2014,48(5):813-819.
YE Xiao-wei,ZHANG Xiao-ming,NI Yi-qing,et al.Bridge Deflection Measurement Method Based on Machine Vision Technology[J].Journal of Zhejiang University(Engineering Science),2014,48(5):813-819.
[7]AN X H,ZHOU L,LIU Z G,et al.Dataset and Benchmark for Detecting Moving Objects in Construction Sites[J].Automation in Construction,2021,122:103482.
[8]XIAO B,KANG S C.Development of an Image Data Set of Construction Machines for Deep Learning Object Detection[J].Journal of Computing in Civil Engineering,2021,35(2):05020005.
[9]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-scale Image Recognition[J].arXiv:1049.1556.
[10]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//IEEE.Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2014:580-587.
[11]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//IEEE.Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE,2016:770-778.
[12]LI Z,PENG C,YU G,et al.DetNet:Design Backbone for Object Detection[C]//FERRARI V,HEBERT M,SMINCHISESCU C,et al.Proceedings of the 15th European Conference.Munich:LNCS,2018:339-354.
[13]陈伟华,南鹏飞,闫孝姮,等.基于深度学习的采煤机截割轨迹预测及模型优化[J].煤炭学报,2020,45(12):4209-4215.
CHEN Wei-hua,NAN Peng-fei,YAN Xiao-heng,et al.Prediction and Model Optimization of Shearer Memory Cutting Trajectory Based on Deep Learning[J].Journal of China Coal Society,2020,45(12):4209-4215.
[14]张 繁,王 通,黄可蒙,等.基于Unity3D的改进实时红外仿真系统[J].计算机辅助设计与图形学学报,2018,30(7):1177-1186.
ZHANG Fan,WANG Tong,HUANG Ke-meng,et al.One Improved Real-time Infrared Simulation System Based on Unity3D[J].Journal of Computer-aided Design & Computer Graphics,2018,30(7):1177-1186.
[15]QIU W,ZHONG F,YI Z,et al.UnrealCV:Virtual Worlds for Computer Vision[C]//LIU Q,LIENHART R,WANG H H.Proceedings of the 25th ACM International Conference on Multimedia.New York:Association for Computing Machinery,2017:1221-1224.
[16]张素兰,郭 平,张继福,等.图像语义自动标注及其粒度分析方法[J].自动化学报,2012,38(5):688-697.
ZHANG Su-lan,GUO Ping,ZHANG Ji-fu,et al.Automatic Semantic Image Annotation with Granular Analysis Method[J].Acta Automatica Sinica,2012,38(5):688-697.
[17]高 慧,张继威,来 扬,等.深度学习的人体图像半自动标注系统[J].北京邮电大学学报,2021,44(1):104-109.
GAO Hui,ZHANG Ji-wei,LAI Yang,et al.Deep Learning Based Semi-automatic Labeling System for Human Images[J].Journal of Beijing University of Posts and Telecommunications,2021,44(1):104-109.
[18]TAN M X,PANG R M,LE Q V.EfficientDet:Scalable and Efficient Object Detection[C]//IEEE.Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:10778-10787.
[19]柯圣财,赵永威,李弼程,等.基于卷积神经网络和监督核哈希的图像检索方法[J].电子学报,2017,45(1):157-163.
KE Sheng-cai,ZHAO Yong-wei,LI Bi-cheng,et al.Image Retrieval Based on Convolutional Neural Network and Kernel-Based Supervised Hashing[J].Acta Electronica Sinica,2017,45(1):157-163.
[20]刘 洋,战荫伟.基于深度学习的小目标检测算法综述[J].计算机工程与应用,2021,57(2):37-48.
LIU Yang,ZHAN Yin-wei.Survey of Small Object Detection Algorithms Based on Deep Learning[J].Computer Engineering and Applications,2021,57(2):37-48.
[21]刘金杉,李元海,卢昱杰,等.基于深度学习的隧道衬砌表观病害模拟检测系统[J/OL].哈尔滨工业大学学报:1-10[2022-02-14].http://kns.cnki.net/kcms/detail/23.1235.T. 20220112.0853.002.html.
LIU Jin-shan,LI Yuan-hai,LU Yu-jie, et al.Tunnel Lining Surface Defect Simulation and Detection System Based on Deep Learning[J/OL].Journal of Harbin Institute of Technology:1-10[2022-02-14].http://kns.cnki.net/kcms/detail/23.1235.T.20220112.0853.002.html.
[22]卫 星,李 佳,孙 晓,等.基于混合生成对抗网络的多视角图像生成算法[J].自动化学报,2021,47(11):2623-2636.
WEI Xing,LI Jia,SUN Xiao,et al.Cross-view Image Generation via Mixture Generative Adversarial Network[J].Acta Automatica Sinica,2021,47(11):2623-2636.

Memo

Memo:
-
Last Update: 2022-07-10