AR-Facilitated Safety Inspection and Fall Hazard Detection on Construction Sites
Jiazhou Liu, Aravinda S. Rao, Fucai Ke, Tim Dwyer, Benjamin Tag, Pari Delir Haghighi · 2024
Read AR-Facilitated Safety Inspection and Fall Hazard Detection on Construction Sites by Jiazhou Liu, Aravinda S. Rao, Fucai Ke, Tim Dwyer, Benjamin Tag, Pari Delir Haghighi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01273.txt.
Introduction
Jiazhou Liu and five co-authors explore head-mounted augmented reality as a safety tool on high-rise building sites. Their work-in-progress paper appeared in two thousand twenty-four. It was written with industry experts. The focus is the safety screens set around each floor's edge. These screens stop people and objects from falling, so gaps in them are a real hazard.
The team tracks which parts of the screens a worker has already checked. Machine learning spots gaps that need a closer look and repair. Reports are then produced with less manual effort. The paper also faces a hard question: watching workers raises privacy worries. It weighs ways to cut those risks. The project is early, and the authors report first progress and open concerns.
Catalog record
| CREATOR | Jiazhou Liu, Aravinda S. Rao, Fucai Ke, Tim Dwyer, Benjamin Tag, Pari Delir Haghighi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01273
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AR-Facilitated Safety Inspection and Fall Hazard Detection on Construction Sites Jiazhou Liu, Aravinda S. Rao, Fucai Ke, Tim Dwyer, Benjamin Tag, Pari Delir Haghighi Abstract Together with industry experts, we are exploring the potential of head-mounted augmented reality to facilitate safety inspections on high-rise construction sites. A particular concern in the industry is inspecting perimeter safety screens on higher levels of construction sites, intended to prevent falls of people and objects. We aim to support workers performing this inspection task by tracking which parts of the safety screens have been inspected. We use machine learning to automatically detect gaps in the perimeter screens that require closer inspection and remediation and to automate reporting. This work-in-progress paper describes the problem, our early progress, concerns around worker privacy, and the possibilities to mitigate these. \onlineid 0 \vgtccategory Research \vgtcinsertpkg \preprinttext To appear in the 23rd IEEE International Symposium on Mixed and Augmented Reality (ISMAR24) SafeAR workshop. \ieeedoi xx.xxxx/ISMAR.xxxxxxx \teaser (left) First-person view of the proposed safety inspection solution via AR headsets. Workers virtually “brush” the inspected areas with coloured and situated digital overlays. (right) A use case scenario where a worker wears an AR headset to conduct a routine safety inspection. Introduction keywords construction site, worker’s privacy, safety inspection, fall from height, augmented reality, skeleton feature extraction, face blurring Augmented Reality (AR) is an emerging and interactive technology with the potential to significantly advance the Architecture, Engineering, and Construction (AEC) industry. The ability of AR systems to bring information “out of the display” and integrate it into the world around us allows digital information to be viewed within the context of objects in the surrounding environment. Recent advances, such as low-latency, wide-field-of-view, and pass-through headsets, may help increase AR adoption across many domains. On the other hand, as reported by industry experts, most building projects still heavily rely on manual processes during construction. For example, routine safety observations and inspections need to be performed by workers or contractors, while the inspection report will be submitted after the inspection. The accuracy and comprehensiveness of the report will be dependent on the worker’s memory. With geo-locating, eye tracking, and built-in recording via head-mounted AR systems, inspectors can track inspected areas and virtually annotate the recorded scenes [ 12 ] . Moreover, AR systems can also be integrated with Artificial Intelligence (AI) and Digital Twins to automate the safety inspection procedure and fall hazard detection on construction sites [ 2 ] . Past studies have explored the benefits of applying AR in the construction industry, such as visualising subsurface utilities [ 10 , 11 ] , in-situ virtual examination of physical sites [ 1 ] , safety training including heavy equipment operation [ 14 ] . Yet, these studies also indicate that there remain challenges in applying AR in building projects, especially when it comes to data security and workers’ privacy. Tackling the challenges in this area demands a joint effort from AEC companies, technology providers, regulators, and industry associations. In this paper, we will first introduce an AR system for construction site safety inspection and fall hazard detection in \autoref sec:prototype. Then, we will discuss the potential concerns and challenges for current and future AR ecosystems for AEC projects and their possible solutions in \autoref sec:challenge. With this paper, we intend to make a first major step towards critically assessing the current AR system design to address potential risks holistically. 1 AR-Facilitated Safety Inspection and Fall Hazard Detection Falling from Height (FFH) accidents, including Fall of Person and Object, are some of the most common causes of vulnerability and fatality in high-rise building projects. In the past decades, building projects mainly relied on manual routine inspections to observe fall hazards, such as gaps in the perimeter safety screens and holes in the floors. With the development of advanced Computer Vision (CV) technologies and methods, as well as increased access to computing resources and publicly available datasets, we have seen a gradual increase in research in this area in recent years. Among the emerging technologies, camera-based CV technology has been the dominant technology for identifying FFH risks and incidents. One of the challenges of using camera-based computer vision technology is the limited availability of public datasets necessary to train the CV models. On the other hand, AR systems facilitate capturing video and images of the surrounding environment and automatically associate them with contextual information (e.g., geo-location data). The snapshots and recorded videos can then be used to train or test CV models. Thus, we propose a pipeline solution for construction site safety inspection and fall hazard detection by integrating AR and advanced CV technology (Large vision model). With the advancement of tracking ability and natural field-of-view, we use a headset-based AR, which could exploit the gaze and geo-location features to track the workers’ progress. 2 Proposed AR System Our proposed system targets AEC-specific requirements by providing the following features. Capture the dynamic construction site environment : we will use the built-in spatial mapping features from the headset to scan the surrounding environment. Indicate the progress and completeness of the inspection : we design a “brushing” feature to highlight areas that have been inspected (see \autoref fig:teaser). Annotations : Users can annotate virtually or add sticky notes to a specific location in 3D space. Voice-transcribed notes can also be generated and exported together with videos or photos. Automatic report generation system : to summarise and produce inspection reports based on notes and snapshots. Integration of Large Vision models : (e.g., segmenting safety hazards) into our AR system to automatically detect safety issues (e.g., gaps on the perimeter safety screens) 3 Challenges for future AR ecosystem for AEC projects Our vision about this AR solution and the potential future AR ecosystem used in the construction industry contains the following common tasks and contexts. • Natural interactions to take photos/ videos from headsets. • Geolocation-based tracking via the headset’s built-in sensors. • Automating safety inspection and generating a report. • Inspectors spend less time conducting site safety inspections. Several challenges and concerns arise from proposing and implementing the aforementioned AR solutions. For example, the headset continuously tracks the inspector’s physical locations. Project managers can monitor the real-time tracking data, posing privacy concerns. Moreover, the snapshots and videos may record workers or bystanders who have not consented to being recorded. Building unions strongly advocate for their workers’ privacy to be protected. Therefore, mechanisms must be implemented to prevent managers from using AR solutions to monitor worker activity closely. 3.1 Key Privacy Aspects Context-aware privacy-preserving mechanisms: Protect privacy by setting the privacy preferences based on location, scene, others’ presence, and hand gestures [ 13 ] . These preferences can be dynamically adjusted in emergencies or areas of security clearances. Federated Learning: Machine learning models are trained across multiple decentralized edge devices or servers while keeping the data locally on the machines. This is useful when we want to analyze data from multiple sites. Differential Privacy: Allows us to share aggregate information about a dataset while withholding information about individuals. This is especially useful when generating reports on worker productivity or safety compliance. Multi-Party Computation: Allows multiple stakeholders to analyze data jointly while keeping the source private. This is beneficial when