Computer Vision-Based Health Monitoring of Aging Rural Bridge Infrastructures

Yajyoo Shrestha, Dr. Chungwook Sim, Ji Young Lee Author

08/02/2020 Added

19 Plays

Description

This poster video describes my summer project which focuses on the use of optical sensors to collect bridge images which can be used by neural network to monitor the bridge health.

Searchable Transcript

Search:

[00:00:01.753]Hello!
[00:00:02.393]My name is Yajyoo Shrestha
[00:00:03.977]and I am a civil engineering
[00:00:05.497]undergraduate student at
[00:00:06.604]University of Nebraska-Lincoln.
[00:00:08.964]This summer, I had the opportunity
[00:00:11.290]of being involved in the Summer Nebraska
[00:00:13.640]Engineering Research Program
[00:00:15.430]and the title of my project is
[00:00:17.670]Computer Vision-Based Health Monitoring
[00:00:19.535]of Aging Rural Bridge Infrastructures.
[00:00:22.920]I have been mentored by faculty member,
[00:00:25.000]Dr. Chungwook Sim and PhD student,
[00:00:28.043]Ji Young Lee.
[00:00:31.464]According to Federal Highway Association,
[00:00:33.804]every state is required to update
[00:00:35.534]the condition of its bridge infrastructure
[00:00:38.360]every 1-2 years. In Nebraska alone,
[00:00:41.880]there are more than 15000 bridges
[00:00:43.818]that are longer than 20 ft and
[00:00:46.041]need monitoring every 1-2 years.
[00:00:49.110]Conventionally, the process of crack
[00:00:51.186]inspection involves skilled human
[00:00:53.086]inspectors to manually mark and chart out
[00:00:55.300]the cracks that they observe on the site.
[00:00:57.720]However, this process is not only
[00:00:59.710]time consuming but also costly
[00:01:01.539]and subjective.
[00:01:03.195]The concept of automated crack detection
[00:01:06.155]for health monitoring is an efficient tool
[00:01:08.585]to aid manual inspection of bridges
[00:01:11.098]because of the large number of
[00:01:12.648]infrastructures that need to be
[00:01:14.178]diagnosed. Computer vision-based health
[00:01:16.700]monitoring is a method wherein the AI
[00:01:18.130]system trains the computers to interpret
[00:01:20.342]the visual world of bridge decks and the
[00:01:22.982]cracks on the decks to interpret the state
[00:01:25.332]of the bridge.
[00:01:27.743]This can be significantly beneficial to
[00:01:29.980]prioritize infrastructures on the basis of
[00:01:32.411]immediate needs and repairs.
[00:01:36.763]In this study, we focused
[00:01:38.181]on making use of optical sensors
[00:01:40.034]to detect and measure cracks.
[00:01:42.347]Although there have been a number of crack
[00:01:44.209]detection researches over the years that
[00:01:45.892]focus on detecting cracks, most of them
[00:01:48.315]have implemented their algorithm only in a
[00:01:50.971]local area.
[00:01:52.759]This research not only focuses on crack
[00:01:56.760]detection on a local area, but rather on
[00:01:59.820]the entire bridge deck.
[00:02:02.050]The proposed system could be used to
[00:02:04.090]create a database that could be used for
[00:02:06.610]future maintenance and in classifying the
[00:02:08.385]constraints that affect the service life
[00:02:10.496]of bridge decks.
[00:02:16.840]An optical sensor camera system
[00:02:19.340]was used in a UAV, which is an unmanned
[00:02:21.687]aerial vehicle,
[00:02:23.058]was flown across a bridge to take multiple
[00:02:25.433]images of the infrastructure.
[00:02:27.352]We used a MAVIC Pro model of UAV in our
[00:02:29.862]field work to capture images of the bridge
[00:02:32.267]and the drone was programmed on the DJI GS
[00:02:35.407]Pro software through which the area over
[00:02:37.221]which the drone was considered
[00:02:38.395]to be flown was marked.
[00:02:42.092]As seen in this picture,
[00:02:43.601]in order to attain the desired results,
[00:02:45.811]we followed three major steps
[00:02:47.639]and they were: data acquisition,
[00:02:49.393]localisation and crack detection.
[00:02:53.294]The first of which is data acquisition.
[00:02:56.198]The UAV was flown over the concrete bridge
[00:02:58.905]located at Pine St at S 70th Street
[00:03:01.965]in Omaha, Nebraska.
[00:03:03.929]The bridge chosen for this experimental
[00:03:05.708]study runs over the Little Papillion Creek
[00:03:07.718]and is located nearby the PKI Institute
[00:03:09.875]at University of Nebraska- Omaha.
[00:03:13.957]Using the DJI GS Pro software,
[00:03:15.944]we first chose the area on a map that we
[00:03:17.854]wanted the drone to cover through
[00:03:20.014]its optical lens. Then, the software
[00:03:21.878]generated a path that would best
[00:03:24.952]cover the selected area as we can see
[00:03:27.972]in Figure 5.
[00:03:29.588]Upon following the path,
[00:03:30.903]the UAV covered an area of 0.39 acres
[00:03:35.331]while flying at an average speed of 1.8mph
[00:03:38.448]at a vertical distance of roughly 20 ft
[00:03:40.845]from the bridge deck.
[00:03:41.970]The UAV captured 120 pictures on its
[00:03:44.770]flight over the bridge deck.
[00:03:46.532]Figure 4 shows an example of one of the
[00:03:50.092]collected images from the optical sensors.
[00:03:55.487]Upon collecting the raw images captured
[00:03:57.352]by the optical sensors, these images were
[00:03:59.604]stitched together in Pix4D software.
[00:04:02.353]The stitched image produced is ultimately
[00:04:03.978]used to label cracks in the bridge deck.
[00:04:07.016]However, in order to input the raw UAV
[00:04:08.985]image files into the neural network
[00:04:10.595]that is used as a toolbox,
[00:04:12.532]this image needs to be split into
[00:04:14.242]multiple smaller images.
[00:04:15.733]This is because the neural network cannot
[00:04:17.583]accept a very high resolution image to
[00:04:20.047]detect cracks as this results in
[00:04:22.042]inefficient and inaccurate results
[00:04:24.852]at times.
[00:04:27.672]In order to detect the cracks
[00:04:28.788]on the bridge deck, a crack heirarchy
[00:04:30.368]is followed using crack pixels
[00:04:31.880]and crack segments.
[00:04:33.785]The crack detention portion of the
[00:04:35.134]research gives us information about the
[00:04:36.923]crack widths on the bridge deck.
[00:04:38.970]The related information about these cracks
[00:04:40.627]such as the increments in the crack gaps
[00:04:42.197]and crack widths are pertinent to
[00:04:43.657]understanding the health condition
[00:04:45.908]of the bridge infrastructure.
[00:04:47.365]The raw images of the bridge captured by
[00:04:49.512]the UAV serve as the input images for our
[00:04:52.383]deep learning neural network.
[00:04:53.868]Whereas, the labelled images act
[00:04:55.844]as output images.
[00:05:00.421]The deep learning neural network
[00:05:01.738]that we used in this project is Mask
[00:05:03.405]Region-based Convolutional
[00:05:06.145]Neural Network or Mask R-CNN in short.
[00:05:08.673]This neural network can separate different
[00:05:10.898]features of an object in an image which
[00:05:12.539]is useful in computer vision.
[00:05:14.014]For this part of the project, we utilised
[00:05:16.200]the supervised learning wherein the input
[00:05:18.423]and output images are both given to us.
[00:05:21.243]We use the Mask R-CNN to use crack
[00:05:23.503]detection from these images and an example
[00:05:26.892]of the diagrammatic representation of the
[00:05:29.652]use of Mask R-CNN is shown in Figure 7.
[00:05:33.102]When the images collected from the bridge
[00:05:34.909]site through the UAV are fed into this
[00:05:36.991]Mask R-CNN that is fully segmented with
[00:05:40.141]domain experts, the neural network can be
[00:05:42.422]trained to learn locations where cracks
[00:05:44.363]exist and what their features look like.
[00:05:48.100]In conclusion,
[00:05:49.069]the use of computer vision in the world
[00:05:51.389]of civil engineering, especially
[00:05:53.299]in the field of bridge infrastructures
[00:05:55.269]can contribute highly to monitor the
[00:05:58.277]health condition of these bridges in an
[00:06:00.537]impactful and time efficient manner.
[00:06:02.570]The application of crack detention
[00:06:04.861]and crack mapping used in this study
[00:06:07.650]is also beneficial to identify the
[00:06:09.810]condition of the entire bridge
[00:06:11.338]and not just a local area.
[00:06:13.116]In future, necessary measurements of
[00:06:16.422]the cracks need to be taken
[00:06:19.862]to maintain a quantitative database
[00:06:21.716]which is beneficial in monitoring the
[00:06:23.247]bridge health.
[00:06:24.174]The changes in crack widths, the changes
[00:06:27.316]in their lengths and gaps over time
[00:06:29.498]indicate the performance of the bridge
[00:06:31.467]over time.
[00:06:35.643]In the end,
[00:06:36.251]I would like to thank
[00:06:37.417]the College of Engineering's
[00:06:38.767]Dean's Office and Engineering Graduate
[00:06:40.810]Programs at the University of
[00:06:42.462]Nebraska- Lincoln through which
[00:06:44.571]the Summer Nebraska Engineering Research
[00:06:46.665]Program was possible.
[00:06:48.412]I would also really like to thank
[00:06:50.040]Dr. Chungwook Sim and Ji Young Lee
[00:06:52.420]for mentoring and guiding me throughout
[00:06:54.684]this experience.
[00:06:56.021]Thank You!

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Computer Vision-Based Health Monitoring of Aging Rural Bridge Infrastructures

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments