Digital Post-Disaster Reconnaissance for Civil Infrastructure in Rural Areas Using Deep Learning

Simone Williams Author

07/25/2020 Added

39 Plays

Description

Digital reconnaissance is an emerging alternative to traditional post-disaster reconnaissance that consists of internet-based data collection through social media, news articles, and local government webpages. The primary objective of this project is to evaluate the feasibility of an automated digital reconnaissance method in rural areas since rural areas are often neglected in post-disaster reconnaissance.

Searchable Transcript

Search:

[00:00:00.536]Hello. My name is Simone Williams.
[00:00:02.446]I am an undergraduate student at Worcester Polytechnic Institute,
[00:00:06.235]and this summer I conducted research in the Department of Civil and Environmental
[00:00:11.032]Engineering at the University of Nebraska – Lincoln under Dr. Christine Wittich.
[00:00:16.264]This poster will discuss Digital Post Disaster Reconnaissance for Civil
[00:00:21.179]Infrastructure in Rural Areas Using Deep Learning.
[00:00:24.555]Post-disaster reconnaissance is a critical on-site activity that is
[00:00:29.301]typically conducted after a natural disaster in order to understand
[00:00:32.951]structural performance and community resilience.
[00:00:35.921]Due to the remote locations and relatively smaller populations,
[00:00:40.051]rural areas are often neglected in post-disaster reconnaissance.
[00:00:44.254]However, rural areas are routinely impacted by natural disasters resulting in
[00:00:49.806]a lack of knowledge regarding rural resilience and rural infrastructure
[00:00:53.786]performance.
[00:00:57.084]As of July 8, 2020, the National Oceanic and Atmospheric Association reported
[00:01:04.045]10 weather and climate disaster events with damages exceeding $1 billion.
[00:01:09.291]Figure 1 shows the location of these disaster events that impacted the United
[00:01:13.312]States between January and June of 2020. Most, if not all of these events,
[00:01:18.734]occurred in rural areas.
[00:01:22.178]Digital reconnaissance is an emerging alternative to traditional reconnaissance
[00:01:26.251]that consists of internet-based data collection through social media,
[00:01:30.100]news articles, and local government webpages.
[00:01:33.403]However, digital reconnaissance is a time-intensive activity and subject to
[00:01:37.813]human bias. Therefore, the primary objective of this
[00:01:41.190]project is to evaluate the feasibility of an automated digital reconnaissance
[00:01:45.251]method in rural areas.
[00:01:48.566]To meet this objective, the following two phases were conducted.
[00:01:51.930]First, an image classification model was developed to identify key rural
[00:01:56.053]structural typologies. And second, an automated pipeline was
[00:02:01.532]developed to scour Twitter images in the aftermath of a disaster, classify those
[00:02:05.745]images for structural typologies, and generate statistics based on the
[00:02:09.601]damaged structures.
[00:02:13.898]Figure 2 here shows the flow of processes followed in order to meet the objective.
[00:02:18.171]Starting here, images of five rural structural typologies were
[00:02:22.755]compiled into a database.
[00:02:24.870]Figure 3 shows classifications and categories used to organize the images.
[00:02:29.278]The rural structures chosen were bridges, buildings, center pivot irrigation systems,
[00:02:34.166]or CPIs, houses, and silos. These structures were selected because
[00:02:39.815]silos, CPIs, and buildings, such as hoop barns
[00:02:42.381]and pole barns, are critical infrastructure in rural areas.
[00:02:47.007]Additionally, houses and bridges are typical structures seen in any area.
[00:02:51.648]Images of structures in damaged and undamaged states were collected because we
[00:02:55.512]only want the model to be able recognize and classify these images, not determine
[00:03:00.205]or provide any information on their structural state.
[00:03:05.843]Next, the image classification model was developed through a transfer learning
[00:03:09.404]approach, in which two existing pretrained image classification networks,
[00:03:13.470]AlexNet and InceptionV3, were trained to specialize in the five rural
[00:03:18.146]structural typologies.
[00:03:20.736]Transfer learning is a machine learning technique that applies knowledge from a
[00:03:23.706]source domain to a target domain. It is applied to convolutional neural network
[00:03:27.960]algorithms so that a user can modify the hyperparameters of a pretrained machine
[00:03:32.621]learning model and train it to classify any set of images.
[00:03:36.566]For this image classification model, 2,600 images were collected to train
[00:03:41.427]the image classification networks. 70% of the images were used for training,
[00:03:45.607]15% were used for validation, and the last 15% were used to test
[00:03:50.296]the final image classification model.
[00:03:53.924]Figure 4 here shows a confusion matrix generated by the final model
[00:03:58.234]after classifying the test images. The numbers in the last column represent
[00:04:02.716]precision metrics. Precision is the positive prediction rate and a perfect
[00:04:07.438]value of precision, or 100%, indicates that there were no false positives.
[00:04:12.650]The numbers in the last row represent recall metrics. Recall is the true
[00:04:17.025]positive rate, or sensitivity, and a perfect value of recall indicates
[00:04:21.761]that there were no false negatives.
[00:04:24.353]The number circled in blue is the overall accuracy of the model.
[00:04:27.673]When it classified the test images, it achieved an accuracy of 98.4%
[00:04:32.990]with the InceptionV3 network.
[00:04:37.665]Going back to Figure 2, once the trained image classification model was completed,
[00:04:42.185]it was used in a case study to classify aftermath images of the
[00:04:45.895]Tennessee Tornado Outbreak that occurred from March 2 to March 4 of 2020.
[00:04:51.725]A MATLAB code was developed to scour Twitter for Tweets containing images,
[00:04:55.558]pertaining to the search term “nashville tornado” or another city
[00:04:59.354]name that had been affected. Images tweeted within seven to ten days
[00:05:03.516]after the tornadoes hit were collected for classification.
[00:05:06.919]Once classified, the results were used to generate data about structural
[00:05:10.838]performance during the tornado event.
[00:05:14.401]The automated pipeline was evaluated by comparing reconnaissance data collected
[00:05:18.801]for the Tenneesse Tornado Outbreak that was gathered in three ways:
[00:05:22.279]method A, field reconnaissance, method B, manual digital reconnaissance,
[00:05:27.191]and method C, automated digital reconnaissance.
[00:05:31.264]The field reconnaissance data was provided by Dr. Richard Wood and Ms. Yijun Liao from
[00:05:35.864]the University of Nebraska - Lincoln and accessed from the Structural Extreme
[00:05:39.857]Events Reconnaissance database. The manual digital reconnaissance was
[00:05:44.121]collected by Arman Moussavi, an undergraduate student, also from UNL.
[00:05:49.438]Here Table 1 displays the number of damaged residential and manufactured homes
[00:05:53.702]accounted for by each reconnaissance method in the aftermath of the
[00:05:57.647]seven tornadoes. 151 damaged houses were counted with
[00:06:01.987]method A. 138 were counted with method B. And method C, the automated pipeline,
[00:06:07.846]identified 79 damaged houses.
[00:06:12.829]Some findings from the image classification model were that, though the
[00:06:16.199]model performed very well, it still displayed some confusion
[00:06:19.161]distinguishing and classifying buildings and houses.
[00:06:22.685]Figure 5 shows an example image of each structure that would cause confusion.
[00:06:27.329]Other sources of confusion were found to be due to the angle at which the image
[00:06:31.049]was taken, other objects in the image, such as debris, and the placement of
[00:06:35.176]the structure within the images frame. The last source of confusion could be a
[00:06:39.712]reason why the model would confuse houses for buildings, and vice versa.
[00:06:45.727]As for the case study results, the automated pipeline was able to
[00:06:49.197]identify damage to certain structural typologies but was not able to meet the
[00:06:53.602]quantities of the manual digital or field reconnaissance data.
[00:06:56.737]This can be explained by the model's inability to classify completely collapsed
[00:07:00.607]structures, bias in the images uploaded to Twitter, and image collages
[00:07:04.530]uploaded to Twitter.
[00:07:06.491]In the future, we hope to continue this research project by enhancing the image
[00:07:10.911]classification model to account for image collages, more severely damaged
[00:07:14.802]structures, and more structural classifications.
[00:07:17.931]Additionally, we want to include additional data sources in the automated
[00:07:21.631]pipeline beyond Twitter, such as other social media platforms,
[00:07:25.056]local news, and government websites. Another future step would be to expand the
[00:07:30.597]digital reconnaissance pipeline beyond images to include text analysis,
[00:07:34.389]such as news articles and Tweets.
[00:07:37.574]Lastly, this research was supported by the University of Nebraska and the
[00:07:41.474]National Science Foundation through an EEC grant.
[00:07:45.205]Thank you for taking the time to watch my poster presentation.

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Digital Post-Disaster Reconnaissance for Civil Infrastructure in Rural Areas Using Deep Learning

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments