Comparing Eye-Tracking Measures of Beginner & Experienced UAV Pilots

Kristen Hallas Author

07/28/2021 Added

50 Plays

Description

Previous aviation studies show that the eye-tracking measures of pilots have demonstrative ability to predict their expertise, yet not much is known about the measures of unmanned aerial vehicle (UAV) pilots. To expand this area of knowledge, we designed a sequence of flight tasks for UAV pilots to complete that increased in complexity over time. We recruited one beginner and one experienced UAV pilot to complete these tasks indoors while wearing a Pupil Core headset that collected real-time gaze and pupil positions. Preliminary results explore differences in the gaze pattern of each pilot throughout the tasks, as well as differences in pupil diameter and blink behavior.

Searchable Transcript

Search:

[00:00:00.000]Howdy! My name is Kristen, I go to the University of Texas Rio Grande Valley, I'm an undergrad and I am also a part of the University of Nebraska Lincoln Summer Research Program. Woo!
[00:00:11.120]So I was in the Applied Unmanned Systems group, I worked in the NIMBUS lab with Grace and Alisha and Brittany and I also worked with the eye-tracking lab alongside Cole and Bonita.
[00:00:22.230]And we did all of this because we wanted to investigate if there were any differences in the eye tracking measures of beginner and experienced UAV pilots - so UAV is just an abbreviation for
[00:00:36.330]an Unmanned Aerial Vehicle. So what motivated our study? Let's just fly into it
[00:00:42.960]In the field of aviation. there have been numerous studies done in eye tracking to be able to identify - hey, is the person flying right now, a novice, or an expert pilot?
[00:00:55.050]And this has been used to advanced training materials for pilots, so that people like you can be flown across the country and arrive safely
[00:01:03.510]If you flew here to Nebraska and you're listening to my presentation, well you've benefited from this research! so, then this lab - here I am woo :') lol
[00:01:12.240]In the NIMBUS lab we wanted to extend this area of research and investigate, What can be applied in the field of aviation to perhaps unmanned aerial vehicles?
[00:01:23.130]So we developed some research questions, based on what we've seen in the literature for aviation. We wanted to see if the same concepts could apply to UAV pilots.
[00:01:33.009]So first, we wanted to look at pupil size - is there a difference between a "beginner" pupil size and "experienced" pupil size?
[00:01:41.009]We also wanted to look at blinks - so, how long are beginner pilots blinking? How many times do experienced pilots blink and for how long?
[00:01:50.509]And we also wanted to look at - so let's say an area of interest (AOI) right now for me i'm wearing my eye tracking goggles - headset - lol
[00:01:59.500]It would be this camera and i'm looking right at this camera so my eyes, right now, are fixated on that camera we wanted to see if there's any area of interest that pilots could be fixating on more than others.
[00:02:11.009]So we designed an experiment! And I'm going to talk to you about some numbers today - first the number is 2! Two is the number of participants, we recruited for our "pilot" study - haha, you see what I did there?
[00:02:24.000]Three? Well that's the number of areas that we are testing for proficiency - we wanted to look at 1) linearity 2) curvature and 3) yaw control to see how well our pilots could fare.
[00:02:36.500]And four? Well that stands for our quadcopters that we had our beginner and experience UAV pilot fly inside. So we designed a series of tasks and we asked our participants to fly them and repeat them anywhere from 30 to 60 seconds.
[00:02:55.560]So I want to show you some pictures as well, this is me hi! I am wearing the Pupil Core, which was developed by Pupil Labs, and this allows for tracking my eyes in an indoor environment.
[00:03:09.340]This is Grace, hi Grace! So Grace is standing here with our calibration marker. We calibrated the eyes of our participants. using this marker right here with a feature called "natural features" calibration. So Grace walked around the cage, held the ladder at different heights, and we asked our participants to focus right here on that center point.
[00:03:31.280]This is our quadcopter woo! It's really fun to fly, like actually you feel a gust of wind blow right at you!
[00:03:38.829]And these are our flight path models. So we have left and right, up and down, a square, a U-shape, a circle shape, a figure-8 and I'll talk more about the yaw control component in a bit here.
[00:03:50.210]But first, woo! So we had eight trials in total, we recorded the data in Pupil Capture, we exported it in Pupil Player and we were able to get the pupils sizes pretty automatically and clean the data in Microsoft Excel.
[00:04:05.430]But the blanks and fixations were a bit more of a manual process, and I'll get into that in a minute here.
[00:04:11.590]First, we can look at this table to see some of the more quantitative data, so we have our beginner in the legend of yellow and our experienced in the legend of blue.
[00:04:21.990]And you can see the task descriptions here on the left the yaw control component is basically you're just rotating while you're in the air and you're flying in the direction that you've rotated
[00:04:33.770]so like let's say rotated 180 degrees. Well your right becomes a left, and your up becomes a down, and you have to maintain stability! So we saw this as a pretty difficult task that we tested for at the end.
[00:04:46.070]Because we wanted to see if there are any profound changes in the number of blinks or the pupil size.
[00:04:51.380]And we did see that for our experienced pilot that their pupil size seem to increase as the tasks progressed over time. We also saw that our experienced pilot did not blink - at all - it was pretty intense.
[00:05:04.370]We watched the data, a couple of times to make sure that was absolutely the case. And so we saw our beginner pilot blink a bit more. Now..
[00:05:12.380]Whether or not these numbers are average or whatnot will probably need more data for, but these preliminary results are promising for our lab.
[00:05:20.950]Next we'll talk a bit more about some qualitative data, so this is all about the fixations that we saw during the tasks conducted.
[00:05:28.620]So, we tested for linearity, curvature, and yaw control - so that's shown in the inner, middle, and outer circles respectively.
[00:05:36.180]And we saw that for the beginner they tended to fixate more on the path throughout the trials, whereas the experienced pilot fixated more on the drone throughout the trials.
[00:05:46.490]Other - well we'll talk about one really interesting example for other, but that includes anything from "it's not on the path" or "not on the drone", "it's off the screen", "it's looking at the controller" so that's kind of our umbrella category here.
[00:06:00.760]So with this preliminary study we do know that both of our pilots had corrective vision, so this could have impacted our data.
[00:06:09.030]Also, what could have impacted our data is the manual data entry because imagine you're me and you're sitting there and you're keeping your eyes open like this going through thousands of frames, making sure that you don't blink at the same time as the pilot.
[00:06:22.250]so of course humans make mistakes, and we hope that that's not the case here. But that is a potential room for error.
[00:06:31.510]We did see the beginner pilot have more jerky eye movements. So imagine I'm here, and I'm fixating on the camera, and all of a sudden there's a fly over there! And my eyes jerk from the camera to that fly.
[00:06:42.070]That's called a saccade, we saw the beginner had a lot more of those and, in addition to that, our beginner fixated a lot more on the center of the given flight path shape.
[00:06:50.880]It's a lot easier to explain with a picture, so let me show you that now
[00:06:54.560]This is our beginner flying the UAV, this is the 54th fixation made and they're flying the U-shape.
[00:07:01.860]So let me just trace the U shape there for you and you can see that the fixation is on the path, but not on the drone itself.
[00:07:09.330]But let's go to the immediate next fixation. So we see here the drone it's in its same configuration and it's in that U shape, but the fixation is actually off of the path, but more towards the Center of the U itself
[00:07:23.690]If you drew a piece of - On a piece of paper, if you drew a U and you marked the center of it, that's right about where this fixation would fall.
[00:07:30.820]And we noticed that a lot, not only for this trial, but a lot of the trials from the beginner and that looks really promising for our future work
[00:07:38.010]So just to sum it up, we ran a pilot study lol
[00:07:40.190]And our preliminary results show that there could be differences between beginner and experienced UAV pilots
[00:07:45.670]So of course in the future right, we want to look at more data, more subjects and, of course, trying to automate our data with computers, because that's what they're for to make our lives easier right
[00:07:54.680]And hopefully, in the future we will have a way to predict expertise with UAV pilots that will improve the training methods so that UAV flying is more accessible for all!
[00:08:06.880]So thanks so much for listening! I just want to acknowledge the NSF, University of Nebraska Lincoln, and the NIMBUS lab - I couldn't have done it without y'all - here are those references I talked about
[00:08:16.070]And thanks so much for listening! I'll now accept questions, V's up!

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Comparing Eye-Tracking Measures of Beginner & Experienced UAV Pilots

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments