Tim Guetterman: “Training and Assessing Communication Skills Using Virtual Human Technology: A Mixed Methods Investigation”

Nebraska Center for Research on Children, Families and Schools Author

04/05/2019 Added

9 Plays

Description

Spring 2019 Emerging Scholar Workshop Tim Guetterman Applied Research Methodologist University of Michigan This presentation describes a long-term, mixed methods investigation to explore, test and evaluate virtual human technology for its potential to enhance training and assessment of health communication skills.

Searchable Transcript

Search:

[00:00:00.021](light, upbeat music)
[00:00:09.830]So my focus is to share some of my empirical research
[00:00:13.700]that's certainly using a mixed methods approach.
[00:00:16.470]So I wanna give you a little bit of the story of kind of
[00:00:19.689]you know, where we're at today
[00:00:21.250]and then some ongoing work that I'm doing right now
[00:00:23.970]while weaving in mixed methods components
[00:00:26.200]because at my heart, I'm a methodologist,
[00:00:29.016]so that's really of utmost interest to me.
[00:00:32.240]When people ask me sort of to pigeonhole
[00:00:35.350]what discipline this work is in, it's really hard, actually.
[00:00:39.220]And I like this project because I think
[00:00:41.100]it demonstrates interdisciplinary work all within one study,
[00:00:45.020]if you look at our team.
[00:00:46.350]But really we're blending certainly education
[00:00:49.170]and educational psychology.
[00:00:51.460]Assessment has a big role in it.
[00:00:53.550]Communications has a big role in it.
[00:00:56.870]Information systems is also playing a major role in it,
[00:01:00.960]and yes of course, health as well.
[00:01:05.100]You know, before I go any further though,
[00:01:06.500]I want to acknowledge all of the collaborators
[00:01:08.700]on this project, Mike Fetters, Mas Jimbo,
[00:01:13.283]Pedja Klasnja, Larry An,
[00:01:15.940]Rich Gonzalez, expert in dyadic data analysis,
[00:01:19.830]all at the University of Michigan.
[00:01:21.870]Erika Rosenberg at UC Berkeley.
[00:01:25.120]She's really an expert in,
[00:01:27.010]and as I will share going forward, nonverbal communication
[00:01:30.330]and how we can assess and recognize that.
[00:01:32.760]Fred Kron, who's sort of on the technical side,
[00:01:35.710]beyond this technology,
[00:01:37.300]and then Mark Scerbo at Old Dominion University.
[00:01:40.020]So the project that I'm going to share
[00:01:41.810]is funded by a grant that I have
[00:01:43.480]through the National Institutes of Health,
[00:01:45.130]and it's actually funded
[00:01:46.330]by the National Library of Medicine,
[00:01:48.250]which is one of the institutes within NIH,
[00:01:50.750]and they actually tend to fund a lot of work
[00:01:52.960]around medical informatics.
[00:01:54.760]So this is what's called a career development award
[00:01:58.020]in biomedical informatics, in data science,
[00:02:01.110]which means that it has a strong research component.
[00:02:04.180]It's a research study,
[00:02:05.710]but it's also including a training component,
[00:02:08.790]so to sort of advance skills
[00:02:10.660]of more early-career scholars going forward.
[00:02:13.280]So I'll talk a little bit about this system,
[00:02:15.367]and it's called MPathic-VR,
[00:02:17.400]and how we're using this technology,
[00:02:20.210]this virtual human technology
[00:02:21.810]to advance communications training.
[00:02:24.220]I'll talk about the mixed methods randomized control trial
[00:02:27.260]that took place, and sort of where I'm picking up
[00:02:30.090]with secondary data analysis of what's been conducted there,
[00:02:33.810]and then moving on to some of the current research
[00:02:36.310]in progress of course.
[00:02:38.090]So communication is really, I would say,
[00:02:41.210]an under-researched and underappreciated,
[00:02:44.360]but really critical part of the healthcare system.
[00:02:48.170]And we have numerous pieces of evidence
[00:02:52.190]and empirical studies that show that
[00:02:54.150]poor communication in the healthcare setting
[00:02:57.140]is related to decreased satisfaction of patients,
[00:03:00.000]which you might expect,
[00:03:01.460]but that also ties to a number of other things,
[00:03:03.700]things like patient attrition.
[00:03:05.320]So a patient simply just going to another provider,
[00:03:08.750]or not actually following through with a plan
[00:03:12.360]to get them to be better over time.
[00:03:14.980]Poorer team functioning.
[00:03:16.380]So most healthcare teams,
[00:03:17.920]and this is really an important and growing interest
[00:03:21.300]in healthcare are interprofessional.
[00:03:23.470]So recognizing that it's not a single physician
[00:03:26.300]who's really in charge of this and running a study
[00:03:29.423]or a treatment plan,
[00:03:31.750]but multiple individuals who are working together.
[00:03:34.950]Actual poorer outcomes of patients
[00:03:37.560]that are linked to poor communication
[00:03:40.210]or communication that hasn't been thought through.
[00:03:42.260]And then things like harm, errors,
[00:03:44.530]malpractice, and lawsuits.
[00:03:46.470]So as you might imagine,
[00:03:47.930]this really garners the attention
[00:03:50.620]of physicians for instance.
[00:03:52.810]You know, thinking about poor communication
[00:03:54.940]could actually lead to you being sued, more often.
[00:03:57.490]And a lot of the work actually that has been done
[00:04:00.930]related to health communications
[00:04:02.810]has kind of taken that angle, actually,
[00:04:05.686]to you know, sort of motivate people to change.
[00:04:08.410]And I'll talk a little bit about that.
[00:04:10.210]So can the provider-patient relationship influence outcomes?
[00:04:15.040]And the short answer is yes.
[00:04:17.020]You know, there's ample evidence that it actually can
[00:04:20.381]in really a pretty substantial way.
[00:04:23.180]So how might it actually influence those outcomes?
[00:04:26.510]So we could break this into
[00:04:28.250]really a couple of different facets or aspects.
[00:04:31.170]There are things like cognitive care,
[00:04:33.880]and then also emotional care
[00:04:36.090]that providers are sort of imparting to patients.
[00:04:39.300]So on the cognitive aspects,
[00:04:41.410]this is sort of more of what you do
[00:04:43.260]at kind of a skills level.
[00:04:44.850]So things like gathering information,
[00:04:48.170]sharing medical information.
[00:04:50.170]So it's actually more of a partnership, if you will,
[00:04:53.330]with the patient, as opposed to more of
[00:04:55.610]a top-down approach going forward.
[00:04:58.730]Things like patient education,
[00:05:00.650]so imparting that knowledge as far as what to do.
[00:05:04.230]Instructing people at a variety of levels,
[00:05:07.160]from you know, how to actually
[00:05:09.370]administer a shot to yourself,
[00:05:11.360]to how to take medications appropriately, to you know,
[00:05:15.060]as we get into the focus of this project,
[00:05:17.240]actually complex care around cancer situations.
[00:05:21.630]And then of course expectation management
[00:05:23.810]comes into play here.
[00:05:25.370]So then that sort of connects to these emotional care items.
[00:05:28.840]So things like empathy and respect,
[00:05:31.780]trust, genuineness, acceptance, and warmth.
[00:05:34.470]So this emotional care is really the space
[00:05:37.910]that I'm operating in within the study,
[00:05:40.200]just to make that really clear.
[00:05:41.410]So that's really what we're trying to influence
[00:05:44.570]in order to ultimately improve outcomes.
[00:05:47.480]And the relationship,
[00:05:49.440]the patient-provider relationship has been linked
[00:05:51.610]to a number of different outcomes.
[00:05:52.880]So things that have been studied
[00:05:55.530]would be basic blood pressure management,
[00:05:57.830]there's ample evidence there.
[00:05:59.550]Pain control, quality of life measures as well.
[00:06:03.100]But the emotional care is
[00:06:05.610]where we're trying to really make a difference
[00:06:07.510]because that tends to be under-addressed
[00:06:10.800]really both in research but also in training
[00:06:14.270]for most providers, especially when we look at
[00:06:17.110]medical providers, so physicians for instance.
[00:06:20.220]So empathy is what I'll talk about a little bit more,
[00:06:23.480]and this is trying to impart empathy,
[00:06:26.490]so trying to understand and, you know,
[00:06:28.500]as though you were going through the situation yourself,
[00:06:31.280]and really use that as a guiding force
[00:06:33.470]throughout a conversation and a patient encounter.
[00:06:36.210]So communication training as it stands right now
[00:06:40.800]tends to be really under-addressed in medical education.
[00:06:44.410]So there might be a lecture for instance in medical school,
[00:06:48.820]and you know, historically that's literally what I mean.
[00:06:51.650]So in a four-year medical education curriculum,
[00:06:54.670]there might be a lecture on communication.
[00:06:56.880]And it tends to be more procedural, actually,
[00:07:00.410]so you know, here's a mnemonic device
[00:07:02.810]for thinking about how to communicate with patients.
[00:07:05.950]What's really needed though is more experiential learning
[00:07:09.590]and actually practice.
[00:07:11.450]So I sort of, you know, draw this analogy
[00:07:15.230]that right now it's a matter of teaching techniques.
[00:07:18.060]So it would sort of be like putting a student
[00:07:21.220]in front of a math book and you know,
[00:07:23.590]telling them what to do,
[00:07:24.830]but not actually having them work any math problems,
[00:07:28.150]or not actually having them practice
[00:07:30.070]conducting any math whatsoever.
[00:07:31.940]And that's really the state of education right now
[00:07:35.650]in terms of communication training.
[00:07:38.090]So another big aspect of this
[00:07:39.860]would be standardized patient instructors,
[00:07:42.300]and I'll talk about these now because it was how we assessed
[00:07:45.930]a lot of outcomes in this randomized control trial.
[00:07:48.840]So a standardized patient instructor you could think of
[00:07:51.530]as sort of a programmed actor, if you will.
[00:07:54.560]This is someone who undergoes a lot of training
[00:07:58.440]in order to maintain consistency,
[00:08:00.610]but they would sit down in front of say,
[00:08:03.280]a physician in training,
[00:08:05.510]could be a resident, a medical student,
[00:08:07.540]they're certainly applied to nurses,
[00:08:08.920]many other fields as well.
[00:08:10.610]And the standardized patient instructor
[00:08:13.300]is acting as if they were a patient with a real condition,
[00:08:16.960]so in our case it might be someone who has cancer,
[00:08:19.540]and they're presenting with this situation.
[00:08:22.740]And the standardized patient instructor
[00:08:25.440]is really trained to respond in certain ways
[00:08:28.400]based on how the conversation goes,
[00:08:30.440]and then they actually score.
[00:08:32.400]So they provide an assessment of the individual in training,
[00:08:36.330]the learner, and that comes in a variety of different forms.
[00:08:39.600]It could be simply a quantitative score.
[00:08:41.800]Ideally, they're actually providing some feedback
[00:08:44.400]as well going forward to that.
[00:08:47.650]So as far as the standardized patient instructors,
[00:08:50.270]this sounds really pretty great, right?
[00:08:52.540]You know, they're doing all of this kind of stuff.
[00:08:54.980]It's a realistic, authentic situation.
[00:08:59.560]We then look at some of the problems along with it though,
[00:09:02.440]and I'll talk about these a little bit more in a minute,
[00:09:04.810]but standardized patient instructors are really expensive
[00:09:08.760]because we have to make sure
[00:09:09.990]that we have people who are operating consistently
[00:09:13.270]when they're assessing
[00:09:14.640]and when they're actually providing this feedback.
[00:09:17.110]So to have 10 different individuals
[00:09:19.510]who are acting as standardized patient instructors
[00:09:21.940]around a common communication goal for instance,
[00:09:25.270]requires a lot of work to ensure consistency,
[00:09:28.100]reliability, and along with that,
[00:09:30.850]standardized patient instructors are people.
[00:09:33.200]So they fatigue, they get tired.
[00:09:36.070]And we often overlook that.
[00:09:38.260]So just to give you an example
[00:09:39.610]though of some of the costs that are associated with this,
[00:09:42.060]you know, a standardized patient instructor program
[00:09:44.270]can easily exceed over a million dollars a year
[00:09:46.980]for a medical school to implement.
[00:09:48.670]So there's substantial costs that are involved with this.
[00:09:53.300]But there's a lot of benefits to it.
[00:09:55.350]So this sort of leads to how
[00:09:57.180]we've really approached communication training,
[00:09:59.450]and we have a few different communication models
[00:10:02.040]that we're really drawing from
[00:10:03.460]to try to really train empathic communication skills,
[00:10:07.590]which is our goal.
[00:10:08.730]And one that we draw from pretty heavily is called SPIKES.
[00:10:12.830]And this is teaching how to break bad news
[00:10:15.870]in really a more empathetic sort of way.
[00:10:18.740]So SPIKES stands for setting up the interview,
[00:10:22.340]assessing the patient's perception of what's going on.
[00:10:25.630]The I is obtaining an invitation.
[00:10:28.040]So you want to ask, you know,
[00:10:29.400]would you like to know more about what's going on?
[00:10:31.610]for instance.
[00:10:32.690]The K is really about knowledge,
[00:10:34.910]so giving knowledge, imparting information.
[00:10:38.320]E, which is again, often overlooked
[00:10:41.500]in most of these sort of communication training modules,
[00:10:44.430]is really about the emotional response.
[00:10:46.610]So addressing and attending to the patient's emotions
[00:10:50.120]and not ignoring those because those are really critical
[00:10:53.260]in terms of all of those other implications
[00:10:55.760]of poor communication that I talked about earlier.
[00:10:58.060]And then finally, some sort of a strategy
[00:11:00.440]or summary of what's going forward.
[00:11:02.470]So I think that most of you, or a lot of you
[00:11:06.270]are probably from education, those who I recognize anyway
[00:11:11.210]so this seems pretty basic, actually.
[00:11:13.590]You know, this is probably what you would want to do
[00:11:15.480]to train someone.
[00:11:17.900]It's actually less apparent
[00:11:19.210]to a lot of medical professionals,
[00:11:21.010]that you know, we have sort of these different models
[00:11:23.750]that we can use to train communication
[00:11:25.963]and to actually do things like train empathy
[00:11:29.220]which I think we've demonstrated is possible.
[00:11:31.920]So onto this idea of really teaching empathy skills,
[00:11:35.680]so really trying to address that E
[00:11:37.750]while maintaining all of this cognitive component as well.
[00:11:41.070]So what we're doing here is trying to impart
[00:11:43.970]reflective listening skills, so you know,
[00:11:46.422]I hear you, I hear what you're saying.
[00:11:48.690]You know, basic things like paraphrasing what someone said,
[00:11:51.830]micro-encouragers.
[00:11:52.960]I see people nodding in the audience right now.
[00:11:55.060]So all of those things are important.
[00:11:57.410]Empathy enhancers, so again,
[00:11:59.050]letting people know that you really understand
[00:12:02.670]what they're saying, especially if someone maybe is upset,
[00:12:05.250]going through something.
[00:12:06.550]Avoiding empathy blockers,
[00:12:08.480]so simply shutting down if someone becomes upset
[00:12:12.030]and just shifting to another topic,
[00:12:13.700]but actually dealing with that in an important way.
[00:12:17.230]Appropriate use of facial expression,
[00:12:20.020]and this is really where a lot of my work has been
[00:12:23.040]in the past year and will be going forward.
[00:12:25.350]So this is things like, you know, eyebrow raises,
[00:12:28.540]smiles, nodding, leaning in, leaning out.
[00:12:31.830]I'll talk a little bit more
[00:12:33.429]about these facial expressions and nonverbals,
[00:12:35.780]but all of these are really important as well.
[00:12:39.390]And as we'll get to a little bit later in terms of nonverbal
[00:12:42.680]it's actually probably most of what
[00:12:45.250]is communicated in a conversation.
[00:12:48.980]So nonverbal communication,
[00:12:50.960]just to give you some statistics here.
[00:12:53.480]So Mehrabian and Ferris have written a lot
[00:12:56.280]about nonverbal communication,
[00:12:58.380]and they reported that only about
[00:13:00.274]7% of emotional communication is conveyed verbally,
[00:13:04.510]where 38% is conveyed through voice tone,
[00:13:07.470]and then 55% through facial expressions.
[00:13:11.090]So what that means is if you're breaking bad news
[00:13:13.460]to someone, so breaking a bad diagnosis,
[00:13:15.840]or something like that,
[00:13:17.180]what you say is actually less important than how it's said.
[00:13:22.490]And again, you know, most of that does not really
[00:13:25.040]tend to be addressed, at least clearly in training,
[00:13:29.480]in medical education for instance.
[00:13:31.380]So from a research side, this is an opportunity.
[00:13:34.520]It's an opportunity to intervene,
[00:13:36.100]and perhaps do something a little bit different.
[00:13:38.410]So that's where really our virtual human work comes in.
[00:13:42.130]So what are virtual humans?
[00:13:45.070]These are often defined as intelligent virtual agents.
[00:13:49.990]What that really means is that it sort of looks
[00:13:52.660]like a pretty high-quality video game-style character.
[00:13:57.000]So it's someone who looks realistic.
[00:13:59.370]They're talking, their face moves.
[00:14:01.450]I'll show you pictures of these as well.
[00:14:03.430]But you can have really a conversation
[00:14:05.860]with this virtual human.
[00:14:08.100]So it's moving from teaching someone about math conceptually
[00:14:12.580]to actually having them work out problems.
[00:14:14.670]So using the virtual human,
[00:14:16.400]we can have people actually practice communication
[00:14:20.470]in a safe setting.
[00:14:21.760]You know, it's not with an actual patient,
[00:14:23.640]someone who could be harmed, experience some outcomes,
[00:14:26.810]but they try to simulate human behavior
[00:14:28.930]and really appearance through technology.
[00:14:32.600]And they're designed with this capability
[00:14:34.760]really to promote interaction.
[00:14:36.860]So it's really a two-way conversation
[00:14:39.620]that we're attempting to mimic.
[00:14:41.400]It's a computer of course,
[00:14:42.970]so some of it is a little bit forced,
[00:14:44.690]but it's really giving a chance
[00:14:46.280]for someone to practice talking,
[00:14:48.130]and then the computer responds in a certain way.
[00:14:50.730]So as far as why virtual humans,
[00:14:54.120]I'll go back to a little bit of what I talked about
[00:14:56.580]with the standardized patient instructors.
[00:14:59.250]Relative to that, virtual humans tend to be cost-effective.
[00:15:02.910]Now, developing this technology is not cheap,
[00:15:05.810]though the cost is coming down quite substantially actually.
[00:15:09.610]But compared to maintaining a virtual human program,
[00:15:12.830]it's substantially more cost-effective.
[00:15:15.660]It also has a lot of advantages in terms of reliability.
[00:15:18.760]So a virtual human could have hundreds of conversations
[00:15:22.020]over and over and you know,
[00:15:24.113]this is what happened in the trial,
[00:15:25.470]you can literally bring 200 people through a conversation
[00:15:28.620]with virtual humans on different machines,
[00:15:31.370]and the virtual human never gets tired or fatigued,
[00:15:34.660]so there are certainly some advantages there.
[00:15:37.410]And it is interactive.
[00:15:39.000]So it is moving from, say watching a video,
[00:15:42.190]or just simply hearing a lecture on communication
[00:15:44.840]to trying to actually experience what it's like
[00:15:47.730]to break bad news, and then receive feedback on that.
[00:15:50.950]And then also we thought, we hoped,
[00:15:54.280]that it would perhaps lead to
[00:15:55.610]some enhanced motivation to learn.
[00:15:58.250]So just moving back to, and it's really from, you know,
[00:16:01.340]who I am and my background, you know,
[00:16:03.340]things like the universal learning model, and you know,
[00:16:05.960]what we can do to actually encourage
[00:16:08.160]that this is not just a one-time thing,
[00:16:09.880]but people are interested in
[00:16:11.370]and actually changing their behavior
[00:16:13.470]and communicating differently.
[00:16:15.560]We really felt that and found that individuals
[00:16:18.560]tended to be more motivated to learn doing this.
[00:16:21.770]As far as why, it could be just because it sounds cool.
[00:16:25.280]You know, it's a virtual human.
[00:16:27.476](audience laughing) I get it, it sound sci-fi,
[00:16:29.410]all that kinda stuff.
[00:16:30.690]But people tended to like it, and that's pretty good.
[00:16:33.380]So I'll get into a little bit more
[00:16:35.670]about what this looks like.
[00:16:38.060]So it started really with some initial grants through,
[00:16:41.370]that Fred Kron and Mike Fetters developed,
[00:16:44.360]and these were really grants
[00:16:46.190]to develop the technology over time,
[00:16:48.950]and those really kind of built
[00:16:50.520]to this randomized control trial
[00:16:52.410]that I'll talk about a little bit.
[00:16:53.700]So I'll refer to MPathic as this system,
[00:16:57.340]but it stands for this acronym, so it's this acronym,
[00:17:00.060]Modeling Professional Attitudes
[00:17:01.730]and Teaching Humanistic Communication in VR.
[00:17:05.440]I love the University of Michigan.
[00:17:08.130]They're great people there,
[00:17:09.510]and they're obsessed with acronyms for everything.
[00:17:13.900]As I talk about my work,
[00:17:15.050]all you'll hear about is acronyms for projects.
[00:17:17.270]So the goal here was to really think about
[00:17:19.410]how this new media technology can be used to,
[00:17:23.500]at its initial part, just to develop a prototype
[00:17:26.430]for training how to break bad news to individuals
[00:17:29.900]in this one-on-one setting with a virtual patient.
[00:17:32.600]So this is just a screenshot,
[00:17:34.590]and I'll show you a little bit more of what the user
[00:17:37.770]might see when interacting with a virtual human.
[00:17:41.650]But what happens is that the virtual, the learner,
[00:17:45.180]I'm sorry, sits down with a computer,
[00:17:48.470]and they're in front of really, this widescreen monitor,
[00:17:51.730]as you see up here.
[00:17:53.310]So there are a few different things.
[00:17:55.300]On the left side, this is Robin is her name.
[00:17:58.750]So she's one of the virtual humans
[00:18:00.230]we have programmed into this program.
[00:18:02.210]So this is someone who presents
[00:18:05.160]really with problems in the emergency department.
[00:18:09.750]And you run some tests,
[00:18:11.370]and now it's your job to really disseminate
[00:18:13.600]what the results of those tests are.
[00:18:15.540]So the person on the right here is the learner, actually.
[00:18:19.000]So we go through some different teaching points initially
[00:18:22.440]as someone sits down with the system,
[00:18:24.410]so it is giving them some instruction on communication,
[00:18:28.410]things like avoiding those empathy blockers,
[00:18:30.710]things like attending to nonverbal communication as well.
[00:18:34.390]So as far as the nonverbal communication,
[00:18:36.170]that's actually not ignored in this system,
[00:18:38.230]and in fact, this little, black box on top
[00:18:40.720]is a Microsoft Kinect sensor.
[00:18:42.980]So we use the Microsoft Kinect sensor
[00:18:44.960]to collect nonverbal data, essentially.
[00:18:47.880]So at the time that this technology came out,
[00:18:51.350]what it could really do is things like detect proximity,
[00:18:55.070]so leaning in, leaning out.
[00:18:57.210]It could detect eyebrow raises.
[00:18:59.210]It could detect smiles, and it puts you through
[00:19:02.430]sort of this training algorithm
[00:19:04.070]where it'll tell you to smile,
[00:19:05.700]and then it circles your smile on the screen,
[00:19:08.010]so to make sure that it recognizes it.
[00:19:10.760]So with that though, we actually have collected
[00:19:13.280]a ton of data on those nonverbals
[00:19:16.230]that each learner was demonstrating
[00:19:18.290]and attempting to demonstrate through that,
[00:19:20.420]and then we also have a camera of each of these interactions
[00:19:23.820]with the virtual human as they go forward.
[00:19:26.680]So as far as the experience,
[00:19:29.200]someone is sitting down in front of the systems.
[00:19:31.810]They put on headphones with a microphone,
[00:19:34.490]and the microphone is so that the system
[00:19:37.170]can actually detect speech,
[00:19:39.190]and let me share in a minute
[00:19:41.070]a little bit more about what that is,
[00:19:43.350]but they click on the MPathic icon,
[00:19:45.150]they select their gender,
[00:19:46.340]and this is really for a voice recognition profile,
[00:19:48.860]to ensure that the system is recognizing their voice.
[00:19:51.420]And then for each scenario,
[00:19:54.330]and we have a number of different interchanges,
[00:19:56.710]so there might be 16 different interchanges
[00:19:59.830]that someone goes through.
[00:20:01.580]The virtual human will essentially speak and say something,
[00:20:06.200]and then right now what we do
[00:20:07.460]is we present three potential response options.
[00:20:10.880]It kind of works like a computerized adaptive test,
[00:20:14.070]if you will.
[00:20:14.903]And the learner can either click on
[00:20:16.960]what response option they feel is appropriate,
[00:20:19.390]or they can just speak in plain language
[00:20:21.480]what the response option is,
[00:20:23.200]and it will detect it.
[00:20:24.380]So the options include sort of a bad response,
[00:20:28.990]an unempathic response, something that's better,
[00:20:31.640]and then the best.
[00:20:32.830]And then we're assigning point values
[00:20:35.010]to each of those responses.
[00:20:37.360]So then the other half of it is that the system is adaptive
[00:20:41.930]in the sense that if someone selects a suboptimal response,
[00:20:46.700]the conversation will escalate.
[00:20:48.730]It will become worse, if you will.
[00:20:51.010]Where otherwise, it might go better,
[00:20:54.010]or it might stay a little bit more neutral.
[00:20:57.690]So what happens is that Robin is presenting.
[00:21:00.760]She has an unstoppable nose bleed
[00:21:02.660]in the emergency department.
[00:21:04.580]Her labs demonstrate that she has a severe form of leukemia,
[00:21:08.530]and the player, the learner,
[00:21:09.960]has to disclose to her that she has cancer.
[00:21:12.660]Essentially, she has leukemia.
[00:21:14.730]So she flares with anger and disbelief at this point.
[00:21:18.740]And what you're doing as a learner is you're going through
[00:21:22.580]a number of different interchanges with Robin,
[00:21:25.980]but you're trying to empathically communicate
[00:21:30.230]this very bad news to her
[00:21:32.440]while supporting her throughout the process.
[00:21:34.920]And the way the system is structured is that,
[00:21:37.790]if someone does well,
[00:21:38.850]they might make it through all of those,
[00:21:40.490]let's say 16 different exchanges.
[00:21:42.920]But it could also escalate to the point where she's so upset
[00:21:46.620]she wants to just check herself out of the hospital,
[00:21:49.230]and the system will really just kick you out at that point.
[00:21:53.230]Yeah.
[00:21:54.090]So we try to, though, make this a learning sort of system,
[00:22:00.570]in terms of giving people feedback
[00:22:02.700]as they're going through this.
[00:22:04.000]So after going through a full interchange
[00:22:07.300]with Robin, for instance,
[00:22:09.620]the system then provides automated feedback
[00:22:12.760]to the learner based on what they did.
[00:22:15.180]So you know how I said that
[00:22:16.470]for all those three response options,
[00:22:18.160]we're sort of recording a value, a point value with those,
[00:22:21.920]so based on that, it's disseminating to them
[00:22:24.690]what they can do differently to improve their communication.
[00:22:28.330]And then they repeat the scenario again.
[00:22:30.830]So they'll have,
[00:22:31.663]it's really the exact same scenario with Robin.
[00:22:35.820]So at this point,
[00:22:36.940]we have really two different scenarios
[00:22:41.600]that we then put into a randomized control trial.
[00:22:44.550]The initial one was simply breaking bad news.
[00:22:47.380]In the trial though, we had two different scenarios.
[00:22:50.160]So the one was breaking bad news
[00:22:52.560]that included an intercultural communication component.
[00:22:55.560]So this is Robin, as you see right here on the left,
[00:22:58.760]and her mother on the right.
[00:23:00.960]So Robin's mother is from El Salvador,
[00:23:03.960]and she has some very strong beliefs
[00:23:06.400]about being involved in her daughter's care.
[00:23:10.010]There's actually a whole backstory
[00:23:12.270]to all of these characters.
[00:23:14.120]The person I mentioned, Fred Kron,
[00:23:16.570]he's actually trained as a physician
[00:23:19.160]so he's behind a lot of this technology,
[00:23:21.970]but he's actually a member
[00:23:23.070]of the Screen Writer's Guild in Hollywood,
[00:23:25.620]so there's actually a story behind all of these scenarios
[00:23:29.140]including, you know, Robin's personal history
[00:23:31.530]and all of that sort of stuff.
[00:23:32.860]But it's to try to make this realistic
[00:23:35.040]and as authentic as it could be
[00:23:36.850]as you're interacting with a virtual human.
[00:23:38.640]And then there's another scenario to try to train
[00:23:43.083]interprofessional communication skills.
[00:23:45.610]And this is really around tension
[00:23:47.820]between the patient and provider.
[00:23:49.830]So it builds, so someone will go through this,
[00:23:52.210]breaking bad news and intercultural communication scenario,
[00:23:55.870]receive feedback, and then they repeat it again,
[00:23:58.970]hopefully did better.
[00:24:00.340]As a preview, the data show that they do actually improve
[00:24:03.420]from time to time. (audience laughing)
[00:24:04.790]And they go to this other scenario,
[00:24:06.660]and this is Nurse Fishetti,
[00:24:08.610]so she's a third virtual human in the program,
[00:24:11.790]and she's upset because she wasn't involved
[00:24:15.230]in this earlier conversation,
[00:24:17.090]and had you come to Nurse Fishetti ahead of time,
[00:24:20.580]you would've learned some really important
[00:24:22.740]background details.
[00:24:24.290]The nurses are often the people
[00:24:25.860]who are really closest to the patient's care.
[00:24:28.550]They have sort of their finger on the pulse
[00:24:30.400]of what's going on, and really,
[00:24:32.660]she's upset and communicating that to you.
[00:24:35.830]So through these two scenarios,
[00:24:37.620]it's trying to hit breaking bad news,
[00:24:40.270]intercultural communication,
[00:24:42.060]and then also interprofessional education skills as well.
[00:24:46.890]So that was the MPathic condition in this trial.
[00:24:49.820]We also had a computer-based control module,
[00:24:53.280]and this is really trying to mimic training
[00:24:56.830]as it is currently, as far as communication.
[00:24:59.990]So it was going through these different scenarios
[00:25:02.220]about standardized communication.
[00:25:04.050]So there's one model, it's called SBAR,
[00:25:07.030]where you're communicating the Situation,
[00:25:09.090]Background and Assessment, and then Recommendation.
[00:25:12.730]So it was really teaching different techniques,
[00:25:14.760]and this is really you know, what happens more typically.
[00:25:17.960]So individuals were randomized.
[00:25:21.210]There were a total of 435 who were randomized.
[00:25:25.710]A few did not continue, but at the end,
[00:25:28.380]it was a fairly balanced design in the trial
[00:25:30.870]where we had 210 in the MPathic condition,
[00:25:35.130]and then we had 205 in the control condition
[00:25:38.510]going through this computer-based module.
[00:25:40.910]So the learners were all medical students,
[00:25:43.890]and they were at three different universities,
[00:25:46.440]really throughout the United States.
[00:25:50.126]And they all agreed to participate in this.
[00:25:53.170]It was really sort of a standardized
[00:25:54.630]part of their curriculum going forward.
[00:25:57.940]So as you might imagine,
[00:25:59.810]we had mixed methods components embedded into this.
[00:26:03.000]So I'll talk a little bit the design for this trial
[00:26:05.650]and then also move into some of what we found
[00:26:09.240]and some of what I think is really pretty promising.
[00:26:12.350]So we had a quantitative attitudinal scale score.
[00:26:15.970]So this is really asking about individuals'
[00:26:18.700]reactions to the training, motivation to learn,
[00:26:22.250]what was helpful, what was not helpful.
[00:26:24.330]It was a fairly short scale that people completed
[00:26:28.360]that was entirely quantitative.
[00:26:30.290]We also had qualitative written reflections.
[00:26:34.020]So this was every student completed an exit reflection,
[00:26:38.270]whether they were in the MPathic condition
[00:26:40.490]or the control group, and it presented individuals
[00:26:44.260]with different questions really
[00:26:45.520]to react on the learning experience, if you will.
[00:26:48.590]And these are pretty detailed responses,
[00:26:51.000]designed to be pretty detailed, so we had you know,
[00:26:53.860]roughly about a paragraph essentially from each student.
[00:26:56.610]So that was a major source of qualitative data,
[00:26:58.990]and then we also had observations of proctors,
[00:27:02.190]individuals in the room as well.
[00:27:04.840]We also have this empathic game score,
[00:27:07.390]so going through each interchange,
[00:27:09.640]again it's recording how people did.
[00:27:12.620]And then we had this,
[00:27:13.750]it's called an Objective Structured Clinical Examination.
[00:27:16.930]So this was the standardized patient instructor,
[00:27:20.210]and this was done about three days to one week
[00:27:25.950]after completing either the control
[00:27:28.690]or the MPathic condition.
[00:27:31.140]They went to this examination
[00:27:33.970]with a standardized patient instructor.
[00:27:36.070]The standardized patient instructor was blinded
[00:27:39.140]to the condition that each learner was in, for instance,
[00:27:42.840]and they were all trained to have
[00:27:45.070]really a pretty similar scenario,
[00:27:47.580]and it was actually a novel scenario.
[00:27:49.840]So it was something that was completely different
[00:27:52.290]from the breaking bad news, intercultural,
[00:27:54.680]or interprofessional.
[00:27:56.320]It actually drew on interprofessional components,
[00:27:58.680]but it was something that was completely different.
[00:28:01.090]So that was really our primary outcome, essentially,
[00:28:04.170]in the randomized control trial.
[00:28:06.340]We then had video recordings of each interaction as well.
[00:28:10.010]So in that Kinect sensor we also had a camera embedded
[00:28:13.680]so we could see, for instance, if the sensor,
[00:28:17.050]and this is sort of where I come in later on,
[00:28:19.810]was actually detecting nonverbal behavior,
[00:28:22.650]you know, in an appropriate way.
[00:28:24.020]And then we have all of this Kinect sensor data.
[00:28:26.510]So this is again, more of a quantitative data source here.
[00:28:30.160]So some of the results.
[00:28:32.580]So we found quantitatively that the MPathic score,
[00:28:35.440]this is the score that the computer is generating,
[00:28:39.460]so we only had this for the MPathic condition, of course
[00:28:42.250]because otherwise people didn't participate in it,
[00:28:45.230]but their pre-post score did improve.
[00:28:47.330]That was statistically significant.
[00:28:50.080]So for the intercultural scenario, there were 16 exchanges.
[00:28:54.020]They were scored on zero to 29 points in total,
[00:28:58.090]and a lower score was actually better,
[00:29:00.350]so they were essentially penalized
[00:29:02.170]if they had a suboptimal response.
[00:29:04.250]For the interprofessional scenario,
[00:29:05.840]there were 13 exchanges.
[00:29:08.130]Total score here ranged from zero to 25, but again,
[00:29:11.390]the pre-post different was statistically significant.
[00:29:14.210]So another quantitative outcome here was this OSCE score,
[00:29:19.080]this Objective Structured Clinical Exam.
[00:29:21.580]So a composite score, this OSCE was broken down
[00:29:24.950]into four different domains, actually,
[00:29:27.210]but we compiled all of them into a composite score,
[00:29:30.370]and again found that there was
[00:29:31.610]a statistically significant difference
[00:29:34.300]in the MPathic group relative to the control group.
[00:29:37.400]So that provided pretty strong evidence to us
[00:29:41.100]through this randomized control trial
[00:29:43.160]that the MPathic condition
[00:29:45.230]was improving communication scores,
[00:29:47.500]and that was detectable as well.
[00:29:50.470]So the standardized patient instructor
[00:29:52.360]was evaluating each student's performance
[00:29:55.070]really on four different domains.
[00:29:57.010]Openness/defensiveness, collaborative/competitiveness,
[00:30:01.060]nonverbal communication, and then presence.
[00:30:03.640]This is sort of more of a global awareness
[00:30:06.080]of others' feelings measure.
[00:30:08.050]And we found a pretty good reliability
[00:30:11.370]actually within this measure as well.
[00:30:14.430]So moving on to another quantitative measure,
[00:30:16.840]this was this attitudinal scale score.
[00:30:19.630]We also found a statistically significant difference here,
[00:30:23.220]and the MPathic students tended to respond more positively.
[00:30:26.510]So we had 12 different items on this Likert type score,
[00:30:29.950]really across four domains,
[00:30:31.970]so there was the clarity of the training.
[00:30:34.483]And the attitudinal score, I should mention,
[00:30:36.960]this is again something that both the control
[00:30:39.300]and the MPathic condition participated in.
[00:30:42.330]So clarity, purpose, utility,
[00:30:44.370]and then the likelihood to recommend
[00:30:45.980]the training experience to others.
[00:30:48.010]We again had a pretty strong reliability
[00:30:50.570]for this scale as well.
[00:30:52.480]So again it's suggesting to us that, you know,
[00:30:55.190]not only is there some, quote-unquote hard evidence
[00:30:58.730]that communication skills improve,
[00:31:00.860]but people seemed to like it as well.
[00:31:04.330]But can we use qualitative data
[00:31:06.220]to understand how they liked it, in what way they liked it,
[00:31:09.280]or things that they didn't like?
[00:31:10.840]And there were certainly things that people didn't like.
[00:31:13.290]So qualitatively we identified really five different themes
[00:31:17.700]throughout this data,
[00:31:18.770]so this is analyzing the qualitative written reflections,
[00:31:22.570]and we had really these three different prompts
[00:31:24.720]that individuals responded to, so reflecting
[00:31:27.460]on how the learning experience could be improved,
[00:31:30.920]the three most important things you learned from it,
[00:31:33.970]and then how interacting really influenced
[00:31:36.180]your understanding of nonverbal communication specifically.
[00:31:39.380]So we found qualitative themes about the verbal,
[00:31:43.140]the nonverbal communication aspects,
[00:31:45.780]this idea of engagement in the training,
[00:31:47.880]so motivation to essentially want to learn more,
[00:31:51.420]or really, on the other hand,
[00:31:52.900]a lack of interest in learning more.
[00:31:55.490]A need for supplemental training,
[00:31:57.260]and that's really important because that's sort of,
[00:32:00.460]you know, this important motivational hurdle that yes,
[00:32:04.570]I actually am starting to recognize
[00:32:06.420]that communication is important,
[00:32:08.390]and it might be a skill that I want
[00:32:09.800]to continue to develop long-term.
[00:32:12.060]So you know, that to me was very positive.
[00:32:14.960]And then, the importance of immediate feedback.
[00:32:17.720]Because remember, as people go through
[00:32:19.620]each of these different scenarios,
[00:32:21.890]they receive automated feedback right then and right there.
[00:32:25.210]You know, we're talking seconds after they've completed it,
[00:32:28.110]so there isn't a long delay.
[00:32:29.490]They're not waiting for a written report
[00:32:31.470]to come out a couple of months later,
[00:32:33.397]and we think that's really important.
[00:32:35.220]So a little bit more about the experience,
[00:32:38.470]and this is really kind of a,
[00:32:39.870]it's hard to read so I'll just talk through it,
[00:32:41.780]but a joint display of our mixed methods findings.
[00:32:44.870]So we looked at these four different domains.
[00:32:47.980]I'm just showing an excerpt of it,
[00:32:49.570]but in terms of verbal communication, nonverbal,
[00:32:52.890]the engagement of the training,
[00:32:54.610]what did we learn quantitatively from our attitudinal scale,
[00:32:58.780]qualitatively through our qualitative,
[00:33:01.950]illustrative quotes from written reflections,
[00:33:05.020]from the intervention relative to the control group,
[00:33:08.530]and then sort of putting all that together,
[00:33:10.390]what did we learn?
[00:33:11.380]So I'll focus kind of on this last column, and I'm sorry,
[00:33:15.980]more of what we learned,
[00:33:17.360]and that was our mixed methods results.
[00:33:19.300]So in terms of our verbal communication, the intervention,
[00:33:22.760]our comments really suggested this deeper understanding.
[00:33:26.170]So it wasn't, you know, surfacey,
[00:33:28.400]understanding these different techniques, mnemonic devices,
[00:33:31.880]but really a higher and more nuanced understanding.
[00:33:36.570]So things like understanding that cultural background
[00:33:39.340]is important when talking to individuals,
[00:33:41.420]and you need to account for that.
[00:33:42.780]And that difference was confirmed
[00:33:44.440]by the quantitative scores as well.
[00:33:46.680]In terms of nonverbal, so again,
[00:33:50.406]we heard comments from the intervention group
[00:33:53.310]about nonverbal, really none from the control group
[00:33:56.090]about nonverbal, and that difference was confirmed
[00:33:58.870]by the attitudinal scores that were significantly higher.
[00:34:01.690]And then in terms of the engagement of training.
[00:34:04.640]So that was really reflected through comments about this.
[00:34:07.900]It was called an after-action review,
[00:34:09.600]providing this automated feedback.
[00:34:12.450]And really, it suggested in the control group,
[00:34:15.840]a need for more interaction.
[00:34:17.810]So people in the control condition
[00:34:19.720]were wanting to practice more, they felt that was lacking,
[00:34:22.850]and that was really reflected quantitatively as well.
[00:34:26.160]So another way to think about it
[00:34:28.790]is looking at kind of our low, medium,
[00:34:32.030]and higher-performing individuals who went through this,
[00:34:35.300]and how did they talk about the system differently?
[00:34:38.590]So I won't go through all of this, for instance,
[00:34:41.480]but these were some major qualitative themes that arose,
[00:34:46.430]so useful communication skills, remembering nonverbals,
[00:34:50.480]motivation, and then also to the extreme,
[00:34:54.040]doubting nonverbals as well.
[00:34:55.830]So it's really interesting though
[00:34:57.090]if you look at some of the patterns that we found here.
[00:34:59.810]So just to focus on a couple here,
[00:35:03.020]so for this theme of useful communication skills,
[00:35:06.010]so really seeing it as useful,
[00:35:08.420]the individuals who performed worse actually,
[00:35:11.490]at the end of this,
[00:35:12.670]they didn't really talk about it that way.
[00:35:15.180]Where it was actually the medium
[00:35:16.570]and higher-performing individuals,
[00:35:18.690]they really tended to see the value in it.
[00:35:20.870]So you know, that suggests to us
[00:35:22.710]that in future iterations of this system,
[00:35:25.280]we need to make sure that we're not
[00:35:27.110]losing individuals going through this.
[00:35:30.270]Things like the nonverbals.
[00:35:31.780]So it was really interesting.
[00:35:33.800]The lower-performing individuals
[00:35:35.810]tended to sort of dismiss communication.
[00:35:38.570]I already know how to communicate.
[00:35:40.070]That was something that, you know, people wrote in this.
[00:35:42.050]I don't need to know this.
[00:35:43.800]The nonverbals they thought, you know,
[00:35:45.730]was really, you know, kind of silly.
[00:35:49.000]Why would I need to actually look at a computer or smile,
[00:35:52.900]that sort of thing.
[00:35:54.270]Well, it's actually really important,
[00:35:55.700]if we go back to some of that original data.
[00:35:58.040]But again, you know, that's suggesting to us that,
[00:36:00.270]you know, how can we tailor this differently
[00:36:02.130]to maybe address this group of learners
[00:36:05.320]in the future iteration of our system.
[00:36:07.150]So putting that all together,
[00:36:09.750]we think we have pretty good evidence of effectiveness.
[00:36:13.070]So this is, you know, more of a real-world trial.
[00:36:16.090]This is actually in medical schools.
[00:36:18.890]Scores improved, and we found this retention of skills
[00:36:22.010]really about a week later.
[00:36:23.630]You know, a week is not a long time, granted.
[00:36:25.560]But, you know, it's better than 30 seconds later, I guess.
[00:36:28.818](audience laughing)
[00:36:29.900]The interactive learning was preferred.
[00:36:32.630]This repeating of the scenario,
[00:36:34.650]so not just going through it once,
[00:36:36.160]but going through it, receiving feedback,
[00:36:38.020]going through it again, actually yielded improvement,
[00:36:41.040]and then the use of virtual humans
[00:36:43.050]really allowed more consistency.
[00:36:45.470]So I was also then interested in thinking about
[00:36:49.190]virtual humans from an assessment perspective.
[00:36:51.930]So how can we use them to assess communication skills?
[00:36:54.870]So just simply assess where people are at.
[00:36:57.840]So this was an earlier study done,
[00:37:01.130]kind of a prequel, if you will, that I conducted,
[00:37:03.850]looking at this breaking bad news scenario,
[00:37:06.520]and we had individuals separated into different groups
[00:37:09.530]to try to gather initial construct validity evidence.
[00:37:12.210]So group A received,
[00:37:15.340]they participated in this MPathic assessment,
[00:37:18.250]we'll call it at that point.
[00:37:19.550]They then received training,
[00:37:20.567]and this was sort of a standardized training
[00:37:22.830]in breaking bad news,
[00:37:24.210]and then they had a post-test score.
[00:37:26.760]So you would hope that the system was detecting improvement
[00:37:30.930]as people went through training.
[00:37:32.500]So this is just kind of a standard way
[00:37:34.630]to gather validity evidence.
[00:37:36.440]Group B because we wanted to account
[00:37:38.610]for pre-test sensitization,
[00:37:40.930]this actually did not have the pre-test.
[00:37:43.760]It went just straight to the seminar,
[00:37:45.520]and then we had a post-test
[00:37:47.140]so we could look at it from different angles,
[00:37:48.970]look at the pre-post difference,
[00:37:50.600]but then also the simple post-test scores.
[00:37:53.000]Were those similar or statistically
[00:37:55.010]significantly different as well?
[00:37:56.860]So through that, we found that there was really
[00:37:59.940]minimal evidence of pre-test sensitization,
[00:38:02.700]so that indicated to us that there is
[00:38:04.430]perhaps use for this tool
[00:38:05.850]in assessing communication skills at a broader level.
[00:38:10.230]The virtual human detected
[00:38:12.090]pre-post seminar differences as well,
[00:38:15.230]and then a post-test seminar comparison
[00:38:18.040]was not significantly different, which is again,
[00:38:21.240]you know, what we hoped essentially,
[00:38:23.520]that there would be really no difference
[00:38:25.600]between group A and group B in this.
[00:38:27.510]So this is another paper that's published
[00:38:30.390]as a result of this study.
[00:38:31.700]So the evidence is kind of mounting at that point,
[00:38:34.370]and this leads to some of my current work
[00:38:36.290]that I'll talk about in the next couple of minutes actually.
[00:38:39.170]So this is the, it's called a K grant that's funded.
[00:38:43.020]So what I'm trying to do is first of all,
[00:38:45.670]use some of the data from the trial,
[00:38:48.030]and try to understand the nonverbal aspects better.
[00:38:51.410]So we have all of this data
[00:38:52.670]that was collected from the nonverbal sensors
[00:38:55.210]and also from the video interchanges, so I'm curious,
[00:38:58.750]you know, how accurate was that Kinect sensor
[00:39:01.480]in detecting what was going on, and then,
[00:39:04.110]is the nonverbal behavior actually connected
[00:39:06.820]to outcomes as well?
[00:39:08.170]So as we see people demonstrate
[00:39:10.370]and practice nonverbal behavior,
[00:39:12.490]are we seeing that reflected then in that post-test score?
[00:39:15.980]Aim two then is really to use grounded theory methods
[00:39:19.520]and develop a new model of nonverbal communication
[00:39:23.600]that would inform future virtual human programs
[00:39:26.840]because most communication models right now
[00:39:29.160]actually don't account for nonverbal skills at all.
[00:39:32.550]Aim three then is to program that technology
[00:39:35.560]and really conduct a quality control check
[00:39:39.710]to see how well we can detect nonverbal behavior
[00:39:45.020]using an improved and refined assessment
[00:39:47.260]of nonverbal behavior.
[00:39:48.760]So a conceptual model that I'm using for this is,
[00:39:51.900]well, one model is human factors
[00:39:54.230]in the virtual learning environment.
[00:39:56.060]So we think about the learner,
[00:39:58.030]kind of at the center of this,
[00:40:00.710]and then how they're interacting.
[00:40:02.040]So of course we don't want them
[00:40:03.340]to be adversely affected by the virtual human.
[00:40:06.370]What communication tasks
[00:40:08.200]are really suitable to virtual human training?
[00:40:12.250]Are there certain learner characteristics
[00:40:14.160]that affect outcomes?
[00:40:16.830]You know, are there things like where someone is from,
[00:40:21.550]for instance, or if they're coming in
[00:40:23.390]with sort of lower academic performance,
[00:40:26.710]they may be more or less likely to change.
[00:40:29.980]Of course, the adverse effects, will there be any?
[00:40:33.410]What virtual human design elements
[00:40:35.300]would enhance performance?
[00:40:36.800]Limitations, how much feedback can the learner process?
[00:40:41.030]So that after-action review is important,
[00:40:43.080]but we don't want to overwhelm people with too much.
[00:40:46.470]And then, what sort of technology improvements are needed?
[00:40:49.170]And this is really at the heart of the study, if you will.
[00:40:52.170]So current conceptual models of communication,
[00:40:56.290]and this is an oversimplification of the actual models
[00:41:00.170]of these core functions of patient-provider communication,
[00:41:05.220]have things like health outcomes at the center.
[00:41:08.730]Responding to emotions is part of it,
[00:41:11.330]exchanging information,
[00:41:12.820]managing uncertainty and making decisions,
[00:41:14.840]but you actually don't see anything
[00:41:16.250]about nonverbal aspects reflected in any of these.
[00:41:19.780]So part of my goal is to think about
[00:41:21.390]how we can have a better model of communication
[00:41:24.320]that actually is including and accounting for
[00:41:26.860]some of those nonverbal aspects
[00:41:28.970]in a little bit more of a structured way.
[00:41:31.940]So for aim one, again, I'm trying to understand
[00:41:35.310]this nonverbal aspect a little bit better.
[00:41:38.160]So some of the data sources
[00:41:39.710]are un-analyzed data from the trial.
[00:41:42.410]We have the video recordings,
[00:41:44.100]these MPathic scores that were collected,
[00:41:46.860]this huge warehouse of nonverbal data,
[00:41:49.470]and then of course their performance scores.
[00:41:52.570]So analyzing that is a little bit challenging,
[00:41:56.310]as you might imagine.
[00:41:57.630]So if we think about how to actually code a video exchange.
[00:42:03.260]So that's arguably both quantitative and qualitative
[00:42:06.930]type of data, as far as how we're looking at it.
[00:42:09.670]The other complication that comes into this
[00:42:11.960]is that it's really a dyad,
[00:42:13.840]so we're sort of violating statistical assumptions
[00:42:17.150]that we might normally be able to use
[00:42:19.320]because it is this interchange
[00:42:20.780]with two different individuals.
[00:42:22.580]We have variability as far as
[00:42:24.540]how the virtual human is responding
[00:42:26.760]and how the patient is responding, if you will.
[00:42:30.180]So can we model that a little bit better?
[00:42:32.690]We do have some systems like the Facial Action Coding System
[00:42:36.210]and other systems that we can use
[00:42:37.990]to quantitatively assess this video.
[00:42:41.200]And what I'm interested in doing
[00:42:42.760]is really testing this relationship
[00:42:44.740]and developing structural equation models
[00:42:47.280]and of course, merging that with the qualitative data
[00:42:50.060]to try to understand mechanisms in a little bit more detail.
[00:42:54.160]So as far as how we're coding this,
[00:42:56.654]that's something I'm working on
[00:42:57.730]with research assistants right now.
[00:43:00.320]There's something called the Systematic Review
[00:43:02.370]of Nonverbal Communication and Patient Satisfaction.
[00:43:07.090]And really what they found, this is a work of Stephen Henry,
[00:43:10.340]is that more global measures of nonverbal behavior
[00:43:13.440]tend to be more reliable than sort of
[00:43:15.580]counting specific instances of what happened.
[00:43:18.910]So I think it's going to be really important
[00:43:21.620]to have more of a global measure,
[00:43:23.350]so that's something that we're analyzing
[00:43:25.100]as we're looking at each video.
[00:43:26.860]So there's less evidence for things like body language,
[00:43:29.560]facial expression, in part because
[00:43:31.920]it's really difficult to research,
[00:43:34.330]and there's a lot of problems with this existing research
[00:43:37.260]that have found not really a strong correlation.
[00:43:40.500]So things like outcomes,
[00:43:42.390]associations between outcomes and facial expressions
[00:43:46.210]are really susceptible to this mutual influence,
[00:43:48.930]so it's really not accounting for the fact
[00:43:50.900]that this is a dyad, so that's problematic in this research,
[00:43:54.270]and I think it could be explaining
[00:43:56.080]why there isn't a lot of evidence.
[00:43:58.380]So another tool that I'm using
[00:44:00.300]is something called this Facial Action Coding System
[00:44:04.020]to try to see if nonverbal communication
[00:44:06.860]is really acting as a mediator,
[00:44:09.100]and then can we also automate feedback
[00:44:11.770]about nonverbal skills?
[00:44:13.110]So this is just a little excerpt of this coding system,
[00:44:17.950]but it's essentially looking at tiny, little movements
[00:44:21.500]in someone's face, and it's a very detailed system
[00:44:24.860]to reliably code those movements
[00:44:28.090]which are connected to various emotions, if you will.
[00:44:31.830]So there's implications for not only assessing
[00:44:34.870]and reading someone's nonverbal behavior,
[00:44:37.290]but perhaps, and this is my hope,
[00:44:39.210]we might actually train providers
[00:44:41.960]to account for and better read
[00:44:44.280]nonverbal behaviors and nonverbal communication.
[00:44:47.870]So that takes us to this next step,
[00:44:50.440]which is really going to be a grounded theory study
[00:44:52.820]that'll be starting in a few months,
[00:44:54.970]really interviewing providers
[00:44:57.860]about their perspectives of nonverbal communication
[00:45:01.040]and integrating that with data learned from aim one
[00:45:04.460]to try to develop really a better model
[00:45:07.220]of nonverbal communication, and then of course,
[00:45:10.270]to program that into the technology and actually assess it.
[00:45:14.310]So some future directions where I want to go,
[00:45:18.770]and we've already started to go would be
[00:45:20.350]handling more difficult conversations.
[00:45:22.830]So we recently submitted a pretty large R01 grant,
[00:45:26.700]that I'm a co-investigator on,
[00:45:28.610]around transitions to palliative care,
[00:45:30.910]and how to have these conversations
[00:45:32.870]for individuals undergoing cancer care.
[00:45:35.770]Longer-term change.
[00:45:37.160]So the trial looked at about a week after
[00:45:39.410]going through this, but in fact,
[00:45:41.610]in the study that went in, the grant that went in,
[00:45:44.740]we were looking at longer-term change.
[00:45:46.720]So is this reliably demonstrated at three months,
[00:45:50.580]six months, perhaps longer on?
[00:45:52.470]We don't want to get too far out
[00:45:53.880]just because it becomes harder to detect that.
[00:45:56.300]And then, can we actually connect this sort of training,
[00:45:59.920]going through this,
[00:46:01.110]with hard patient outcomes and patient satisfaction?
[00:46:05.990]In the cancer setting,
[00:46:07.640]there's actually a number of existing measures
[00:46:09.770]that a lot of clinics are assessing already.
[00:46:12.390]So we're interested in seeing if, by going through this,
[00:46:16.370]training providers on having
[00:46:18.020]better communications empathically,
[00:46:20.980]just to give you an example
[00:46:21.950]around transitions to palliative care,
[00:46:24.840]are we seeing improvement in outcomes?
[00:46:27.210]So for example, people aren't transitioning
[00:46:29.910]to palliative care in the last week of their life,
[00:46:32.820]but it's something that's come up much sooner, for instance.
[00:46:35.760]So that's something that's problematic in that arena.
[00:46:38.800]And then of course,
[00:46:39.633]bringing in patient satisfaction measures as well.
[00:46:43.080]I think there's a number of applications for this,
[00:46:45.860]training difficult conversations in general.
[00:46:48.890]So my hope is to someday write a grant
[00:46:52.450]to use this type of technology to train first responders,
[00:46:56.500]perhaps police officers.
[00:46:58.440]You know, maybe we could reduce
[00:46:59.670]some of the negative incidents that we see
[00:47:02.630]if people have more training
[00:47:03.990]on how to have empathic interactions with individuals.
[00:47:08.570]Providing training for the counseling world, of course.
[00:47:12.090]Focusing on nonverbal cues is really largely untouched
[00:47:15.610]in communication training as well.
[00:47:18.340]Flipping it the other way, actually,
[00:47:20.640]and using the system to train patients
[00:47:23.470]how to talk with their providers better.
[00:47:26.200]To be honest, I think this is going to have
[00:47:28.570]much more of a large impact in the long run
[00:47:31.760]than training providers.
[00:47:33.240]Because there are a lot more patients out there,
[00:47:35.780]and there are some people who really feel
[00:47:37.640]like the best way to change,
[00:47:39.410]say physician behavior or provider behavior,
[00:47:42.410]is really through their patients who are coming to them.
[00:47:45.430]And then simply, just thinking about how we can use this
[00:47:48.450]to better understand two-way communication that's happening.
[00:47:52.510]So that's a little bit of the work that I've been doing,
[00:47:55.510]one of my main grants and where I spend a lot of my time.
[00:47:59.280]I'd be happy to answer any questions at this point.
[00:48:02.760](audience applauding)

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Tim Guetterman: “Training and Assessing Communication Skills Using Virtual Human Technology: A Mixed Methods Investigation”

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments