WS1-ID Summit Workshop: Proctoring Limitations and Alternative Assessments
ID Summit Workshop: Proctoring Limitations and Alternative Assessments
ID Summit: As colleges and universities shifted to remote learning in response to COVID-19, several institutions grappled with how to ensure academic integrity and assessment. Many turned to third-party proctoring services for assistance in ensuring academic integrity. However, the widespread use of these services exposed critical issues related to privacy, inclusion, and legality. This workshop highlights some of these national issues and shares how University of Nebraska campus faculty members and instructional designers advocated for and implemented alternative assessment strategies to counter these national issues.
icon search Searchable Transcript
Toggle between list and paragraph view.
[00:00:03.010]`- [Michael] The ID summit each year,
[00:00:05.220]instructional designers from the system
[00:00:08.160]get together to discuss relevant and timely issues
[00:00:11.350]and instructional design
[00:00:13.230]to share ideas and collaborate on solutions
[00:00:15.920]to pedagogical challenges.
[00:00:18.250]This year, we've decided to open an ID summit
[00:00:23.340]to all participants of the symposium
[00:00:26.090]as the first time that we've had
[00:00:28.340]the ID summit open to everyone
[00:00:32.410]who is participating or who registered for the summit.
[00:00:36.130]So welcome to those of you
[00:00:38.520]who are not instructional designers.
[00:00:41.427]We hope that this event is fruitful for you.
[00:00:48.080]I want to talk a little bit just about what we intend to do
[00:00:52.170]in today's workshop.
[00:00:53.310]We've got three different sections
[00:00:56.090]we're gonna talk about.
[00:00:59.360]I'll give you kind of the synopsis
[00:01:01.270]as colleges and universities
[00:01:02.831]kind of shifted to remote learning
[00:01:05.440]in response to the pandemic.
[00:01:07.840]A lot of our institutions grappled
[00:01:10.470]with how to ensure academic integrity
[00:01:12.960]and how to handle assessments.
[00:01:16.530]Many institutions turned to third-party proctoring solutions
[00:01:22.270]for assistance and ensuring that academic integrity,
[00:01:24.964]however, the widespread use of these services
[00:01:28.950]kind of exposed some critical issues
[00:01:31.040]related to privacy, inclusion
[00:01:33.160]and in some cases the legality,
[00:01:36.410]this workshop was gonna highlight
[00:01:38.190]some of these national issues.
[00:01:40.510]We're gonna share how a university of Nebraska campuses
[00:01:44.770]and faculty members and instructional designers
[00:01:48.150]advocated for an implemented alternative assessments
[00:01:52.610]and identify what some of those strategies are.
[00:01:55.880]We have two faculty guests
[00:01:59.670]who are going to talk about
[00:02:02.140]what they've done in their courses
[00:02:04.600]and kind of highlights some of the solutions
[00:02:08.270]that they've implemented.
[00:02:09.400]So we've got some good time
[00:02:10.900]for them to showcase what's been done in their course
[00:02:14.805]as well as answer questions of the group.
[00:02:18.513](Michael clears throat)
[00:02:19.680]We're going to have a couple of instructional designers
[00:02:22.190]kind of share what's been done with the campuses
[00:02:25.450]across our university system.
[00:02:27.730]And then we've got some resources to share with you as well.
[00:02:32.210]Those will most likely get shared
[00:02:34.741]at the end of the presentation.
[00:02:38.440]And then we'll also make the slide deck available
[00:02:42.850]on the symposium agenda
[00:02:46.863]after the symposium is over
[00:02:49.510]after we've gathered all of this information.
[00:02:54.101]So what I'd like to do
[00:02:55.140]is I'm gonna go ahead
[00:02:57.420]and we have some of the goals.
[00:02:59.580]As I mentioned, we wanna kind of understand
[00:03:01.630]some of those national issues that are related
[00:03:03.710]to the use of third-party proctoring services.
[00:03:06.330]Want to hear how colleagues have modified some assessments
[00:03:09.680]and then identify some of those strategies.
[00:03:12.930]Just to give you a little bit of housekeeping.
[00:03:16.587]My name's Michael Jolley.
[00:03:18.290]I'm the assistant director
[00:03:20.711]for the university of Nebraska online.
[00:03:22.850]I'll be moderating the session.
[00:03:25.700]We also have some tech support
[00:03:29.380]in the form of Nate Morris, who is one of our co-hosts.
[00:03:35.700]If there are technical issues
[00:03:37.610]please feel free to mention them in chat.
[00:03:41.220]And we'll try to get those
[00:03:42.630]Natal will try to get those resolved during the session.
[00:03:46.910]I know that we're going to have time for Q and a.
[00:03:50.230]And so we'll try to manage that as best that we can.
[00:03:54.880]You can either ask questions and chat,
[00:03:57.584]or I think if we've got a good block of time near the end
[00:04:00.880]we might just open it up for people
[00:04:02.330]to be able to kind of share their screen
[00:04:04.640]or unmute and ask them question that way
[00:04:08.150]so we can have some discussion in that format.
[00:04:13.840]So the first thing that I kind of wanted to talk about
[00:04:18.670]and unfortunately, a couple of our colleagues
[00:04:20.430]are not able to make it this morning
[00:04:23.101]Kristen and Michael had another commitment this morning
[00:04:28.200]that has prevented them from joining.
[00:04:30.510]So I'm gonna try to-
[00:04:31.920]I'm gonna try to cover for them as best that I can.
[00:04:35.210]So when we talk about third-party proctoring solutions
[00:04:39.210]we're talking about those solutions,
[00:04:42.616]using tools such as respondentsq monitor
[00:04:45.980]or Proctorial or Proctor U
[00:04:48.765]things of that nature,
[00:04:50.390]where students have to sign up to use the service,
[00:04:55.400]the services, using video technology
[00:04:59.320]to take a look at the place
[00:05:00.880]in which they're taking the assessment.
[00:05:02.550]And there are fees
[00:05:04.160]and in some cases associated with these services
[00:05:08.051]and the widespread use
[00:05:10.810]of these third-party proctoring services
[00:05:12.940]has led to some challenges nationally
[00:05:18.081]in the last year or so.
[00:05:20.230]We've seen a few different news articles
[00:05:23.710]that have come out
[00:05:24.920]that have talked about proctoring solutions
[00:05:29.020]whether they are truly helping
[00:05:32.220]with academic integrity issues.
[00:05:33.920]There've been student privacy concerns.
[00:05:37.540]There have been institutions
[00:05:40.158]that have decided to reject remote proctoring altogether
[00:05:45.723]how this is becoming a larger issue
[00:05:48.885]as students complain about privacy issues,
[00:05:53.833]as they complain about equity issues.
[00:05:57.610]So this is the national challenges
[00:06:00.210]associated with third-party proctoring solutions
[00:06:05.930]and we have some several different resources
[00:06:09.099]that I think are available for you
[00:06:12.830]in our resources in this particular slide
[00:06:15.210]qdeck to take a look at if you're interested,
[00:06:18.520]we know that this has been an proctoring
[00:06:21.420]that third party proctoring has exploded during this time
[00:06:24.299]but we really wanted to take the opportunity
[00:06:27.559]from a pedagogical standpoint to examine
[00:06:31.084]what was being done on our campuses
[00:06:34.388]and how the use of alternate assessment solutions
[00:06:39.710]may be a better way forward.
[00:06:42.560]So what I'd like to do now
[00:06:44.180]is I'd like to introduce a couple of our colleagues,
[00:06:48.742]Jullian Manza from university of Nebraska
[00:06:51.330]Lincoln and Steve McGann
[00:06:52.790]from UNK to talk about just a few of the different ways
[00:06:59.049]that our campuses responded
[00:07:02.791]to the pandemic from a proctoring standpoint
[00:07:06.870]from an assessment standpoint
[00:07:08.800]and highlight just a few of those particular issues
[00:07:12.300]and challenges that our campuses are facing.
[00:07:17.360]So with that, I'm gonna turn it over to Jillian and Steve.
[00:07:23.520]So I'm gonna go ahead and start.
[00:07:25.083]Julian has another thing
[00:07:26.540]she's going to talk about afterwards,
[00:07:28.130]but we want to start off by saying
[00:07:29.800]that none of us are saying
[00:07:31.500]that proctoring is not a solution that can be employed ever.
[00:07:35.240]One of the things that were one of the challenges
[00:07:38.200]that we've faced over the last year or so
[00:07:40.578]with COVID is the swell in proctoring
[00:07:43.920]and how much more people are relying on proctoring
[00:07:47.290]as a solution for academic integrity
[00:07:50.120]and there are issues that come along with that.
[00:07:53.760]I'm going to talk about two
[00:07:55.370]Jullian's going to talk about one,
[00:07:56.490]but the first is at UNK.
[00:07:58.870]And I know at other, at UNL
[00:08:03.010]one of the issues that pops up constantly
[00:08:05.140]is the expense of proctoring
[00:08:07.240]and whether it's institutionally funded
[00:08:09.770]or whether it's student funded
[00:08:12.600]there is a large expense
[00:08:15.160]that is associated with any proctoring
[00:08:17.990]from 10 to $15 an hour for a live Proctor
[00:08:21.083]to somewhere around two to $3 a test
[00:08:25.688]for some of the automated systems.
[00:08:29.502]And if the university takes that on,
[00:08:32.330]obviously as a strain on our financial resources,
[00:08:34.610]but more importantly if we require the students
[00:08:36.840]to do some of that
[00:08:38.330]test Proctor and can easily total up
[00:08:40.290]to be about the same as what a textbook would cost.
[00:08:43.930]And so balancing those issues
[00:08:46.190]is one of the things that we're really looking at
[00:08:48.415]across the universities
[00:08:50.256]of how we balance what we're expecting the students to do
[00:08:55.547]and how much proctoring is actually necessary.
[00:08:58.860]And that may not just be the proctoring service itself.
[00:09:02.430]It may also include equipment that the students have to buy.
[00:09:05.570]It may include time
[00:09:06.830]that the faculty have to spend supporting it.
[00:09:08.900]The institution has to spend supporting it.
[00:09:11.050]So there are a lot of different things
[00:09:12.150]to go along with that.
[00:09:13.472]But one of the things I wanted to talk about in here
[00:09:16.360]is actually something that doesn't apply specifically
[00:09:19.210]to the proctoring side of thing
[00:09:20.730]but it is something that is very germane
[00:09:24.980]to what we're talking about.
[00:09:25.947]And that is the issue of rigor
[00:09:28.566]or whether or not we're asking students
[00:09:31.610]to be rigorous in their online
[00:09:33.180]in their courses,
[00:09:34.140]not just online, sorry, I work in online
[00:09:36.310]but also whether are we having them do busy work or crunch.
[00:09:43.620]And one of the things that we see
[00:09:45.400]is there's a lot of adding of things on
[00:09:49.205]you add pieces in all the time,
[00:09:52.270]but is it really necessary?
[00:09:54.320]And one of the presentations coming up
[00:09:56.740]they're going to talk about, you know
[00:09:58.390]kind of pulling away from some of that.
[00:10:01.235]But when we work with faculty,
[00:10:04.025]one of the things that we try to talk about
[00:10:06.462]is finding the pieces that you absolutely need to assess
[00:10:11.540]and not doing things like double assessing materials.
[00:10:14.260]Do you need to do both a paper and a test
[00:10:17.410]to get those materials
[00:10:18.800]to properly assess those materials?
[00:10:21.160]Or can we back off of the tests
[00:10:22.820]as the tests tend to be a little bit more of an issue,
[00:10:26.603]finding those ways to authentically assess
[00:10:30.345]is really what the instructional designers
[00:10:32.720]at all the campuses
[00:10:34.150]have been trying to do.
[00:10:35.510]And the cracks in the system
[00:10:37.340]have been exposed over the last year
[00:10:39.000]that have made us really come in and focus more on that.
[00:10:41.690]And with that, I'm going to stop.
[00:10:43.580]Cause I only had a couple of minutes to talk
[00:10:44.990]and I'm sure it went a little over
[00:10:46.080]and turn it over to Jillian
[00:10:50.293]Yeah. Thanks Steve.
[00:10:51.590]So as he kind of mentioned we know that the coronavirus
[00:10:54.610]certainly didn't start the online proctoring trend
[00:10:57.540]but what we do know is it certainly exacerbated it.
[00:11:00.420]And we, along with many other institutions
[00:11:03.610]across the country,
[00:11:04.740]quickly pivoted last spring
[00:11:06.300]and various proctoring technologies were deployed.
[00:11:09.160]They were deployed swiftly
[00:11:10.340]and unforeseen equity issues
[00:11:12.790]really rose to the surface and very quickly.
[00:11:15.340]And so that's a little bit
[00:11:16.200]of what I'm going to be covering today.
[00:11:18.490]So I don't have time to go into detail on these issues
[00:11:21.390]but really the key takeaway here
[00:11:22.893]is that remote proctoring technologies
[00:11:25.410]can disproportionately hurt
[00:11:27.120]and disadvantage our marginalized students.
[00:11:29.543]In particular, the instances I'm going to list
[00:11:32.310]are students of lower economic status and students of color.
[00:11:35.740]So as we prepare for return to campus this fall,
[00:11:39.204]we know that the volume of remote proctoring
[00:11:42.410]is likely to decrease.
[00:11:43.820]However, the equity issues associated
[00:11:46.230]with remote proctoring for exams
[00:11:47.898]are still going to exist.
[00:11:49.730]So I'm going to list out some of the examples
[00:11:52.660]that have been reported in multiple articles
[00:11:54.840]across the country throughout the past year,
[00:11:57.420]just to kind of give you a feel
[00:11:58.670]of what some of those equity issues are
[00:12:00.560]that have been experienced particularly
[00:12:02.416]by our already marginalized students.
[00:12:05.530]So some examples reports from students
[00:12:08.173]where they talked about instances
[00:12:11.550]where facial recognition software failed
[00:12:14.240]to identify students of color
[00:12:15.731]or those wearing religious garb
[00:12:17.830]And in some of those cases,
[00:12:19.250]students were locked out of tests
[00:12:20.901]because they were unable
[00:12:22.390]to be recognized by the face and gaze detection software.
[00:12:25.927]In other reports, students with disabilities
[00:12:28.727]notice that they were inaccurately flagged
[00:12:31.470]for possible academic integrity violations
[00:12:34.890]due to their involuntary movements like muscle spasms
[00:12:37.547]others reported significant mental health concerns.
[00:12:41.159]Students stated that they had an increase in anxiety
[00:12:44.910]due to the discomfort of having either an unknown Proctor
[00:12:48.040]often referred to as a stranger
[00:12:49.839]or just the AI technology monitoring their movements
[00:12:53.339]specifically trying to discern
[00:12:56.027]whether they're cheating or not.
[00:12:58.960]And often students stated
[00:13:00.260]that this heightened anxiety and over monitoring
[00:13:03.040]really negatively impacted their mental health
[00:13:05.900]which then of course led to a decrease or decline
[00:13:08.970]in their performance on an exam or assessment
[00:13:11.990]there's cost as Steve mentioned,
[00:13:13.540]creating a significant issue of inequity
[00:13:15.750]for students in many ways
[00:13:16.960]whether they're picking up the cost
[00:13:18.270]for paying for that proctoring service
[00:13:21.020]or having to pay for an expensive textbook
[00:13:23.971]that has the cost of that proctoring fee
[00:13:27.040]embedded into its cost.
[00:13:29.000]Additionally, some other cost related issues
[00:13:31.728]relating to having a computer
[00:13:33.436]that meets the technological requirements
[00:13:36.350]necessary to support proctoring service.
[00:13:38.840]This can be really challenging for many students to attain.
[00:13:41.612]Also many households
[00:13:44.020]are what are considered one device households.
[00:13:46.500]So having multiple devices might not be possible
[00:13:49.130]which limits students access to that computer
[00:13:51.930]or that technology
[00:13:53.300]on that has the specifications needed for that exam taking,
[00:13:57.027]having access to continuous high speed internet connectivity
[00:14:00.947]is a requirement for students to be able to take the exams.
[00:14:05.903]I know in my own household
[00:14:07.410]sometimes that's even a little bit challenging.
[00:14:09.900]And so unfortunately, to go elsewhere
[00:14:12.188]and find alternative high-speed internet is challenging.
[00:14:15.885]We know that free and public wifi is certainly not an option
[00:14:19.970]because it can't support the proctoring service.
[00:14:22.675]Rural test takers here in Nebraska also
[00:14:24.874]have found this to be extremely challenging
[00:14:27.630]to meet those technological requirements.
[00:14:30.300]Lastly, location is certainly another issue
[00:14:33.920]in terms of the requirement
[00:14:35.359]that says things like you need a quiet, well lit space
[00:14:39.250]that might not be easily accessible for all of our students
[00:14:41.890]to have this quiet well-lit space.
[00:14:43.950]Students with children in particular mothers
[00:14:45.621]are certainly impacted by this.
[00:14:48.604]Or if you have a busy household,
[00:14:50.300]you might find it challenging to have uninterrupted minutes
[00:14:52.664]to take exams.
[00:14:54.190]And then you are flagged
[00:14:55.351]for environmental noises in the room,
[00:14:58.330]or if anyone or anything comes into your camera's view.
[00:15:01.700]So environmental scans are really challenging
[00:15:04.172]for those tests taking environments.
[00:15:06.080]We know they're common
[00:15:07.500]but students consider them to be extremely intrusive.
[00:15:10.080]And in many cases a live Proctor is joining that test taker.
[00:15:14.336]And so there's an element there
[00:15:16.655]of environmental or visual overshooting
[00:15:19.460]that occurs where they're seeing
[00:15:20.610]a lot of someone's environment
[00:15:22.400]that may not be a very comforting experience
[00:15:25.220]for that test taker.
[00:15:26.650]So that's a long laundry list
[00:15:28.967]of some of the issues of inequity
[00:15:31.318]that are being experienced by our students.
[00:15:33.550]Like I said, again,our low socioeconomic status students
[00:15:36.079]and students of color and many others.
[00:15:38.402]So as I said, while the volume and usage
[00:15:41.840]we know is likely to go down
[00:15:43.274]as many of us return to campus
[00:15:45.011]remote proctoring will still exist.
[00:15:47.880]So I encourage you to think about your assessments
[00:15:50.740]and we encourage you
[00:15:51.880]and that's what we're about to talk about today.
[00:15:53.593]If you can explore some alternative assessments
[00:15:55.833]and perhaps move away from exams
[00:15:58.670]and things that require that remote proctoring service
[00:16:04.230]Thank you, Jean And Steve
[00:16:06.470]I appreciate your comments.
[00:16:07.760]And we recognize that we're giving you
[00:16:09.777]a very high level 30,000 foot view
[00:16:13.025]of what's going on right now.
[00:16:16.140]what we really wanted to do
[00:16:18.190]what we wanted to make, kind of the meat of the summit
[00:16:21.355]are a couple of examples from our faculty.
[00:16:26.690]And so what I'd like to do
[00:16:30.790]is I'm going to go ahead and have Steve introduce
[00:16:33.859]a couple of folks.
[00:16:39.800]I'll have introduced a couple of folks
[00:16:42.960]from the university of Nebraska medical center.
[00:16:46.270]My apologies, I got my slides out of order there
[00:16:48.391]to talk about what they have done
[00:16:51.948]in their course.
[00:16:57.460]So I am so honored to be able to introduce Lindsey Donner
[00:17:00.430]and Marnie I'm off,
[00:17:01.370]who are faculty with the medical laboratory sciences program
[00:17:05.440]here at UMC.
[00:17:07.292]And you know, I've worked with Linsey and Marnie
[00:17:09.860]in the past, and they are educators/ innovators
[00:17:13.820]who have been able to rethink assessment
[00:17:17.300]and to apply it, especially in the past year
[00:17:19.303]when we've had so many challenges.
[00:17:22.050]And so I will leave it to Linsey and Marnie
[00:17:24.650]to fill you in on all the great work that they've done
[00:17:28.250]regarding the MLS 310 course.
[00:17:32.970]And Marnie I was going to go ahead and make you the host
[00:17:36.460]so you can share your screen
[00:17:38.750]unless you want me to make Linsey the host.
[00:17:41.800]If you could actually make Linsey the host
[00:17:44.721]'cause she's going to actually share her screen
[00:17:47.314]I'm just gonna chat her away.
[00:17:51.574]Good morning everybody.
[00:17:53.871]Thank you for coming in and listening to us talk.
[00:17:57.119]So as Linsey and I developed
[00:17:59.862]a fully online microbiology course
[00:18:03.880]it's a pathogenic micro organism course
[00:18:06.270]in infectious disease.
[00:18:08.180]The course is open to any college student
[00:18:10.370]who wants to take a micro course.
[00:18:11.433]It's not actually housed
[00:18:12.860]within the medical laboratory science program
[00:18:15.130]as part of program requirements.
[00:18:16.860]So anybody can take the course
[00:18:19.170]since it is fully online
[00:18:20.826]with no in-person laboratory.
[00:18:23.436]It makes it a challenge to teach microbiology concepts.
[00:18:26.700]So we had to develop a way
[00:18:28.263]for students to be taught,
[00:18:30.094]how to do laboratory techniques
[00:18:32.470]how to interpret cultures
[00:18:34.200]that we place for lack of a better word
[00:18:37.480]in the microbiology lab.
[00:18:39.410]And we did this by recording laboratory demonstration videos
[00:18:42.407]and we also recorded what we call unknown laboratory videos.
[00:18:46.874]So those videos, the unknowns ones
[00:18:48.996]we use actually in assessment processes
[00:18:52.281]that we built into our learning management system canvas.
[00:18:56.710]So students work through that assessment
[00:18:59.340]just like they would,
[00:19:00.280]if they were sitting in a microbiology lab
[00:19:03.104]we walk them through that whole process.
[00:19:05.910]So they have to interpret colony morphology
[00:19:08.230]from a microbiology agar plate
[00:19:10.920]They have to interpret tests that
[00:19:12.760]we have recorded for them or taken pictures of.
[00:19:15.698]So it walks them through that cognitive piece
[00:19:18.930]of actually identifying bacteria or yeast
[00:19:22.520]in a laboratory setting.
[00:19:24.240]So it was just a different way of thinking
[00:19:25.780]how do we teach laboratory techniques
[00:19:27.610]when we don't have them physically in a classroom?
[00:19:30.640]And it makes them engage with that material
[00:19:33.688]and really think through
[00:19:35.550]how they use what we're teaching them theoretically
[00:19:39.330]with that, I'm actually going to turn it over to Linsey.
[00:19:41.560]Who's going to show you one of the unknowns
[00:19:44.100]that we actually built into canvas as a quiz.
[00:19:50.380]Okay. So let me open up canvas.
[00:19:54.010]So we have our online microbiology course in canvas
[00:19:59.150]Can everybody see that?
[00:20:00.360]Okay. Can you see it, Marnie?
[00:20:01.880]Okay. So here's an unknown example.
[00:20:05.770]We built the unknowns as quizzes.
[00:20:08.120]So they're graded quizzes in canvas
[00:20:10.636]And before I show you the unknown
[00:20:14.260]I'll kind of show some of the settings here.
[00:20:17.740]So you can see it's a graded quiz.
[00:20:19.630]It falls within the assignment group of lab unknowns
[00:20:22.490]and the unknowns correspond
[00:20:24.138]to the theory component in our course.
[00:20:27.510]So we have lectures and then study questions and quizzes
[00:20:32.039]and then there's a laboratory component with each module
[00:20:35.560]that corresponds with what they're learning with the theory.
[00:20:38.487]And we built them as where the student can not backtrack.
[00:20:43.434]So once they submit the answer
[00:20:45.670]they can't go to the previous question.
[00:20:49.423]And the reason being is because we wanted
[00:20:53.100]to use this as a graded assignment
[00:20:55.504]and we are giving the student the answer
[00:20:58.669]and the next question.
[00:21:00.270]So that way they can stay on track.
[00:21:02.744]So we didn't want them to be penalized
[00:21:05.400]for getting completely off track
[00:21:07.380]but we wanted to keep them in that thought process
[00:21:11.010]as they are working through the culture.
[00:21:14.090]So those are the settings that we've used in canvas
[00:21:17.840]and then I'll just hit preview.
[00:21:19.300]So we can see similar to what the student would see
[00:21:23.120]which canvas does give them the notification
[00:21:25.600]that they cannot change their answer once they submit it
[00:21:29.005]and we'll hit begin.
[00:21:33.120]There were some instructions at the beginning
[00:21:35.071]it kind of tells the student what to reference
[00:21:41.519]in the beginning we tell them what references to use
[00:21:44.900]to help kind of guide their thought process.
[00:21:46.920]But by the end of the semester,
[00:21:48.386]they're completely independent.
[00:21:51.010]But here you can see, we have question number one
[00:21:53.923]where we've embedded a video
[00:21:56.000]where the student is watching the video
[00:21:58.952]in order to interpret the colony morphology.
[00:22:04.020]And I'm gonna skip over some of the branding
[00:22:06.180]and initial slides that we have,
[00:22:08.400]but you can see we have the similar to how they would be
[00:22:11.320]in the clinical microbiology laboratory.
[00:22:13.530]This is exactly how we interpret cultures.
[00:22:16.324]We worked with UNL that IHNR media services
[00:22:22.214]to create some of our videos
[00:22:24.178]and the student can pause and rewind
[00:22:27.751]to interpret the video as they go along.
[00:22:32.090]If they wanna pause it to view it more closely, they can.
[00:22:35.940]And then we have the question with the video
[00:22:38.978]and then they would type their answer as you can see here
[00:22:42.931]and click next
[00:22:46.640]we built the unknowns where the pictures and videos
[00:22:50.310]are embedded in the questions.
[00:22:52.700]So here you can see they are interpreting a picture
[00:22:55.885]and they type their answer
[00:22:57.588]and anything that's a short answer.
[00:23:00.520]We would go in and manually grade later
[00:23:02.980]where if it's a multiple choice question
[00:23:06.132]canvas would grade it for you.
[00:23:09.030]And the students are told ahead of time
[00:23:11.200]they need to keep track
[00:23:12.710]of their culture interpretation notes,
[00:23:16.500]similar to how they would
[00:23:17.740]in the real clinical microbiology laboratory
[00:23:19.830]where you're always documenting every step, what you do.
[00:23:22.480]So that way someone can follow up
[00:23:25.510]the next day or with those results.
[00:23:29.150]But you can see here
[00:23:30.310]that we have embedded both.
[00:23:34.307]These are like a biochemical tests that we've recorded.
[00:23:37.925]So we have lemon lab demonstration videos
[00:23:40.998]which explain the procedure
[00:23:43.240]that they watch in their assignments.
[00:23:45.290]The unknowns are completely silent
[00:23:47.270]where they're just watching the video
[00:23:49.370]and interpreting the test.
[00:23:52.450]Okay. So then I'm just gonna submit
[00:23:55.840]even though we haven't finished.
[00:23:57.130]So that way you guys can see what the student sees
[00:23:59.370]after they submit the unknown quiz
[00:24:05.050]but after they hit submit,
[00:24:06.349]the feedback is given to the student
[00:24:08.850]and that it shows their answer
[00:24:10.373]the correct answer and any provided feedback
[00:24:14.870]that we have pre-built into canvas.
[00:24:17.040]So since it is an asynchronous course
[00:24:19.960]we do give them immediate feedback
[00:24:21.890]where then they can go back and review the answers
[00:24:24.570]and go over that information
[00:24:27.152]and make it kind of those
[00:24:28.970]that formative assessment
[00:24:30.300]where it's still a part of their learning
[00:24:31.840]but it is a graded sort of low stakes part of their grade.
[00:24:36.215]So that was a little bit about the unknowns.
[00:24:39.801]We also have just an array of laboratory videos
[00:24:45.121]that correspond with the procedures
[00:24:48.080]that the students are required to interpret.
[00:24:51.534]So for example, you guys just saw the catalyst test
[00:24:55.730]on one of our gram-positive organisms,
[00:24:58.160]but they can go and also watch the procedure video
[00:25:01.490]if they want beforehand
[00:25:03.182]that will show them how to perform the procedure.
[00:25:07.761]And then also the interpretation of the tests.
[00:25:11.640]So they can use that to help supplement their learning also.
[00:25:16.090]But this is a way where-
[00:25:18.356]so they're not in the clinical laboratory
[00:25:21.930]or a student laboratory,
[00:25:23.185]but they get a-
[00:25:24.729]they're not actually manipulating the cultures
[00:25:27.220]but they're still working through that culture process
[00:25:30.306]to understand at each step
[00:25:32.680]this is what I need to do
[00:25:34.500]and think about what they should do for their next step.
[00:25:37.240]In that culture interpretation and correlation process.
[00:25:43.400]We do have a session I was gonna mention this afternoon.
[00:25:47.385]We wanted to mention at 2:30
[00:25:49.700]which goes a little bit more into depth
[00:25:51.700]on how we develop the online unknowns
[00:25:53.787]in lab demonstration videos.
[00:25:55.820]If you're interested in watching that this afternoon
[00:25:58.660]or on a future date,
[00:26:09.600]I think that was all that I had.
[00:26:11.980]Marnie, did you have anything else to input?
[00:26:14.660]No, I think it was just a different way for us
[00:26:17.180]to develop an assessment
[00:26:19.440]since we didn't have them in person.
[00:26:21.451]And we've actually implemented the same concepts
[00:26:25.091]into poor medical laboratory science students
[00:26:27.840]to supplement what they are actually physically doing
[00:26:30.440]in an in-person lab.
[00:26:32.230]It just really helps them see meant concepts.
[00:26:36.260]And we actually can see their thought process
[00:26:37.993]because when somebody's sitting in front of you in the lab,
[00:26:40.110]you don't always see that.
[00:26:41.770]So it's a way for them to-
[00:26:44.090]that feel like a better word verbalize how they're thinking.
[00:26:47.550]So maybe we can catch
[00:26:48.870]some of those critical thinking errors
[00:26:51.467]and get that corrected before a more formative assessment.
[00:26:57.749]If anybody has any questions
[00:26:59.740]for Marnie and Linsey
[00:27:01.589]you're more than welcome
[00:27:02.910]to either unmute your microphone and ask away,
[00:27:06.280]or if you would prefer to type that into the chat window
[00:27:09.570]we'd be happy to facilitate questions through chat as well.
[00:27:28.230]I might add,
[00:27:29.120]even though this is very microbiology specific
[00:27:31.864]I think anybody could employ these same concepts
[00:27:37.350]for any number of topics
[00:27:41.750]see how a student is thinking through a process.
[00:27:52.260]Marnie, we've got a question in the chat.
[00:27:54.750]Have you thought of using question banks for randomization?
[00:28:03.520]We don't have question banks,
[00:28:05.780]but we do have extra unknowns that we have created
[00:28:12.120]that can be switched out throughout the year
[00:28:17.310]or between semesters.
[00:28:29.375]We do have question banks for our exams(laughing)
[00:28:36.800]One question that I would ask
[00:28:38.621]could you maybe talk about, how long it took
[00:28:43.920]to create these alternative assessments
[00:28:47.700]kind of give folks an expectation of effort if you will.
[00:28:53.820]I'm sure that this has been a lot of efforts
[00:28:56.905]but I'm curious to hear what you have to say about that too.
[00:29:03.770]Developing once we got going,`
[00:29:05.947]it got a lot easier
[00:29:07.233]but we did a lot of story boarding.
[00:29:09.723]based on theory,
[00:29:12.940]what they were learning theory wise
[00:29:14.420]what did we want them to get out of the lab piece of it?
[00:29:18.900]So there was a lot of story boarding
[00:29:20.570]of what unknowns we do we wanna go through
[00:29:24.518]finding those videos,
[00:29:26.250]putting it all together
[00:29:27.890]It was time-consuming.
[00:29:29.800]I think once I started getting on a roll
[00:29:31.520]maybe it took me 30 minutes to put them together
[00:29:34.220]but that's after like all the videos have been recorded.
[00:29:36.990]The picture had been taken
[00:29:39.280]that process of getting the videos
[00:29:41.082]recorded, edited, developed
[00:29:43.324]that's a whole other workload
[00:29:50.060]that that was a significant workload
[00:29:51.840]to get to that piece
[00:29:52.780]where we could actually put the unknowns together.
[00:29:55.750]So that, to me, that was the longest piece
[00:29:57.670]was developing the videos and taking the pictures
[00:30:00.440]actually putting together
[00:30:01.670]what we call our unknowns or assessment.
[00:30:03.978]But, you know, like I said, anywhere
[00:30:05.790]from 30 minutes to an hour
[00:30:07.020]to really think through that process
[00:30:08.560]and then get them built into canvas.
[00:30:10.839]Once we kind of got that whole workload worked out
[00:30:15.577]We've got another question and chat.
[00:30:18.894]why do you not use the time setting option?
[00:30:23.280]Do you feel students would prepare better
[00:30:24.860]If the exams were timed?
[00:30:27.930]Our formative assessments,
[00:30:29.380]we do use the timer for these
[00:30:31.850]since they're meant to be more of a learning concept
[00:30:34.840]and we actually let them use outside resources
[00:30:36.930]they can refer to procedures
[00:30:39.130]they can look at testing flow charts.
[00:30:41.089]We didn't wanna use that time feature
[00:30:42.900]cause we really wanted them to learn from them
[00:30:45.700]and learn to use our resources
[00:30:47.450]and to cement those concepts.
[00:30:53.650]We've got another,
[00:30:54.730]what do you say to students
[00:30:55.850]that like to go back and double-check work
[00:30:57.733]before submitting assessments
[00:31:00.226]when you prevent them from going back to previous answers
[00:31:05.431]We do try to do a really good job
[00:31:07.900]of explaining up front the purpose of these assessments
[00:31:10.610]and saying it's supposed to simulate
[00:31:12.180]what you are doing in the laboratory and the lab.
[00:31:15.169]You really shouldn't be going back and repeating testing
[00:31:17.844]cause that's consuming consumables
[00:31:23.260]So we're trying to teach them really the concepts
[00:31:25.880]of what you really would be doing a lab.
[00:31:27.560]You wouldn't be repeating tests
[00:31:28.760]over and over and over in the lab
[00:31:31.500]just because it's time consuming
[00:31:32.830]and it costs a lot of money.
[00:31:34.010]So it's simulating that setting.
[00:31:36.400]They do have the ability
[00:31:37.640]and plus we give them a lot of times the answer
[00:31:40.440]and the next question.
[00:31:42.250]So a lot of times, like if they interpret it,
[00:31:44.280]for example the catalyst test and they said it was negative
[00:31:46.540]was actually a positive.
[00:31:48.070]We tell them the catalyst test was positive.
[00:31:50.170]What's your next step?
[00:31:51.520]So we've actually given them the answer
[00:31:56.780]Any kind of final thoughts you wanna leave folks with
[00:32:00.161]before we move on
[00:32:04.319]Just the student feedback on the unknowns is incredible
[00:32:09.420]and that it's tends to be their favorite part of the course
[00:32:14.290]but also a lot of students have commented that it's also
[00:32:17.669]you know, one of the most difficult components
[00:32:20.270]because you can't just Google the answer.
[00:32:22.400]Like you really have to be correlating what you're learning
[00:32:25.629]with working through the unknowns.
[00:32:28.001]And they do really come out
[00:32:29.980]and how they like how it takes them exactly
[00:32:32.110]through the thought process that they would be doing
[00:32:34.646]in the real clinical microbiology laboratory.
[00:32:37.137]So it's making them stop think,
[00:32:39.587]"okay this is what I should do next"
[00:32:42.110]or "let's interpret these results."
[00:32:44.350]So the feedback
[00:32:45.610]for both the online medical microbiology course
[00:32:49.100]and then within our medical laboratory science program
[00:32:51.567]save the unknowns, help them prepare
[00:32:54.016]for either the clinical bench or the exams
[00:32:59.210]and the help them understand the clinical microbiology lab.
[00:33:02.640]So they do enjoy them.
[00:33:06.960]Well, I just wanted to thank both of you,
[00:33:08.430]Marnie and Linsey for sharing your course
[00:33:11.500]and this alternative assessment.
[00:33:13.350]And just to remind folks
[00:33:15.060]that both Marnie and Linsey are presenting.
[00:33:18.247]I think it's later this afternoon.
[00:33:20.399]So if you would like a little bit more detail,
[00:33:24.145]please visit their session for more information.
[00:33:30.723]Next. I'm going to have Steve McGahn
[00:33:34.686]introduce our next guest speaker.
[00:33:38.388]And we will learn a little bit about
[00:33:42.385]this psych 203 course.
[00:33:48.140]Thank you, Michael.
[00:33:49.730]I am introducing Dr. Jean Mandernach
[00:33:52.160]who I've had the pleasure of working with
[00:33:54.300]almost my entire career here at UNK.
[00:33:56.790]She and I worked together on a project
[00:34:00.073]a little over two years ago,
[00:34:04.470]looking at how to create an authentic assessment
[00:34:07.603]in large format online classes
[00:34:11.560]that didn't have -
[00:34:12.393]that only had one professor and didn't have an entire,
[00:34:15.630]it didn't have a group of graduate assistants
[00:34:17.430]assisting with the grading.
[00:34:19.274]And so with that, I'm going to turn it over to Jean
[00:34:23.300]to talk about a psychology 203 course.
[00:34:26.508]Thank you, Steve.
[00:34:28.900]And as Steve mentioned
[00:34:30.100]when we originally started this project
[00:34:32.070]it was a lot about how do we handle large enrollment.
[00:34:35.695]But I think that becomes really pivotal
[00:34:37.810]when we talk about getting rid of proctored exams
[00:34:40.170]or the challenges with online remote proctoring
[00:34:42.752]because large classes historically
[00:34:45.740]rely on those proctored exams.
[00:34:47.703]There's just so many students
[00:34:49.970]that when you start talking about adding
[00:34:52.027]alternate assessments and doing other creative things,
[00:34:56.040]it doesn't become feasible
[00:34:57.210]And it's completely overwhelming
[00:34:58.710]when you're looking at classes that have 75, 100
[00:35:01.420]or more students in them.
[00:35:03.620]And so if we think about those general studies
[00:35:06.270]large enrollment classes,
[00:35:07.710]we often really cling to those proctored exams
[00:35:10.754]because we say, you know we can assess their learning
[00:35:13.890]because they're nervous and worried and studying.
[00:35:17.090]It gives them a way to promote interaction
[00:35:19.431]and engagement with that course content.
[00:35:22.110]And particularly in large classes
[00:35:23.950]where we know may not be familiar with them,
[00:35:26.320]we can verify their identity
[00:35:27.974]and it's just one big test or maybe a couple big tests.
[00:35:32.057]So it's manageable for our workload
[00:35:35.800]making things even more challenging.
[00:35:37.580]When we start looking at the online classroom
[00:35:39.933]we also are trying to think about
[00:35:42.133]how do we make an environment that has,
[00:35:44.500]really good interaction and instructional presence
[00:35:47.530]and students have an opportunity for feedback.
[00:35:50.250]So when Steve and I sat down,
[00:35:51.810]it wasn't just a matter
[00:35:53.350]of how do we get more students in there
[00:35:55.340]but how do we make pedagogical choices
[00:35:58.211]about the entire assessment experience
[00:36:02.420]and how assessment interacts with teaching and learning
[00:36:05.500]to really make a high quality online class
[00:36:08.446]that could have a lot of students
[00:36:10.606]that we're not just defaulting to relying on,
[00:36:13.923]you know, kind of the big picture assessment
[00:36:16.510]where we might have a midterm and a final exam.
[00:36:19.380]And so our goal was really about maintaining
[00:36:22.201]how do we have a large class
[00:36:24.113]but still have good students satisfaction,
[00:36:27.570]and good learning and good interaction and good feedback.
[00:36:31.350]And if we think about this
[00:36:32.390]in terms of alternative assessment,
[00:36:35.120]and we go back and layer this
[00:36:36.720]on kind of why we often default to proctored exams?
[00:36:42.460]we really need to shift our thinking a little bit
[00:36:45.300]because we can address each of these goals independently
[00:36:48.870]to get a different experience.
[00:36:50.870]So we can assess learning.
[00:36:52.790]If we go back and move away from a traditional exam
[00:36:55.910]into something that really is more about the objectives
[00:36:59.255]what is it that we're trying to accomplish?
[00:37:01.626]How do we get there students
[00:37:03.210]to really engage with our material,
[00:37:04.830]if there's not the threat of a high stakes exam
[00:37:07.365]and what are some other opportunities
[00:37:09.240]to verify their identity
[00:37:10.697]and how do we do this in a manner
[00:37:12.526]that makes it amenable to a lot of different students.
[00:37:17.020]Now, historically in small classes
[00:37:19.660]they've just approached this by having a lot going on.
[00:37:22.830]We didn't worry about identifying-
[00:37:24.570]you know, verifying their identity
[00:37:26.130]because there was just so many small touch points
[00:37:29.510]that we said it would be really hard
[00:37:31.210]to hire somebody to come in and do that
[00:37:33.580]or find somebody to (indistinct)
[00:37:35.600]If we have lots of little touch points
[00:37:37.670]throughout the semester.
[00:37:39.140]And that actually works pretty well
[00:37:40.640]if you have a small class,
[00:37:42.700]but traditionally, if we have a larger class
[00:37:45.084]like what is often the case in general studies courses
[00:37:48.389]we can't maintain and that throughout the semester.
[00:37:51.710]So we end up having really small touch points
[00:37:54.660]throughout the semester
[00:37:55.725]and then we have this big final exam
[00:37:57.924]possibly even a big midterm and then a big final exam.
[00:38:01.871]What Steve and I did is we said,
[00:38:04.227]"let's try to flip the model.
[00:38:05.904]Let's recognize that students
[00:38:08.300]need interaction presence and feedback
[00:38:10.944]but that we need to do it in a manageable way."
[00:38:14.010]So rather than having high touch points consistently
[00:38:17.958]we designed an assessment approach
[00:38:20.320]that said every single assessment we work into that course
[00:38:23.810]is not only going to be driven
[00:38:25.434]by the learning objectives of the course,
[00:38:28.010]but also as a function of making sure
[00:38:30.770]we have regular opportunities
[00:38:32.428]for interaction, presence, and feedback
[00:38:34.898]and that we're not doing that every week for every student
[00:38:38.930]because that wouldn't be manageable for the instructor.
[00:38:42.550]The course we were looking at
[00:38:43.810]was a general psychology course.
[00:38:45.760]And I think this is really important
[00:38:47.850]because the context matters
[00:38:49.346]in this case we are not talking about something
[00:38:52.940]where there is a kind of like a certification exam
[00:38:56.696]or a licensure exam.
[00:38:58.432]We are talking about a general studies course
[00:39:01.709]that has lots of students
[00:39:04.120]that will never take another psychology course
[00:39:06.130]in their life.
[00:39:07.078]It tends to be a high enrollment course.
[00:39:09.810]And we're when we're looking at the online environment
[00:39:12.540]it can often be a bottleneck.
[00:39:14.000]If we use our kind of traditional small approaches
[00:39:19.403]in this context we really want students
[00:39:21.180]to be introduced to the field,
[00:39:22.720]We want them to reflect on their own life,
[00:39:25.010]We want them to be effective consumers of psychology
[00:39:27.530]and the world around them,
[00:39:29.330]but we're not talking about
[00:39:30.745]really detailed in-depth information.
[00:39:34.570]And so what we did
[00:39:35.575]is we said," let's think about how we meet
[00:39:38.530]all these big picture pedagogical goals
[00:39:42.043]as they crossover with the specific learning objectives
[00:39:45.300]of the course
[00:39:46.750]we know in this particular course
[00:39:48.740]if we can just get students to read the textbook
[00:39:50.970]they're going to understand it.
[00:39:52.260]It's not a really in-depth course."
[00:39:54.683]So we started by integrating weekly mastery quizzes
[00:39:58.500]and those quizzes were designed exclusively
[00:40:01.220]to get students to read
[00:40:02.630]so they can take the quiz as many times as they'd like
[00:40:05.440]because we just want them to read and get involved.
[00:40:08.620]And those are all automatically graded
[00:40:10.300]right in the learning management system.
[00:40:12.710]Now we still have the final exam and the final exam review
[00:40:16.750]but that was mainly just to motivate
[00:40:19.333]ongoing preparation and studying.
[00:40:22.020]We just wanted to push it across the semester
[00:40:24.680]so that students didn't feel
[00:40:25.980]like they were cramming at the end.
[00:40:28.290]So we wanted them to connect,to integrate
[00:40:31.090]and to continue to study
[00:40:33.200]but rather than having a lot, going on every week
[00:40:36.330]we looked at all of our different objectives
[00:40:38.956]and we identified five key assignments
[00:40:42.112]that would allow us to meet those objectives.
[00:40:44.492]And so the assignments that we picked out
[00:40:46.732]were a discussion that allowed students to interact.
[00:40:50.343]They could engage, they could connect with the instructor
[00:40:53.610]in a traditional online discussion.
[00:40:56.180]They also had a journal
[00:40:57.600]that was really about reflecting it,
[00:40:59.870]applying it to their own lives.
[00:41:01.510]So very personalized, very intimate
[00:41:03.950]to the individual student,
[00:41:05.518]a research analysis
[00:41:07.173]that was geared exclusively
[00:41:09.356]to get students to understand
[00:41:11.177]and appreciate psychology as a science
[00:41:14.116]a video exploration that was really about them delving
[00:41:17.860]into their own interests,
[00:41:19.600]them connecting with the material,
[00:41:21.280]finding things that they found
[00:41:22.610]personally relevant and valuable.
[00:41:24.840]And then a current event analysis,
[00:41:26.870]connecting psychology to the world around them
[00:41:29.560]and what they see around them.
[00:41:31.640]Now, from a design perspective
[00:41:33.367]we picked these different assignments,
[00:41:36.251]looking specifically at the big picture learning objectives
[00:41:39.840]of the course.
[00:41:41.070]We want students to interact.
[00:41:42.400]We want them to reflect
[00:41:43.490]we want them to apply to the world around them.
[00:41:45.960]So all of the assignments
[00:41:47.490]were specifically designed to meet those objectives.
[00:41:51.490]And then pedagogically,
[00:41:52.950]we went through and said, "okay,
[00:41:54.530]we know we need interaction, presence, and feedback."
[00:41:57.279]So we had to make sure
[00:41:59.099]that any given assignment
[00:42:00.920]was also going to ensure
[00:42:02.151]that students would get those pedagogical experiences
[00:42:05.770]in the course.
[00:42:06.970]So a discussion is going to give them
[00:42:08.740]a lot of in the moment feedback and interaction
[00:42:11.250]with the instructor
[00:42:12.385]but it's not gonna do much on the research end.
[00:42:14.933]The Journal's going to be very personalized
[00:42:17.202]and get them feedback
[00:42:18.742]but it's not going to allow for interaction.
[00:42:21.580]So we picked these specifically
[00:42:23.502]because they would allow us to touch on every area
[00:42:26.778]but students wouldn't have to do it every week.
[00:42:29.970]So rather than every week, every assignment
[00:42:33.470]we did a distributed workload
[00:42:35.262]and we divided the course into three blocks.
[00:42:38.286]And those blocks were just to ensure
[00:42:40.582]that students got distributed interaction,
[00:42:44.362]presence and feedback.
[00:42:46.450]We didn't want a student
[00:42:47.620]to do all their discussions at first
[00:42:50.100]and then not have really good interaction
[00:42:51.890]with the instructor.
[00:42:52.723]For the remainder of the course.
[00:42:54.450]We wanted them to get into a pattern
[00:42:56.206]where they would regularly get each
[00:42:58.250]of those three key components.
[00:43:00.365]So the course was divided then into three blocks.
[00:43:03.920]And the purpose of the blocks was just to distribute
[00:43:07.015]the interaction, presence, and feedback.
[00:43:09.563]And we told students every block
[00:43:11.915]you're going to do one of the assignment options each week.
[00:43:16.230]And then when we start the next block
[00:43:18.060]you're going to again, pick
[00:43:19.180]and do one of the assignment options each week.
[00:43:21.687]It didn't really matter to us which ones they pick
[00:43:24.980]because at the end of the course,
[00:43:26.503]they will have each gotten
[00:43:28.212]each assignment option three times
[00:43:30.592]and it will be distributed throughout the course.
[00:43:33.750]And then from a grading perspective,
[00:43:35.890]we really reduced the weight on that final exam
[00:43:38.780]because it was not going to be a proctored final exam.
[00:43:41.940]The purpose of it was simply to keep students
[00:43:44.760]trying to connect and review
[00:43:46.411]and really have distributed learning
[00:43:48.690]throughout the entire course.
[00:43:50.528]So we implemented this.
[00:43:51.928]We originally had a course of
[00:43:53.972]I believe the first time we did it was 25 students
[00:43:56.970]before we ever made the changes.
[00:43:59.070]After we made the changes,
[00:44:00.240]we went to 60 we've then gone to 80
[00:44:02.510]and even up to a hundred.
[00:44:04.630]And we found a really consistent pattern of results.
[00:44:08.130]Not only could we have more students in the class
[00:44:10.480]without having to integrate the proctored exam
[00:44:13.330]to verify their identity,
[00:44:14.874]the students have said,
[00:44:16.078]"we love this new structure.
[00:44:19.640]We're doing something different all the time.
[00:44:22.090]I got to see the instructor with the videos.
[00:44:24.270]I got to feel like I was connecting with them
[00:44:26.800]but I also got to dive into the research."
[00:44:29.057]They said they did not like having a discussion
[00:44:32.050]every single week
[00:44:33.150]because I got monotonous.
[00:44:34.680]So in this new course,
[00:44:36.280]they really got into those discussions
[00:44:38.410]because they would only do three of them
[00:44:40.130]throughout the entire semester.
[00:44:42.560]Most important, their learning was the same.
[00:44:45.479]So we did not have every student
[00:44:48.338]doing everything all the time.
[00:44:50.820]And when we looked at final exam scores
[00:44:53.170]when we looked at quiz grades
[00:44:54.430]when we looked at distributions, it was equivalent.
[00:44:58.499]There was no change
[00:44:59.332]in their overall comprehension
[00:45:00.570]or retention of the information,
[00:45:02.730]but their experience increased.
[00:45:05.580]They felt like they were more connected to the instructor.
[00:45:08.230]They felt like they had higher interaction.
[00:45:10.190]They felt like it was more personalized
[00:45:12.160]and they were much more satisfied.
[00:45:15.047]And an interesting side note
[00:45:16.741]is when we tracked my amount of time teaching
[00:45:19.933]it actually decreased.
[00:45:22.170]So I could take on more students
[00:45:24.390]and not have a greater workload
[00:45:26.370]while students were experiencing
[00:45:28.350]a more enjoyable course
[00:45:30.610]from their perspective
[00:45:31.800]without any decrease in the learning.
[00:45:34.640]So ups, I should go back to that one
[00:45:37.110]with that said I think I just left a couple of minutes here
[00:45:40.120]for questions, thoughts, comments.
[00:45:43.140]We had a lot going on.
[00:45:44.350]So if you want me to explain one part of that a little more,
[00:45:47.020]feel free to let me know
[00:45:50.110]rather than just a assignment
[00:45:51.490]we really tried to approach wholistically.
[00:45:53.700]How do we take an alternate approach to assessment
[00:45:56.930]that just decreases that reliance on a final exam?
[00:46:01.160]Yeah I have quick question
[00:46:03.293]were these embedded into a tool like canvas,
[00:46:08.416]yes.Everything was managed in canvas.
[00:46:12.504]I will say, if you wanna know,
[00:46:14.190]what's the downside of this approach,
[00:46:16.690]we didn't have an effective way to trick canvas
[00:46:19.770]into managing the options.
[00:46:21.549]So the way I approached it
[00:46:23.019]is every module.
[00:46:24.278]It actually listed all of those as assignments.
[00:46:27.840]And I would do a video and I would explain to students
[00:46:30.050]you're not going to complete all of them
[00:46:31.980]but canvas is going to keep telling you that they're due.
[00:46:34.820]So you're going to have to ignore
[00:46:36.260]that canvas is saying you have overdue
[00:46:37.880]and missing assignments.
[00:46:39.077]And then I set up the grade book in canvas
[00:46:41.670]to only record the highest three grades
[00:46:44.240]for each assignment category.
[00:46:45.860]So they were set in categories little bit clunky.
[00:46:49.160]I will say, if anybody has good tips or tricks
[00:46:52.190]to trick canvas into managing the options better
[00:46:55.500]that would be the one downside of this.
[00:46:57.320]There was a little bit of student confusion
[00:46:59.100]about the options and that canvas kept telling them
[00:47:02.000]they had things due that were not actually due for them.
[00:47:11.570]Are there any other questions for Jean?
[00:47:22.920]Well, Jean, I just wanna thank you very much
[00:47:24.438]for presenting and sharing your experience in the course
[00:47:28.860]has I think a wonderful example
[00:47:30.598]of how alternative assessments can really change
[00:47:34.154]how a course looks and feels and works for students.
[00:47:38.859]So I really appreciate Steve and you coming forward
[00:47:43.310]and sharing that with everybody here
[00:47:46.025]before we go, we've got I think one or two minutes left,
[00:47:51.370]which is kind of exactly what we thought
[00:47:53.730]we were going to end up with.
[00:47:55.750]I want to share a few different resources for folks.
[00:47:58.851]We will have the slide deck available
[00:48:02.015]on the symposium agenda page
[00:48:07.629]after the symposium is over.
[00:48:10.160]So these resources will be available for you.
[00:48:13.178]We've got some resources that are available
[00:48:15.730]from a few different places
[00:48:17.090]from the center for transformative teaching at UNL
[00:48:19.700]they have a couple of different resources
[00:48:21.420]a Proctor proctoring decision tree,
[00:48:23.435]which I think is great.
[00:48:24.808]And aiding faculty in determining
[00:48:27.246]whether a proctored exam meets your needs.
[00:48:30.016]There's also a link for rethinking assessment
[00:48:33.684]for hands-on learning.
[00:48:35.252]We've also got a couple of resources
[00:48:37.856]that are related perhaps more specifically to stem courses
[00:48:42.588]and alternative assessments for stem.
[00:48:45.370]And then we also included some selected strategies
[00:48:50.610]that we kind of ran across while we were discussing
[00:48:54.642]what to do for the summits.
[00:48:58.632]So we've got a couple of different items
[00:49:00.790]that are listed here just describing
[00:49:02.310]what those items are.
[00:49:04.325]And like I said, these will be included
[00:49:06.360]in the slide deck after the symposium
[00:49:08.660]we've included all of our resources as well.
[00:49:10.990]So you can check those out at your leisure.
[00:49:13.220]I will ask that we really appreciate your feedback
[00:49:18.527]on the sessions and the workshops
[00:49:21.187]you can navigate to the agenda page
[00:49:25.230]on the symposium website.
[00:49:26.871]And that agenda page will provide links
[00:49:31.434]to the session evaluations.
[00:49:34.445]We take the evaluations very seriously
[00:49:37.310]and they do help to form
[00:49:38.813]inform how we make changes from each year for the symposium.
[00:49:43.811]And then lastly, I just wanted to,
[00:49:46.030]again thank our sponsors for helping us out
[00:49:48.940]and making the technology possible
[00:49:51.287]for what we're doing here.
[00:49:54.563]Thank you all very, very much.
[00:49:58.427]I appreciate all of your work
[00:50:02.420]and I wanna especially thank
[00:50:04.139]the instructional designers, Steve and Jullian,
[00:50:09.510]and on and Mike Kozack and Kimberly
[00:50:14.120]and Kendra for really coming forward and helping out
[00:50:19.036]with this particular session.
[00:50:21.880]I hope it's been useful for everyone.
Log in to post comments