`- [Michael] The ID summit each year,
instructional designers from the system
get together to discuss relevant and timely issues
and instructional design
to share ideas and collaborate on solutions
to pedagogical challenges.
This year, we've decided to open an ID summit
to all participants of the symposium
as the first time that we've had
the ID summit open to everyone
who is participating or who registered for the summit.
So welcome to those of you
who are not instructional designers.
We hope that this event is fruitful for you.
I want to talk a little bit just about what we intend to do
in today's workshop.
We've got three different sections
we're gonna talk about.
I'll give you kind of the synopsis
as colleges and universities
kind of shifted to remote learning
in response to the pandemic.
A lot of our institutions grappled
with how to ensure academic integrity
and how to handle assessments.
Many institutions turned to third-party proctoring solutions
for assistance and ensuring that academic integrity,
however, the widespread use of these services
kind of exposed some critical issues
related to privacy, inclusion
and in some cases the legality,
this workshop was gonna highlight
some of these national issues.
We're gonna share how a university of Nebraska campuses
and faculty members and instructional designers
advocated for an implemented alternative assessments
and identify what some of those strategies are.
We have two faculty guests
who are going to talk about
what they've done in their courses
and kind of highlights some of the solutions
that they've implemented.
So we've got some good time
for them to showcase what's been done in their course
as well as answer questions of the group.
(Michael clears throat)
We're going to have a couple of instructional designers
kind of share what's been done with the campuses
across our university system.
And then we've got some resources to share with you as well.
Those will most likely get shared
at the end of the presentation.
And then we'll also make the slide deck available
on the symposium agenda
after the symposium is over
after we've gathered all of this information.
So what I'd like to do
is I'm gonna go ahead
and we have some of the goals.
As I mentioned, we wanna kind of understand
some of those national issues that are related
to the use of third-party proctoring services.
Want to hear how colleagues have modified some assessments
and then identify some of those strategies.
Just to give you a little bit of housekeeping.
My name's Michael Jolley.
I'm the assistant director
for the university of Nebraska online.
I'll be moderating the session.
We also have some tech support
in the form of Nate Morris, who is one of our co-hosts.
If there are technical issues
please feel free to mention them in chat.
And we'll try to get those
Natal will try to get those resolved during the session.
I know that we're going to have time for Q and a.
And so we'll try to manage that as best that we can.
You can either ask questions and chat,
or I think if we've got a good block of time near the end
we might just open it up for people
to be able to kind of share their screen
or unmute and ask them question that way
so we can have some discussion in that format.
So the first thing that I kind of wanted to talk about
and unfortunately, a couple of our colleagues
are not able to make it this morning
Kristen and Michael had another commitment this morning
that has prevented them from joining.
So I'm gonna try to-
I'm gonna try to cover for them as best that I can.
So when we talk about third-party proctoring solutions
we're talking about those solutions,
using tools such as respondentsq monitor
or Proctorial or Proctor U
things of that nature,
where students have to sign up to use the service,
the services, using video technology
to take a look at the place
in which they're taking the assessment.
And there are fees
and in some cases associated with these services
and the widespread use
of these third-party proctoring services
has led to some challenges nationally
in the last year or so.
We've seen a few different news articles
that have come out
that have talked about proctoring solutions
whether they are truly helping
with academic integrity issues.
There've been student privacy concerns.
There have been institutions
that have decided to reject remote proctoring altogether
how this is becoming a larger issue
as students complain about privacy issues,
as they complain about equity issues.
So this is the national challenges
associated with third-party proctoring solutions
are widespread,
and we have some several different resources
that I think are available for you
in our resources in this particular slide
qdeck to take a look at if you're interested,
we know that this has been an proctoring
that third party proctoring has exploded during this time
but we really wanted to take the opportunity
from a pedagogical standpoint to examine
what was being done on our campuses
and how the use of alternate assessment solutions
may be a better way forward.
So what I'd like to do now
is I'd like to introduce a couple of our colleagues,
Jullian Manza from university of Nebraska
Lincoln and Steve McGann
from UNK to talk about just a few of the different ways
that our campuses responded
to the pandemic from a proctoring standpoint
from an assessment standpoint
and highlight just a few of those particular issues
and challenges that our campuses are facing.
So with that, I'm gonna turn it over to Jillian and Steve.
-: So I'm gonna go ahead and start.
Julian has another thing
she's going to talk about afterwards,
but we want to start off by saying
that none of us are saying
that proctoring is not a solution that can be employed ever.
One of the things that were one of the challenges
that we've faced over the last year or so
with COVID is the swell in proctoring
and how much more people are relying on proctoring
as a solution for academic integrity
and there are issues that come along with that.
I'm going to talk about two
Jullian's going to talk about one,
but the first is at UNK.
And I know at other, at UNL
one of the issues that pops up constantly
is the expense of proctoring
and whether it's institutionally funded
or whether it's student funded
there is a large expense
that is associated with any proctoring
from 10 to $15 an hour for a live Proctor
to somewhere around two to $3 a test
for some of the automated systems.
And if the university takes that on,
obviously as a strain on our financial resources,
but more importantly if we require the students
to do some of that
test Proctor and can easily total up
to be about the same as what a textbook would cost.
And so balancing those issues
is one of the things that we're really looking at
across the universities
of how we balance what we're expecting the students to do
and how much proctoring is actually necessary.
And that may not just be the proctoring service itself.
It may also include equipment that the students have to buy.
It may include time
that the faculty have to spend supporting it.
The institution has to spend supporting it.
So there are a lot of different things
to go along with that.
But one of the things I wanted to talk about in here
is actually something that doesn't apply specifically
to the proctoring side of thing
but it is something that is very germane
to what we're talking about.
And that is the issue of rigor
or whether or not we're asking students
to be rigorous in their online
in their courses,
not just online, sorry, I work in online
but also whether are we having them do busy work or crunch.
And one of the things that we see
is there's a lot of adding of things on
you add pieces in all the time,
but is it really necessary?
And one of the presentations coming up
they're going to talk about, you know
kind of pulling away from some of that.
But when we work with faculty,
one of the things that we try to talk about
is finding the pieces that you absolutely need to assess
and not doing things like double assessing materials.
Do you need to do both a paper and a test
to get those materials
to properly assess those materials?
Or can we back off of the tests
as the tests tend to be a little bit more of an issue,
finding those ways to authentically assess
is really what the instructional designers
at all the campuses
have been trying to do.
And the cracks in the system
have been exposed over the last year
that have made us really come in and focus more on that.
And with that, I'm going to stop.
Cause I only had a couple of minutes to talk
and I'm sure it went a little over
and turn it over to Jillian
-: Hi everyone.
Yeah. Thanks Steve.
So as he kind of mentioned we know that the coronavirus
certainly didn't start the online proctoring trend
but what we do know is it certainly exacerbated it.
And we, along with many other institutions
across the country,
quickly pivoted last spring
and various proctoring technologies were deployed.
They were deployed swiftly
and unforeseen equity issues
really rose to the surface and very quickly.
And so that's a little bit
of what I'm going to be covering today.
So I don't have time to go into detail on these issues
but really the key takeaway here
is that remote proctoring technologies
can disproportionately hurt
and disadvantage our marginalized students.
In particular, the instances I'm going to list
are students of lower economic status and students of color.
So as we prepare for return to campus this fall,
we know that the volume of remote proctoring
is likely to decrease.
However, the equity issues associated
with remote proctoring for exams
are still going to exist.
So I'm going to list out some of the examples
that have been reported in multiple articles
across the country throughout the past year,
just to kind of give you a feel
of what some of those equity issues are
that have been experienced particularly
by our already marginalized students.
So some examples reports from students
where they talked about instances
where facial recognition software failed
to identify students of color
or those wearing religious garb
And in some of those cases,
students were locked out of tests
because they were unable
to be recognized by the face and gaze detection software.
In other reports, students with disabilities
notice that they were inaccurately flagged
for possible academic integrity violations
due to their involuntary movements like muscle spasms
others reported significant mental health concerns.
Students stated that they had an increase in anxiety
due to the discomfort of having either an unknown Proctor
often referred to as a stranger
or just the AI technology monitoring their movements
specifically trying to discern
whether they're cheating or not.
And often students stated
that this heightened anxiety and over monitoring
really negatively impacted their mental health
which then of course led to a decrease or decline
in their performance on an exam or assessment
there's cost as Steve mentioned,
creating a significant issue of inequity
for students in many ways
whether they're picking up the cost
for paying for that proctoring service
or having to pay for an expensive textbook
that has the cost of that proctoring fee
embedded into its cost.
Additionally, some other cost related issues
relating to having a computer
that meets the technological requirements
necessary to support proctoring service.
This can be really challenging for many students to attain.
Also many households
are what are considered one device households.
So having multiple devices might not be possible
which limits students access to that computer
or that technology
on that has the specifications needed for that exam taking,
having access to continuous high speed internet connectivity
is a requirement for students to be able to take the exams.
I know in my own household
sometimes that's even a little bit challenging.
And so unfortunately, to go elsewhere
and find alternative high-speed internet is challenging.
We know that free and public wifi is certainly not an option
because it can't support the proctoring service.
Rural test takers here in Nebraska also
have found this to be extremely challenging
to meet those technological requirements.
Lastly, location is certainly another issue
in terms of the requirement
that says things like you need a quiet, well lit space
that might not be easily accessible for all of our students
to have this quiet well-lit space.
Students with children in particular mothers
are certainly impacted by this.
Or if you have a busy household,
you might find it challenging to have uninterrupted minutes
to take exams.
And then you are flagged
for environmental noises in the room,
or if anyone or anything comes into your camera's view.
So environmental scans are really challenging
for those tests taking environments.
We know they're common
but students consider them to be extremely intrusive.
And in many cases a live Proctor is joining that test taker.
And so there's an element there
of environmental or visual overshooting
that occurs where they're seeing
a lot of someone's environment
that may not be a very comforting experience
for that test taker.
So that's a long laundry list
of some of the issues of inequity
that are being experienced by our students.
Like I said, again,our low socioeconomic status students
and students of color and many others.
So as I said, while the volume and usage
we know is likely to go down
as many of us return to campus
remote proctoring will still exist.
So I encourage you to think about your assessments
and we encourage you
and that's what we're about to talk about today.
If you can explore some alternative assessments
and perhaps move away from exams
and things that require that remote proctoring service
Michael: Thank you, Jean And Steve
I appreciate your comments.
And we recognize that we're giving you
a very high level 30,000 foot view
of what's going on right now.
what we really wanted to do
what we wanted to make, kind of the meat of the summit
are a couple of examples from our faculty.
And so what I'd like to do
is I'm going to go ahead and have Steve introduce
a couple of folks.
I'm sorry.
I'll have introduced a couple of folks
from the university of Nebraska medical center.
My apologies, I got my slides out of order there
to talk about what they have done
in their course.
-: Hi everyone.
So I am so honored to be able to introduce Lindsey Donner
and Marnie I'm off,
who are faculty with the medical laboratory sciences program
here at UMC.
And you know, I've worked with Linsey and Marnie
in the past, and they are educators/ innovators
who have been able to rethink assessment
and to apply it, especially in the past year
when we've had so many challenges.
And so I will leave it to Linsey and Marnie
to fill you in on all the great work that they've done
regarding the MLS 310 course.
-: And Marnie I was going to go ahead and make you the host
so you can share your screen
unless you want me to make Linsey the host.
-: If you could actually make Linsey the host
Michael: okay
-: 'cause she's going to actually share her screen
Michael: Perfect
-: I'm just gonna chat her away.
So(laughing)
-: Good morning everybody.
Thank you for coming in and listening to us talk.
So as Linsey and I developed
a fully online microbiology course
it's a pathogenic micro organism course
in infectious disease.
The course is open to any college student
who wants to take a micro course.
It's not actually housed
within the medical laboratory science program
as part of program requirements.
So anybody can take the course
since it is fully online
with no in-person laboratory.
It makes it a challenge to teach microbiology concepts.
So we had to develop a way
for students to be taught,
how to do laboratory techniques
how to interpret cultures
that we place for lack of a better word
in the microbiology lab.
And we did this by recording laboratory demonstration videos
and we also recorded what we call unknown laboratory videos.
So those videos, the unknowns ones
we use actually in assessment processes
that we built into our learning management system canvas.
So students work through that assessment
just like they would,
if they were sitting in a microbiology lab
we walk them through that whole process.
So they have to interpret colony morphology
from a microbiology agar plate
They have to interpret tests that
we have recorded for them or taken pictures of.
So it walks them through that cognitive piece
of actually identifying bacteria or yeast
in a laboratory setting.
So it was just a different way of thinking
how do we teach laboratory techniques
when we don't have them physically in a classroom?
And it makes them engage with that material
and really think through
how they use what we're teaching them theoretically
with that, I'm actually going to turn it over to Linsey.
Who's going to show you one of the unknowns
that we actually built into canvas as a quiz.
-: Okay. So let me open up canvas.
So we have our online microbiology course in canvas
Can everybody see that?
Okay. Can you see it, Marnie?
Okay. So here's an unknown example.
We built the unknowns as quizzes.
So they're graded quizzes in canvas
And before I show you the unknown
I'll kind of show some of the settings here.
So you can see it's a graded quiz.
It falls within the assignment group of lab unknowns
and the unknowns correspond
to the theory component in our course.
So we have lectures and then study questions and quizzes
and then there's a laboratory component with each module
that corresponds with what they're learning with the theory.
And we built them as where the student can not backtrack.
So once they submit the answer
they can't go to the previous question.
And the reason being is because we wanted
to use this as a graded assignment
and we are giving the student the answer
and the next question.
So that way they can stay on track.
So we didn't want them to be penalized
for getting completely off track
but we wanted to keep them in that thought process
as they are working through the culture.
So those are the settings that we've used in canvas
and then I'll just hit preview.
So we can see similar to what the student would see
which canvas does give them the notification
that they cannot change their answer once they submit it
and we'll hit begin.
There were some instructions at the beginning
it kind of tells the student what to reference
in the beginning we tell them what references to use
to help kind of guide their thought process.
But by the end of the semester,
they're completely independent.
But here you can see, we have question number one
where we've embedded a video
where the student is watching the video
in order to interpret the colony morphology.
And I'm gonna skip over some of the branding
and initial slides that we have,
but you can see we have the similar to how they would be
in the clinical microbiology laboratory.
This is exactly how we interpret cultures.
We worked with UNL that IHNR media services
to create some of our videos
and the student can pause and rewind
to interpret the video as they go along.
If they wanna pause it to view it more closely, they can.
And then we have the question with the video
and then they would type their answer as you can see here
and click next
we built the unknowns where the pictures and videos
are embedded in the questions.
So here you can see they are interpreting a picture
and they type their answer
and anything that's a short answer.
We would go in and manually grade later
where if it's a multiple choice question
canvas would grade it for you.
And the students are told ahead of time
they need to keep track
of their culture interpretation notes,
similar to how they would
in the real clinical microbiology laboratory
where you're always documenting every step, what you do.
So that way someone can follow up
the next day or with those results.
But you can see here
that we have embedded both.
These are like a biochemical tests that we've recorded.
So we have lemon lab demonstration videos
which explain the procedure
that they watch in their assignments.
The unknowns are completely silent
where they're just watching the video
and interpreting the test.
Okay. So then I'm just gonna submit
even though we haven't finished.
So that way you guys can see what the student sees
after they submit the unknown quiz
but after they hit submit,
the feedback is given to the student
and that it shows their answer
the correct answer and any provided feedback
that we have pre-built into canvas.
So since it is an asynchronous course
we do give them immediate feedback
where then they can go back and review the answers
and go over that information
and make it kind of those
that formative assessment
where it's still a part of their learning
but it is a graded sort of low stakes part of their grade.
So that was a little bit about the unknowns.
We also have just an array of laboratory videos
that correspond with the procedures
that the students are required to interpret.
So for example, you guys just saw the catalyst test
on one of our gram-positive organisms,
but they can go and also watch the procedure video
if they want beforehand
that will show them how to perform the procedure.
And then also the interpretation of the tests.
So they can use that to help supplement their learning also.
But this is a way where-
so they're not in the clinical laboratory
or a student laboratory,
but they get a-
they're not actually manipulating the cultures
but they're still working through that culture process
to understand at each step
this is what I need to do
and think about what they should do for their next step.
In that culture interpretation and correlation process.
We do have a session I was gonna mention this afternoon.
We wanted to mention at 2:30
which goes a little bit more into depth
on how we develop the online unknowns
in lab demonstration videos.
If you're interested in watching that this afternoon
or on a future date,
I think that was all that I had.
Marnie, did you have anything else to input?
-: No, I think it was just a different way for us
to develop an assessment
since we didn't have them in person.
And we've actually implemented the same concepts
into poor medical laboratory science students
to supplement what they are actually physically doing
in an in-person lab.
It just really helps them see meant concepts.
And we actually can see their thought process
because when somebody's sitting in front of you in the lab,
you don't always see that.
So it's a way for them to-
that feel like a better word verbalize how they're thinking.
So maybe we can catch
some of those critical thinking errors
and get that corrected before a more formative assessment.
Michael: If anybody has any questions
for Marnie and Linsey
you're more than welcome
to either unmute your microphone and ask away,
or if you would prefer to type that into the chat window
we'd be happy to facilitate questions through chat as well.
-: I might add,
even though this is very microbiology specific
I think anybody could employ these same concepts
for any number of topics
to have-
see how a student is thinking through a process.
Michael: Marnie, we've got a question in the chat.
Have you thought of using question banks for randomization?
-: We don't have question banks,
but we do have extra unknowns that we have created
that can be switched out throughout the year
or between semesters.
We do have question banks for our exams(laughing)
Michael: One question that I would ask
could you maybe talk about, how long it took
to create these alternative assessments
kind of give folks an expectation of effort if you will.
I'm sure that this has been a lot of efforts
but I'm curious to hear what you have to say about that too.
-: Developing once we got going,`
it got a lot easier
but we did a lot of story boarding.
based on theory,
what they were learning theory wise
what did we want them to get out of the lab piece of it?
So there was a lot of story boarding
of what unknowns we do we wanna go through
finding those videos,
putting it all together
It was time-consuming.
I think once I started getting on a roll
maybe it took me 30 minutes to put them together
but that's after like all the videos have been recorded.
The picture had been taken
that process of getting the videos
recorded, edited, developed
that's a whole other workload
that that was a significant workload
to get to that piece
where we could actually put the unknowns together.
So that, to me, that was the longest piece
was developing the videos and taking the pictures
actually putting together
what we call our unknowns or assessment.
But, you know, like I said, anywhere
from 30 minutes to an hour
to really think through that process
and then get them built into canvas.
Once we kind of got that whole workload worked out
-: We've got another question and chat.
why do you not use the time setting option?
Do you feel students would prepare better
If the exams were timed?
-: Our formative assessments,
we do use the timer for these
since they're meant to be more of a learning concept
and we actually let them use outside resources
they can refer to procedures
they can look at testing flow charts.
We didn't wanna use that time feature
cause we really wanted them to learn from them
and learn to use our resources
and to cement those concepts.
-: Thank you.
We've got another,
what do you say to students
that like to go back and double-check work
before submitting assessments
when you prevent them from going back to previous answers
-: We do try to do a really good job
of explaining up front the purpose of these assessments
and saying it's supposed to simulate
what you are doing in the laboratory and the lab.
You really shouldn't be going back and repeating testing
cause that's consuming consumables
it's costly.
So we're trying to teach them really the concepts
of what you really would be doing a lab.
You wouldn't be repeating tests
over and over and over in the lab
just because it's time consuming
and it costs a lot of money.
So it's simulating that setting.
They do have the ability
and plus we give them a lot of times the answer
and the next question.
So a lot of times, like if they interpret it,
for example the catalyst test and they said it was negative
was actually a positive.
We tell them the catalyst test was positive.
What's your next step?
So we've actually given them the answer
-: Any kind of final thoughts you wanna leave folks with
before we move on
-: Just the student feedback on the unknowns is incredible
and that it's tends to be their favorite part of the course
but also a lot of students have commented that it's also
you know, one of the most difficult components
because you can't just Google the answer.
Like you really have to be correlating what you're learning
with working through the unknowns.
And they do really come out
and how they like how it takes them exactly
through the thought process that they would be doing
in the real clinical microbiology laboratory.
So it's making them stop think,
"okay this is what I should do next"
or "let's interpret these results."
So the feedback
for both the online medical microbiology course
and then within our medical laboratory science program
save the unknowns, help them prepare
for either the clinical bench or the exams
and the help them understand the clinical microbiology lab.
So they do enjoy them.
-: Well, I just wanted to thank both of you,
Marnie and Linsey for sharing your course
and this alternative assessment.
And just to remind folks
that both Marnie and Linsey are presenting.
I think it's later this afternoon.
So if you would like a little bit more detail,
please visit their session for more information.
Next. I'm going to have Steve McGahn
introduce our next guest speaker.
And we will learn a little bit about
this psych 203 course.
Steve
-: Thank you, Michael.
I am introducing Dr. Jean Mandernach
who I've had the pleasure of working with
almost my entire career here at UNK.
She and I worked together on a project
a little over two years ago,
looking at how to create an authentic assessment
in large format online classes
that didn't have -
that only had one professor and didn't have an entire,
it didn't have a group of graduate assistants
assisting with the grading.
And so with that, I'm going to turn it over to Jean
to talk about a psychology 203 course.
-: Thank you, Steve.
And as Steve mentioned
when we originally started this project
it was a lot about how do we handle large enrollment.
But I think that becomes really pivotal
when we talk about getting rid of proctored exams
or the challenges with online remote proctoring
because large classes historically
rely on those proctored exams.
There's just so many students
that when you start talking about adding
alternate assessments and doing other creative things,
it doesn't become feasible
And it's completely overwhelming
when you're looking at classes that have 75, 100
or more students in them.
And so if we think about those general studies
large enrollment classes,
we often really cling to those proctored exams
because we say, you know we can assess their learning
because they're nervous and worried and studying.
It gives them a way to promote interaction
and engagement with that course content.
And particularly in large classes
where we know may not be familiar with them,
we can verify their identity
and it's just one big test or maybe a couple big tests.
So it's manageable for our workload
making things even more challenging.
When we start looking at the online classroom
we also are trying to think about
how do we make an environment that has,
really good interaction and instructional presence
and students have an opportunity for feedback.
So when Steve and I sat down,
it wasn't just a matter
of how do we get more students in there
but how do we make pedagogical choices
about the entire assessment experience
and how assessment interacts with teaching and learning
to really make a high quality online class
that could have a lot of students
that we're not just defaulting to relying on,
you know, kind of the big picture assessment
where we might have a midterm and a final exam.
And so our goal was really about maintaining
how do we have a large class
but still have good students satisfaction,
and good learning and good interaction and good feedback.
And if we think about this
in terms of alternative assessment,
and we go back and layer this
on kind of why we often default to proctored exams?
we really need to shift our thinking a little bit
because we can address each of these goals independently
to get a different experience.
So we can assess learning.
If we go back and move away from a traditional exam
into something that really is more about the objectives
what is it that we're trying to accomplish?
How do we get there students
to really engage with our material,
if there's not the threat of a high stakes exam
and what are some other opportunities
to verify their identity
and how do we do this in a manner
that makes it amenable to a lot of different students.
Now, historically in small classes
they've just approached this by having a lot going on.
We didn't worry about identifying-
you know, verifying their identity
because there was just so many small touch points
that we said it would be really hard
to hire somebody to come in and do that
or find somebody to (indistinct)
If we have lots of little touch points
throughout the semester.
And that actually works pretty well
if you have a small class,
but traditionally, if we have a larger class
like what is often the case in general studies courses
we can't maintain and that throughout the semester.
So we end up having really small touch points
throughout the semester
and then we have this big final exam
possibly even a big midterm and then a big final exam.
What Steve and I did is we said,
"let's try to flip the model.
Let's recognize that students
need interaction presence and feedback
but that we need to do it in a manageable way."
So rather than having high touch points consistently
we designed an assessment approach
that said every single assessment we work into that course
is not only going to be driven
by the learning objectives of the course,
but also as a function of making sure
we have regular opportunities
for interaction, presence, and feedback
and that we're not doing that every week for every student
because that wouldn't be manageable for the instructor.
The course we were looking at
was a general psychology course.
And I think this is really important
because the context matters
in this case we are not talking about something
where there is a kind of like a certification exam
or a licensure exam.
We are talking about a general studies course
that has lots of students
that will never take another psychology course
in their life.
It tends to be a high enrollment course.
And we're when we're looking at the online environment
it can often be a bottleneck.
If we use our kind of traditional small approaches
in this context we really want students
to be introduced to the field,
We want them to reflect on their own life,
We want them to be effective consumers of psychology
and the world around them,
but we're not talking about
really detailed in-depth information.
And so what we did
is we said," let's think about how we meet
all these big picture pedagogical goals
as they crossover with the specific learning objectives
of the course
we know in this particular course
if we can just get students to read the textbook
they're going to understand it.
It's not a really in-depth course."
So we started by integrating weekly mastery quizzes
and those quizzes were designed exclusively
to get students to read
so they can take the quiz as many times as they'd like
because we just want them to read and get involved.
And those are all automatically graded
right in the learning management system.
Now we still have the final exam and the final exam review
but that was mainly just to motivate
ongoing preparation and studying.
We just wanted to push it across the semester
so that students didn't feel
like they were cramming at the end.
So we wanted them to connect,to integrate
and to continue to study
but rather than having a lot, going on every week
we looked at all of our different objectives
and we identified five key assignments
that would allow us to meet those objectives.
And so the assignments that we picked out
were a discussion that allowed students to interact.
They could engage, they could connect with the instructor
in a traditional online discussion.
They also had a journal
that was really about reflecting it,
applying it to their own lives.
So very personalized, very intimate
to the individual student,
a research analysis
that was geared exclusively
to get students to understand
and appreciate psychology as a science
a video exploration that was really about them delving
into their own interests,
them connecting with the material,
finding things that they found
personally relevant and valuable.
And then a current event analysis,
connecting psychology to the world around them
and what they see around them.
Now, from a design perspective
we picked these different assignments,
looking specifically at the big picture learning objectives
of the course.
We want students to interact.
We want them to reflect
we want them to apply to the world around them.
So all of the assignments
were specifically designed to meet those objectives.
And then pedagogically,
we went through and said, "okay,
we know we need interaction, presence, and feedback."
So we had to make sure
that any given assignment
was also going to ensure
that students would get those pedagogical experiences
in the course.
So a discussion is going to give them
a lot of in the moment feedback and interaction
with the instructor
but it's not gonna do much on the research end.
The Journal's going to be very personalized
and get them feedback
but it's not going to allow for interaction.
So we picked these specifically
because they would allow us to touch on every area
but students wouldn't have to do it every week.
So rather than every week, every assignment
we did a distributed workload
and we divided the course into three blocks.
And those blocks were just to ensure
that students got distributed interaction,
presence and feedback.
We didn't want a student
to do all their discussions at first
and then not have really good interaction
with the instructor.
For the remainder of the course.
We wanted them to get into a pattern
where they would regularly get each
of those three key components.
So the course was divided then into three blocks.
And the purpose of the blocks was just to distribute
the interaction, presence, and feedback.
And we told students every block
you're going to do one of the assignment options each week.
And then when we start the next block
you're going to again, pick
and do one of the assignment options each week.
It didn't really matter to us which ones they pick
because at the end of the course,
they will have each gotten
each assignment option three times
and it will be distributed throughout the course.
And then from a grading perspective,
we really reduced the weight on that final exam
because it was not going to be a proctored final exam.
The purpose of it was simply to keep students
trying to connect and review
and really have distributed learning
throughout the entire course.
So we implemented this.
We originally had a course of
I believe the first time we did it was 25 students
before we ever made the changes.
After we made the changes,
we went to 60 we've then gone to 80
and even up to a hundred.
And we found a really consistent pattern of results.
Not only could we have more students in the class
without having to integrate the proctored exam
to verify their identity,
the students have said,
"we love this new structure.
It's different.
We're doing something different all the time.
I got to see the instructor with the videos.
I got to feel like I was connecting with them
but I also got to dive into the research."
They said they did not like having a discussion
every single week
because I got monotonous.
So in this new course,
they really got into those discussions
because they would only do three of them
throughout the entire semester.
Most important, their learning was the same.
So we did not have every student
doing everything all the time.
And when we looked at final exam scores
when we looked at quiz grades
when we looked at distributions, it was equivalent.
There was no change
in their overall comprehension
or retention of the information,
but their experience increased.
They felt like they were more connected to the instructor.
They felt like they had higher interaction.
They felt like it was more personalized
and they were much more satisfied.
And an interesting side note
is when we tracked my amount of time teaching
it actually decreased.
So I could take on more students
and not have a greater workload
while students were experiencing
a more enjoyable course
from their perspective
without any decrease in the learning.
So ups, I should go back to that one
with that said I think I just left a couple of minutes here
for questions, thoughts, comments.
We had a lot going on.
So if you want me to explain one part of that a little more,
feel free to let me know
rather than just a assignment
we really tried to approach wholistically.
How do we take an alternate approach to assessment
that just decreases that reliance on a final exam?
Member: Yeah I have quick question
were these embedded into a tool like canvas,
-: yes.Everything was managed in canvas.
I will say, if you wanna know,
what's the downside of this approach,
we didn't have an effective way to trick canvas
into managing the options.
So the way I approached it
is every module.
It actually listed all of those as assignments.
And I would do a video and I would explain to students
you're not going to complete all of them
but canvas is going to keep telling you that they're due.
So you're going to have to ignore
that canvas is saying you have overdue
and missing assignments.
And then I set up the grade book in canvas
to only record the highest three grades
for each assignment category.
So they were set in categories little bit clunky.
I will say, if anybody has good tips or tricks
to trick canvas into managing the options better
that would be the one downside of this.
There was a little bit of student confusion
about the options and that canvas kept telling them
they had things due that were not actually due for them.
Member: Thanks.
Michael: Are there any other questions for Jean?
Well, Jean, I just wanna thank you very much
for presenting and sharing your experience in the course
has I think a wonderful example
of how alternative assessments can really change
how a course looks and feels and works for students.
So I really appreciate Steve and you coming forward
and sharing that with everybody here
before we go, we've got I think one or two minutes left,
which is kind of exactly what we thought
we were going to end up with.
I want to share a few different resources for folks.
We will have the slide deck available
on the symposium agenda page
after the symposium is over.
So these resources will be available for you.
We've got some resources that are available
from a few different places
from the center for transformative teaching at UNL
they have a couple of different resources
a Proctor proctoring decision tree,
which I think is great.
And aiding faculty in determining
whether a proctored exam meets your needs.
There's also a link for rethinking assessment
for hands-on learning.
We've also got a couple of resources
that are related perhaps more specifically to stem courses
and alternative assessments for stem.
And then we also included some selected strategies
that we kind of ran across while we were discussing
what to do for the summits.
So we've got a couple of different items
that are listed here just describing
what those items are.
And like I said, these will be included
in the slide deck after the symposium
we've included all of our resources as well.
So you can check those out at your leisure.
I will ask that we really appreciate your feedback
on the sessions and the workshops
you can navigate to the agenda page
on the symposium website.
And that agenda page will provide links
to the session evaluations.
We take the evaluations very seriously
and they do help to form
inform how we make changes from each year for the symposium.
And then lastly, I just wanted to,
again thank our sponsors for helping us out
and making the technology possible
for what we're doing here.
Thank you all very, very much.
I appreciate all of your work
and I wanna especially thank
the instructional designers, Steve and Jullian,
and on and Mike Kozack and Kimberly
and Kendra for really coming forward and helping out
with this particular session.
I hope it's been useful for everyone.