-: I have the outfit (mumbles)
ready to go for the talk. (laughing)
So I'm Heather Richards Rissetto
and soon I'll be joined by Carrie Heitman in a bit
but imma start out the talk
so before we begin, we wanna thank the CAS Inquire committee
and June Griffin for bringing to (mumbles) this program
and for inviting us to speak today.
We'd also like to thank our anthro colleague Taylor
for directing CAS Inquire
and congratulate him on his great success
in its inaugural year.
While computers now influence some might say infiltrate
nearly every aspect of our lives,
they were originally designed for number crunching
that is they were designed to perform basic calculations
on big data sets, much faster than humans could.
Today, computers are integral to education, healthcare,
entertainment, transportation, agriculture, our social lives
well, basically everything.
While we may be aware of their presence,
we might often not notice their influence
on our daily decision making.
Everyday computers are making decisions for us, or are they?
Are computers really taking the place
of human decision making?
Yes and no.
Our objective in this talk is to delve deeper
into this question, through exploration of algorithms
and their influence on culture.
But before we delve into how algorithms affect our lives,
whether positively or negatively, and then reflecting
on the future, let's begin with the basics.
What are algorithms?
And how do they relate to artificial intelligence
and machine learning two terms currently inundating media?
What exactly is an algorithm?
We hear this word almost every day
and yet to most of us, algorithms are still daunting
and mysterious but algorithms are simply a set of guidelines
that describe how to perform a task.
Remember in elementary school when you had to stand
up in front of the class and give a speech
on how to perform a task such as baking a ranza?
Or how to navigate traffic on a (mumbles) game day?
Well, essentially you created an algorithm.
A sequence of instructions on how to do something.
In our digital age however, algorithms typically refer
to a sequence of instructions telling a computer
what to do rather than a human.
Basically, an algorithm is just computer code
and the software is fundamentally composed of algorithms.
So, not that mysterious after all.
So, how do algorithms impact our daily lives?
Algorithmic operations are behind all of the responses
that we get from digital assistance such as Siri, Cortana,
or Alexa, when we ask them a question,
they use one set of algorithms to interpret
or get a sense of what you've asked,
and then another set to acquire the information
you've requested so, just this morning I asked Siri,
what should I have for breakfast?
She responded broccoli
So I think the algorithm might be a little bit (mumbles)
but we'll see unless you love broccoli so.
Social media is wrought with algorithms,
Instagram, Twitter, Facebook,
and these algorithms are constantly changing.
Prior to late 2019, Facebook's newsfeed algorithm ranked
and displayed posts for users based on relevance
driven primarily by businesses, brands and media.
Now Facebook is using a new algorithm to select news feed
based on the likelihood that a user will have a positive
reaction, they do this by studying your friends,
family and groups so that's not creepy at all.
And oops, back here (mumbles)
And recently, McDonald's bought an algorithm
for data driven meal choices that can automatically change
menus depending on the weather, time of day
and traffic and of course suggest an order
based on your license plate.
So how does such algorithms relate
to artificial intelligence and machine learning?
Artificial intelligence or ai is an umbrella term,
referring to instances where machines simulate human
behavior using human intelligence.
When a machine completes tasks based on a set
of instructions, that is an algorithm,
this intelligent behavior is what we call artificial
intelligence, machine learning is simply a type of ai.
In the case of algorithms, instead of repeatedly processing
a stable set of instructions, systems based on machine
learning, rewrite themselves as they work.
In other words, humans write the original algorithm
or computer code, but then the computer modifies
this original algorithm based on what it's told to learn,
what it's learning from that is its training set
and how it's learning.
So this brings up two critical points.
First, we must remember is Cathy O'Neil,
author of " Weapons of Math Destruction" writes,
algorithms are opinions embedded in code.
Humans write the rules that comprise algorithms,
and who typically writes these rules?
Well those, typically with wealth and power.
Second, in a similar vein, Meredith Whittaker,
a former Google employee was quoted saying,
" What's frightening about ai isn't terminators
and super intelligent machines, it's the way ai works
to centralize knowledge and power in the hands
of those who already have it, and further disempower
those who don't."
But before we go down that road,
let's talk about some potential positives of algorithms.
For example, self driving vehicles or buses
can potentially lead to safer roads or fewer traffic jams
reducing carbon emissions.
So here you're seeing Chancellor Green,
avoiding a self driving bus in Lincoln's 2018
autonomous shuttle project.
Another potential positive is in retail,
using real time geo fencing,
which location based algorithms trigger notifications
on sales, best prices and even offer coupons
as you walk down the aisles of your favorite store.
(chuckling)
That makes me wonder what Big Brother was,
but in this week, (chuckling)
so text and speech recognition algorithms can recognize
and translate written text into spoken language
and vice versa, and vice versa translate spoken language
into text or even into drawings.
Narrator: There is a moon at night,
two trees in the field.
Some flowers.
Far away there's a house.
(chattering)
(mumbles)
So this Google experiment turns real time free speech
into animated drawing, such tools can possibly facilitate
web accessibility for disabled persons,
foster alternative modes of communication and much more.
With global population growth and aging populations,
healthcare is one of the world's fastest growing
and most important industries
and algorithms are changing medical practice from diagnosing
and even predicting disease to discovering new medications.
Google software can look into our eyes to assess our risk
for a heart attack or stroke, other algorithms can spot DNA
mutations and tumors and artificial intelligence
is being used to improve breast cancer screening
or predict lung cancer, which is the deadliest cancer
in the world and the sixth most common cause of death.
Recently, an algorithm discovered a much needed,
entirely new antibiotic called Palicin
which successfully destroys several bacteria straints
including some of the most dangerous drug resistant bacteria
on the World Health Organization's most wanted list.
In 2018, Google's art and culture released an app
enabling you to find your art doppelganger,
what could be better?
You took a selfie and Google's facial recognition algorithm
searched the database of art from 17 museums worldwide
to find your twin.
(upbeat music) -: Do I look like her?
(upbeat music)
-: I feel very fancy looking at these, like all of them have
a Victorian collar, their hair is curled,
they have a lot of jewelry (chattering)
and the painting styles are really different
too, like I can tell they're from different eras.
(upbeat music)
-: I think the beard might be a little misleading
because all the results have beards so far,
so I think it's pretty much focusing on that aspect
as opposed to other things.
(upbeat music)
-: I would say it picked up well on the nose.
-: So, will your selfie be in a museum one day?
Nop.
While finding your museum doppelganger is fun,
the science behind it is scary and Arwa Mahdawi,
a columnist for The Guardian writes
in reference to the doppelganger project, I quote,
" Google's latest experiments is less about art
than it does about the burgeoning science
of facial recognition technology
and the rise of the surveillance state."
So as great as it might sound to have your face
or your selfie in a museum one day as archeologists,
Carrie and I can't help to ponder this excerpt
from the New Yorker by Adrian channel.
" What does it mean that our cultural history,
like everything else is increasingly under the watchful eye
of a giant corporation whose business model
rests on data mining?
One dystopian possibility offered by critics
in the wake of the Google selfie app,
was that Google was using all of the millions
of unflattering photos to train its algorithms."
Those that you saw in the video of people using.
" Google has denied this but training goes both ways,
as Google scans and processes more of the world's cultural
artifacts, it will be easier than ever to find
ourselves in history, so long as we offer
ourselves up to the computers gaze."
So on that sobering note,
I'm gonna turn it over to Carrie.
-: Hi everybody?
It's a pleasure to be here.
So there's no doubt that artificial intelligence
or ai is seeping into nearly every aspect of our lives
whether we know it or not.
According to UC Berkeley computer scientist, Dr. Stuart
Russell, the upside of such technologies is enormous
a potential golden age for humanity
but as informed citizens, we also need to understand
the associated risks, the application of complex algorithms
is continuing to expand in ways that many find alarming
or at the very least, outstripping our ability to assess
the associated risks and while this is not an exhaustive
list of such potential negative consequences,
we'd like to share with you a few potential examples.
Let's see if this will work.
(mumbles) okay
So fortune 500 companies such as Hilton,
Goldman Sachs and Unilever are using ai
in their assessments of job candidates.
They work with third party vendors like HireVue
to use a candidate's computer or cell phone
camera to record job interviews,
their proprietary algorithm,
then analyzes the person's facial movements, word choice,
speaking voice before ranking them against other applicants
based on an automatically generated employability score.
This is called affect recognition
and HireVue claims that they can measure qualities
such as grit and track how many times a candidate smiles
and if you are interested, you should just go on YouTube,
and there's just like oodles of like videos
about how to trick the system to try to game
yourself in terms of your employability score.
Companies like HireVue and VCV claim that ai
removes human biases that are rampant in hiring practices
in their 2019 annual report
however, the ai Now Institute points
out that affect recognition
has not been proven to be scientifically valid
and that such technologies may simply automate
discrimination, Whittaker, who heard from earlier
of ai Now states, quote,
" It's profoundly disturbing,
that we have proprietary technology that claims
to differentiate between a productive worker
and a worker who isn't fit based on their facial movements,
their tone of voice and their mannerisms."
A 2019 study published in the Journal of Science,
found evidence of racial bias in one algorithm widely used
within the US healthcare system.
According to the researchers,
the bias occurred because the algorithm uses health costs
as a proxy for health needs.
The author's estimated that this racial bias
reduces the number of black patients
identified for extra care by more than half,
less money is spent on black patients
who have the same level of need
and the algorithm does falsely concludes
that black patients are healthier
than equally sick white patients.
This is a case where existing structural inequalities
based on race, were already part of the data input
and thus were exacerbated
by the Optum proprietary algorithm.
The good news is that the researchers found
that in the racial bias in predicting who needs care,
this could be eliminated by reformulating the algorithm
so that it no longer uses cost as a proxy for needs.
In addition to algorithmic biases in healthcare,
they've also been a number of high profile cases
of racial discrimination in facial recognition algorithms.
One with Google's facial recognition algorithm
that misidentified black faces and another were Nikon's
Coolpix digital camera exhibit bias against Asian faces
with regard to its " Did Someone Blink Morning?"
With this case, in terms of criminal prediction
and sentencing, we see just one of the troubling examples
of algorithmic governance,
where algorithms are now being used to govern citizens.
In 2016 Propublica published an investigative expose
of how criminal justice officials are using risk prediction
equations to inform their decisions about bail,
sentencing and early release.
They collected the risk assessment scores
for more than 10,000 people arrested for crimes
in Florida's Broward County and check to see
how many were charged with further crimes within two years.
They were looking to see if those with higher risk
assessments went on to be arrested for new crimes
compared to those with lower risk assessments
what they found was a racial disparity.
The data showed that black defendants were twice
as likely to be incorrectly labeled as higher risk
than white defendants, conversely, white defendants
labeled low risk were far more likely to end
up being charged offenses than African Americans
with comparably low compass risk score.
You can see one example highlighted on the screen
where the white defendant was given a low risk score
of three but went on to commit three subsequent offenses
within the two year study,
whereas an African American didn't was given a higher risk
score of 10 but went on to commit no subsequent offenses.
Within their 10,000 person study,
the results showed that the risk assessment predictions
overestimated risk for African American defendants
and underestimated risk for white defendants.
The good news here is that the in the aftermath
of this investigative journalism piece,
there's been a concerted effort to help improve
the compass algorithm to reduce racial bias
in risk assessment.
Warning, things are gonna get even more grim (chuckling)
According to some ai experts,
the most serious short term risk
involves autonomous weapons.
The company, STN, for instance,
is working with the Turkish Government
to create swarm drones that use facial recognition
technology to kill a target.
(upbeat music)
Given what we already know about the risks and weaknesses,
of facial recognition algorithms,
this military applications is particularly chilling.
In response to this growing risk a coalition of ai
researchers and advocacy organizations commission to 2017
film called " Slaughterbots."
The film asks the question," What happens
when miniature drones with ai facial recognition
and three grams of explosive programmable to kill
a specific individual become available."
Spoiler alert, it's not good.
The film was presented to the United Nations Convention
on conventional weapons, hosted by the campaign
to stop killer robots.
On a more academic note, a research on ai
and their underlying algorithms has created
by what some account as a crisis in science
a foundation of scientific inquiry is the expectation
that results can be replicated, verified,
unpublished codes and sensitivity to training conditions
have made it difficult for ai researchers
to produce many key results.
We can add to this broad concern the issue of large scale
data sets used in training, which are largely in the hands
of just a few for profit corporations like Amazon,
Google, Facebook, IBM, and Microsoft.
Those of you paying close attention may have noticed
a recurring pattern in the negative consequences
we've mentioned thus far.
These issues were brought forward not by the industries,
or the corporations using the algorithms in question
but by outside researchers, journalists,
and concerned citizens.
So looking ahead, we might reasonably ask the question,
" Do we trust corporations who are using these technologies
for profit and competitive advantage to police themselves?"
There are various other areas of concern
that we'll briefly just mention
here, and for those of you who are interested,
we'd urge you to look at the 2019 report published
by the ai Now Institute.
According to their latest report,
algorithms are increasing the power asymmetry
between workers and employees.
Employers sorry through applications
such as algorithmic wage control,
governments are increasingly adopting ai systems to surveil
and control people but without regulation,
and corporations are expanding the scale
and scope of algorithmic health infrastructures.
These technologies are disproportionately impacting
vulnerable communities, people of color, women and the poor.
It is also important to note that these ai systems
are vulnerable to bad actors who can utilize backdoor access
to retrain the system on bad data.
An algorithm optimized for click through for instance,
can be used to manipulate public opinion.
At a broader level, what are the issues at play
that we should think about in the deployment
of these technologies across such diverse domains?
Campolo and Crawford have argued that there's a discourse
of enchanted determinism,
which surrounds artificial intelligence.
Ai systems are portrayed as exceeding human intelligence
or performance a form of magical or alien genius.
This rhetoric they argue, serves to shield the creators
of these systems from the consequences
their designers may not fully anticipate,
understand or control.
This leads us to an accountability gap,
where deep learning systems might be considered
beyond typical ethical responsibilities
and legal liabilities.
In 2019, Crawford and Shultz published an article
in the Columbia Law Review titled
" Ai systems as state actors."
In the article, they observe how governments
are increasingly turning to third party vendors
to provide algorithmic architecture for public services,
including welfare benefits and criminal risk assessments
but when challenged, many state governments
have disclaimed any knowledge or ability to understand,
explain, or remedy problems created by the ai systems
they procured from those third party vendors.
This accountability gap then means that algorithmic systems
are responsible for government decision making,
without any mechanism for accountability or liability.
Stated differently according to Kitchin
in a relatively recent piece and these are his words,
" We are now entering an era of widespread
algorithmic governance wherein algorithms play
an increasing role in the exercise of power,
a means through which to automate the disciplining
and controlling of societies,
and to increase the efficiency of capital accumulation."
End quote and yet, what we generally lack as a public,
according to Diakopoulos is clarity
about how algorithms exercise their power
over us so, moving forward,
how do we assure transparency and accountability
in these algorithmic systems?
In recent years, we've seen the advent of research
Institute's dedicated to this topic.
There seems to be a widespread recognition
that corporations who control many of these proprietary
algorithms, and much of the underlying data
are not going to police themselves.
In the interest of time we'll share
just a few notable examples.
The AI Now Institute at New York University
was co founded by Meredith Whittaker ,
Whittaker notably, previously founded Google's open research
group and was a court organizer in the Google walkouts
Ai Now and this is their mission statement
is an interdisciplinary research center
dedicated to understanding the social implications
of artificial intelligence.
The Institute has been issuing various reports
some of which you see here is just a smattering.
Also, individuals that are part of the Institute
have been testifying before the United States
House of Representatives Committee on Oversight
and Reform, and they've been publishing various legal
and technical academic journals in various journals.
Similarly, the Ada Lovelace Institute in London
is an independent research and deliberative body
with a mission to ensure data
and ai work for people and society.
They've been pursuing similar strategies,
trying to make the case for the social value of data
and addressing the asymmetries of power and data justice.
Their approach is focused on changing regulations
as you can see here, changing narratives
and changing practices.
It's also worth noting that our neighbor to the north
recently embarked on a Pan Canadian artificial intelligence
strategy and in Europe, there are moves to use
the e-use of general data protection regulation or GDPR,
to address transparency,
and help regulate algorithmic systems.
In the US, it seems most of the activity in this area
is coming not from the government
but from advocacy groups and research centers,
like I mentioned, the Ai Now Institute.
One thing is clear, algorithmic systems,
both hard to say are advancing rapidly,
with tremendous promise
and tremendous peril in equal measure.
We hope this presentation has given you all a sense
of the breadth and complexity for how algorithms
are being applied, and what challenges lie ahead
for all of us.
And in closing, we'd like to share three important
and optimistic considerations issued
by the Ada Lovelace Institute
in their rethinking data report.
So they're calling upon all of us, as actors in society
to consider the following:
how data might narrow any inequalities
rather than widen them.
How data might tackle power imbalances
rather than entrench them.
And how data might help us address the global issues
of resource allocation, climate change and migration,
rather than exacerbate them
and with that thanks very much for your attention.
(clapping)
-: That's it?
-: Heather has to turn on her leg
-: There we go (chuckling)
-: All right, are there any questions?
From anyone in the audience?
Yes?
Student: You gave us the perspective
(mumbles) having control over ai technology
in what way can the everyday person kind of take back,
in what way do we have power
ourselves to influence (mumbles) ai
(mumbles)
-: Great question.
(mumbles)
-: You wanna start and then I'll take over
Female Student: Can you repeat the question
so that everybody can hear it?
-: So the question was related to, we're hearing
about how big corporations, organizations,
et cetera are using big data potentially in negative ways.
So how as an everyday person can we counter
that and what strategies might be used to do so?
So to begin to address that one,
I think we could reiterate what Carrie
had said near the end right?
It sort of joined some advocacy groups, right?
Be aware of what's going on and that's really the first
thing is to, you know, step back, reflect
and think about what's happening,
like when you're using social media,
whether it's Facebook, Instagram
or whatever it might be thinking
about what it is that you're seeing
and what it might impact it might have
maybe on you individually or in a larger sense right?
So that idea of being self reflective,
potentially being proactive in ways
you could become a computer programmer, I guess.
(chuckling) right?
In addition to bringing your humanities background
cross the two together and sort of address
algorithmic development in alternative ways
and alternative perspectives so.
-: I mean I think I'll just add to that, I think
demanding greater transparency
in how these things are working
is one thing that we can we can do, I think
that companies that are using algorithms that may impact
our daily lives in positive and negative ways
are aloof to release those and frankly,
they might be a bit opaque to us anyway
if we did actually see all of the source code,
but to reiterate Heather's point,
I think that getting involved,
being aware, educating yourself about all the ways
these are seeping into our lives is one way to start.
And in some, relatively,
it might seem like relatively simple studies
have actually led to scale changes,
you know, people have done things like just looking
for how for instance how when you Google search
something it auto fills right?
What it thinks you're gonna be asking for just by starting
to ask questions about like, "Okay, well, if I'm logging in
from home or from the state of Nebraska,
what does that look like?"
And then asking the same question actually,
when they are somewhere else, to try to start
to unravel what it is the algorithm thinks
it knows about you and then studies have been built
upon those kinds of observations
to make larger scale changes
or the observations about how these are impacting our lives?
Question over here?
Foreign Student: My question is from biases,
color discrimination so we know the people who created
(mumbles) to (mumbles) about discrimination,
that they have to do with the people
who create (mumbles)?
Who did not do that in (mumbles) why
because I believe machines only produce
what we (mumbles) on them
so could these biases be also be with the creators
of these algorithms?
And how can we change that? -: Absolutely,
though I think that's absolutely right.
I mean, Heather's on top of one of her earlier slides said,
you know, these are opinions embedded in code, right?
And so I think you're right to point out that these biases
came from people right?
That they didn't just fall out of the sky, right?
And so we need to be attentive
to that, who are the programmers?
What are the demographics of the people
who are writing these codes?
And how do we change that?
How do we change, how to do we expand that?
One of the things that the folks at the AI Now Institute
and other places have been pointing out is precisely
that disparity in terms of who's controlling
these algorithms?
Who's writing them?
Who's (mumbles)fortunately impacted by them?
And that we need to change address that power imbalance
within the tech industry as well.
So I don't know if that answers your question?
But you're absolutely right,
that we need to change who's behind the machine
to be able to be more attentive to these issues.
-: In addition, it relates to the data
that we have the data that's on hand
that's in the databases, right?
And what's being used to train sort of machine learning,
or ai and so the data we have is from certain
socio economic classes or different parts of the country
or different parts of the world
and it's not equitable right?
Across the globe and so that it also impacts
sort of the outcomes that we're getting
from the algorithms in addition to the bias
maybe perhaps written into them so.
Yes? Student: (mumbles)
to that if they're producing
discriminatory of bias results in case we have only bias
data to work with, how can we continue to use algorithms
to handle problems that they don't have the consciousness
to understand that they cannot understand
that the data itself is bias
and there's no way for an algorithm (mumbles)
to mend that so how can you continue using them for almost
any societal problem when our data is (mumbles)
-: Right well so the question relates to how can we continue
to use algorithms right?
If the data that's going into them is potentially biased.
And it would say that all data or biased, right?
And I think you're really clear to point
out that yeah, the algorithm itself, the computer
itself, can't make those changes necessarily, right?
We have to look at and be self reflective,
as a society and as individuals and sort of see
what those biases might be in the data
and make light of them and certain changes have occurred.
As Carrie pointed out with some healthcare industries
and others, and when people become aware of those things,
you can rewrite the algorithms so for example
to take out health costs out of an algorithm in that sense.
-: Yeah and just to add, I would (mumbles) to guess
that you know it was some of the ones
that we've been critical of here today,
would simply point out that with time
and with more data input these will improve right?
And this will ameliorate some of the concerns
that have been brought to light,
the challenge though of course is the the fact
that the billions of dollars of investment and the pace
of this is far outstripping or outpacing our ability
to be some sort of checks and balance
about the unintended consequences of these algorithms.
-: [Deep Voiced Student] Is there a list
of references that you all sighted available?
-: We can make them available.
We did not include a slide
with them on there but are happy,
I don't know if there are PowerPoints that then get shared
as part of CAS Inquire but we could add
a bibliography slide (laughing)
that would have all of these
on it -: [Deep Voiced Student] Be
quite beautiful (mumbles) -: Yeah sure
no, thanks happy to share that.
Student: Yeah, I'm really struck
by sort of tension between (mumbles)
all presented as totally revolutionary right?
Partly because (mumbles)
and the (mumbles) is for, well humans have always done
that, (mumbles) to each other when new technology
comes along and (mumbles) some more
and there's sort of this continuity here rather
than the total (mumbles) that you have with the good stuff
so I was curious if you see the discrimination
and the sort of the negative things as just continuity
as more of the same for new technology (mumbles)
do something revolutionary in the negative side as well
if there's (mumbles)
-: Swear we have to put on our archeological
hats -: Yeah (chuckling)
how about this one?
-: I mean I wouldn't say that
it's interesting to make that observation,
given our different components brought together
so I appreciate that piece of feedback
in terms of how these two different sets of observations
were presented here today
I as an archeologist I certainly think that these biases
have always been there and that what has changed,
in my mind is the scale of their impact right?
You know, I study a society that is maybe five,
10,000 people living congregated in a place
in North Central New Mexico, right?
That's a very different scale than the impact
of an algorithm that goes wrong upon 10s of millions,
if not billions of people.
(coughing)
(mumbling) (chuckling)
Female Student: (mumbles) this is really interesting
I have a question that juxtaposes my experience
with my son whose a computer science major
and he works in a lab and he has done this, he loves ai
and that's what he wants to do, (mumbles) he spent
a lot of the break in December and January doing this really
what I consider just kind homeless archaic (mumbles)
circling like the fish in the ocean
because the computer has to be taught to recognize
that this is a fish in the ocean and it's just so basic
and fundamental, to juxtapose that which just seems
really very fundamental, it seems so early
in the (mumbles) fish (mumbles) intelligence area
to all the concerned about like Google and Facebook
and Face Recognition Efforts at China for example
I mean it seems like the research (mumbles)
cause he's working in a research lab at the University
versus kind of this other stuff, there's such a huge
difference between those two and I get scared
when I hear about ai as it relates to the commercial side
in the government (mumbles) versus what I see
my son doing (chuckling)
(mumbles) and I just wonder if what your thoughts are.
-: I mean, there's probably various ways
to think about that, I mean,
there is a difference, often with the scale
of what's happening at a research level, right?
And what can be funded by large organizations.
I think part of that relates to when research,
I think, as scientists, we step back, we reflect
we often we'll take more care to look at our data,
to look at our algorithms, to look at what's happening,
and be clear about what's happening
rather than sort of the end goal is somewhat
maybe but maybe not the end goal is different
but the process is much different
and then often with larger corporations and organizations,
there's the profit maybe what they're searching for
when they're getting, you know, that's what they're trying
to get and so you often don't have that slow,
reflective, methodological iterative reflection
on what you're doing and you kind of throw it out
there and then that potentially can be it is more scary
and dangerous, right?
And so what we're doing and it relates to one
of the other questions in some senses
is we're letting the public and people
be our guinea pigs, right?
Oops that didn't work right.
So how many people were impacted when we come back
and go oops let's change that variable?
So that's I think one side.
-: The thing that, your comment made me think
about is that and I just know a little bit
about that but Google has gotten in trouble in recent,
last year so with regard to lower paid contract employees
versus full time, people with benefits, right?
And that there's just this huge army of Google
kind of contractual hourly workers
and my sense is that what Google has those people doing
in part on the algorithmic side
is doing exactly the same kind of training work
that you're son is working on
and that, you know, so there's this disparity
too in terms of the labor and the impact and the outcomes,
in terms of how this is being approached
by their staff, employees.
We had Antia, you had.
Antia: To seek to share
my skepticism about regulation ever being
a pathway in the US where people are notoriously
sort of skeptical about regulations
for some of the very reasons you've talked about
but half the watch groups in the US we made
any attempt to sort go that route or push
towards that or is the sort of concern that regulation
itself will sort of override
so it's too overpowering for them to even
it seems to be an area that cost calls
for some form of at least sort of writing it into law
that people provide open access and open sources right?
So then you could look and it would probably be
a disinsentive to people like Google
if they had to just make everything public
then it might not be yearly as profitable
to do all of this so (mumbles)--
-: I mean I'm heartened when somebody like Whittaker
is asked to come to Washington and testify right?
So I'm heartened by that, an expert
that's been on the Google commercial side
and is now on the non-profit advocacy side
so I find that heartening, I think also the case
that in my observation the place that we're seeing
regulation come to the floor is more at the state level.
There was a case in New York City about housing
discrimination using biometric recognition
and then there was a pushback and a task force
and kind of got close to regulation and then didn't
but I think there are cases about ai regulation making
it into a state law in Oregon, and in Michigan,
and of course in reaction to some things
that went terribly wrong.
-: Yeah (mumbles)
-: [Foreign Female Student] I'm curious
about what is the (mumbles)
the conversation that's come about before
about the fact that diversifying some fields
of work can help to prevent those biases
or like change the data that we put into the algorithms
so they're not bias and I was wondering, how far along
do you think that we are too diversifying more
those fields so we have less of those biases?
-: That's a great question and I think
that that would have some potentially positive impacts
if we were able to sort of bring in wider perspectives
and more diverse perspectives.
In terms of I think there's an awareness
of that you'll see headlines periodically
saying, you know Apple is looking for not simply
computer programmers, but computer programmers
with a cultural awareness, understanding,
a human humanities background, which is great
that's the first kind of at least there's awareness
on that level but it comes again, I think back to scale
in some sense it is when it comes down to sort of getting
one sort of getting the profit out and getting things done,
right, maybe mass producing sort of in the more traditional
way, but hopefully through time, policies
and cultural practices and organizational practices
will change right to be more diversified,
include more diverse biases through advocacy,
through regulation, through other kinds of means.
-: Yeah I'll just add that I mean, I don't have the numbers
(mumbles) are they're ready to be able
to give you a clear response
to how that's changed over time
and have we reached some sort of point that we're all happy
with right that we're feeling like people
are really represented at the highest levels of decision
making in these contexts
but I think that's one of the cases
where people within the general public have an opportunity
to educate themselves about what do those trends look like,
demand to have access to those kinds of numbers
and are the employees, the kind of contractual level
sort of within the Google model, right?
That these are hourly employees versus full time paid staff
like what are those numbers and demographics look like?
At those two different scales and I think
too that's where a lot of these outside research institutes
like AI Now and Ada Lovelace and others,
have a critical role to play in terms of gathering
those data, and helping to share
that with all of us so we know how far we have to go
and where are the areas that we need
to concentrate our efforts.
(coughing) Male Student: Sort of
(mumbles) you think (mumbles)
kind of come out or do you think it'll be kind of harder
or would be less difficult in the sense of changing
those or updating those algorithms due to people
who create in the sense of like the whole
racial discrimination (mumbles) like medical,
hiring like all the different (mumbles)
would you guys (mumbles) or (mumbles)
and keeping it current (mumbles)
-: (mumbles) I'm trying to understand the question.
So the question is then with more data,
do we, Male Student: So
-: Yeah sorry, thank you
Yeah, yeah thanks Male Student: (mumbles)
Sorry I (mumbles) right now
basically (mumbles) algorithms
in the first place, overtime with data like,
(clearing throat) updates and gets more accurate
per se, do you (mumbles) the algorithms
like in what they teach in the (mumbles) companies
do you think they'll be easier to regulate
or necessarily take those biases
away towards racial profiling towards (mumbles)
what the medical practices and everything?
Do you think it'll be easier over time
or it might be more difficult to do this
you know, practice it if they keep teaching
them the traditional (mumbles)
or making it better (mumbles)?
-: I think there's this strong potential with more data
and more inclusivity right?
Including more data for more diverse parts of the world
and diverse populations that potentially
some of the algorithms may in one hand, work differently
and of course have different outputs right?
With a more diverse data on the other side, yes
so as you're saying the algorithms
themselves potentially need to be changed
because the variables that are being used
and so I think that definitely, I would be more positive
than negative in the sense and I think definitely
the potential exists.
The question is where you're at in the world, potentially,
how, it might be used differentially in different countries
and based on different regulations,
social political systems, ideologies, et cetera right?
How that may influence the way algorithms are controlled
and who is operating them and to what extent
they have a large impact or not.
-: Heather wore yellow cause she's a more positive
person than me (chuckling)
and I wore black, so we color coded
ourselves, I guess my concern,
I'd like to be optimistic that yes, over time,
these things will improve and perhaps they will.
The challenge though, of course,
is that we're not allowed to know how these decisions
You know, what the black box is doing?
Then it's hard to know years hence, is it better?
Is it worse right?
And some of the cases that we profiled here today,
were highlighted only because people, you know, gathered
the original data and actually challenged
is this doing what people say it's doing?
If you go to the HireVue hiring page
and listen to the pitch
about what that assessment tool will do in terms of hiring
practices, it's gonna move heaven and earth right?
It's gonna equalize everyone,
you're gonna interview more people,
you're gonna diversify your workforce
and so on, how do we know? (chuckling)
How do we hold them accountable
that's actually what it's doing?
As opposed to simply creating,
as I said, an automated form of discrimination
in terms of who's getting hired
and who has the higher employability score.
-: Yeah
(mumbles)
-: Sure (chuckling)
(mumbles) (chuckling)
Student: (clearing throat) What is this litigation over
this topic (mumbles), I'm thinking about the (mumbles)
to play out the couple of ways to finding
genuine material when I wanna use
that (mumbles) in a corporation
and a professional association (mumbles) building
with it more liable to (mumbles)
so this is more like the development of these tools
is more than halfway to an (mumbles)
back in (mumbles) like medicine
or something like that may be an analogy
and so is this gonna be saddled with like malpractice
like lawsuits?
Or you see litigation going along that front
or is it never gonna get to litigation?
-: Yeah I think it's already being litigated
and I think that's where we're seeing Taantia's question,
the pressure for greater regulation or task force oversight
because in some of these high profile cases
you have, like the government themselves, the City Council,
or whomever throwing up their hands and saying,
" Well we didn't know, we didn't understand
how this is gonna work, you can't hold us responsible
because we were ignorant (chuckling) right?
Blame them, blame the third party vendor right?
And so I do think that that's part of the reason
why the folks for instance at Ai Now
are positioning these sought pieces
and co authoring with legal scholars
in places like the Columbia Law Review
to get that message out.
Archeologist: As a fellow archeologist,
I have a very general question,
(chuckling)
We are very interested in political systems and (mumbles)
that initially we thought (mumbles) all of these algorithms
(mumbles) equalizer as you said but then we see
there is a (mumbles) that couldn't leave (mumbles)
So I just wonder what are your thoughts
whether (mumbles)
(chuckling)
-: That's a big question.
(chuckling) -: Yeah it is a big question.
Firstly, I don't think I have any heavy observations
about that I guess I'm in the same spirit
as we mentioned a few minutes ago,
I guess I'm heartened that there are really smart people
who are holding governments accountable
and are pushing as hard as they can to try
to hold the political systems,
the powers that be accountable if and when they choose
to use algorithmic systems in terms of automating governance
and so I guess I'm hopeful that I don't know
I don't know if your question is leading
towards like do we think there's a revolution
coming? (chuckling)
I don't know but I guess I am taking solace from the fact
that there are a lot of smart people
that are working very hard
to hold these governments accountable,
-: I mean yeah, just along those lines,
I just think it's important to think about these things,
not yes is there's a global scale
to it but there's also local scales
of these in various countries throughout the world
that are not as big a players in this right?
And they have different social political systems currently
already and so I think when we think
about the impacts of algorithms on the political level,
we have to sort of delve a little deeper
into the more local scale to think
about how that impact may be at different places.
-: Yeah? -: Jim?
Jim: To what extent from your perspective
is ai limited by definition to what it could do in (mumbles)
it is, ai really is a product of binary logic
and to what extent are you limited to applying binary
logic (mumbles)
Is there a limitation
that defies the maximum use,
the maximum application of ai
as opposed to an assumption that (mumbles) out there that ai
can form us an (mumbles) on a scale
much wider than the (mumbles)?
-: That's a great, it's a great question.
I mean, I would say, just as a more personal response,
yeah, I think you're limited (chuckling) right?
I mean, I think you know, humans are not zeros and ones.
We're not binary and sort of proves the underlying theme
of part of this right?
Talk is that humans in the beginning are the ones
programming and creating these and so that's always going
to be sort of the underlying basis of it now that may morph
into many different ways depending on how algorithms
are rewritten based off of training sets
but I don't think that, I think
that they're never gonna fundamentally
if I'm getting your question right,
either, you know, replace humans or become the same
as human decision makers-- Jim: (mumbles)
More restricted to the question that ai,
the assumption is ai could be function (mumbles)
so unless limited to (mumbles)
what areas can ai truly beat function (mumbles)?
And what are the limitations
because of it's binary structure, to what extent
can it (mumbles) regardless of how much it continues
to develop (mumbles)
-: I don't know that we know the answer to that yet
because it's still relatively early days
in terms of the broad scale application
of these algorithmic systems in a whole host
of areas of life, you know I,
a couple of things come to mind
in response to your question,
I guess there was like a 1950s dystopian novel
you guys can correct me, I'm sure there are people
in the audience that are more astute
about this than I am by Alan Turing who talks about
how computer systems there's only a matter of time
before they are our overlords
and I do think based on the work that you know
the research that we have done that there is an expectation
out there that automation whether it's decision making
in governments or otherwise is only gonna continue
a pace as more and more investment goes
into these and the other thing,
I'll just say in response to your question Jim,
is there's something else that comes to mind
that you know, in developing these algorithms,
we as humans define objectives
and it's been the observations of computer scientists
that we are terrible (chuckling)
at defining those objectives and so we have an ill defined
objective that we feed into a computer system,
and then it kind of spirals out right?
And that's where this kind of enchanted determinism
no one's quite sure what's happening under the hood,
after more and more training data are going
in and the algorithms being refined over time
by the system in ways that we don't quite understand.
So I guess that's my response to your question
about what the limits are, I don't know
that we know yet.
Male Student: Previously (mumbles)
described (mumbles) is coming
and I think (mumbles) is no longer binary
(mumbles) (chuckling)
-: No longer limited by
-: Should we wrap? -: Oh Priscilla,
Priscilla: I'm wondering if you have some advise
for us as we are living in this age of social media
is probably not an option to go off the grid
(chuckling)
-: But-- -: Though tempting.
Priscilla: Practices are there things
from your perspective that we should or should not be doing
in order to expose us more or less
to the kind of (mumbles)
that might happen or are we so hopeless
(mumbles) down the road that and there's so much data
out there that it doesn't matter
how we (mumbles)?
-: You're wearing the yellow
sc-- (laughing)
-: I guess we (mumbles) the positive side here, well,
I do think that there is a lot more data
about all of us than we realize, right?
Out there that's being captured in many different ways.
I think that we shouldn't completely just throw up our,
throw the towels aside
but I think that we can...
Yeah I'm trynna get this positive spin
here on this one (chuckling)
-: I think that if I might interject,
-: Sure -: The
positive side is what we all do right?
Is education, in terms of trying to understand
how are these technologies impacting our lives,
what are the ways in which these are bubbling
up to the surface, whether it's in health care,
whether it's in city, state federal governance right?
So to educate ourselves about how those things are happening
and what some of those pitfalls are, I think
is really the first step.
Like you I don't know that there's any way to really stem
the tide in terms of what's coming
and what's already being utilized.
So I think the key will simply be education and advocacy
for the issues that we care about.
(mumbles)
-: Look into Taylor, Taylor (chuckling)
Female Student: Do you wanna take on more
or -: Sure
Female Student: With people starting to learn
coding and algorithm in earlier age, you know
couple of years ago (mumbles)
college (mumbles)
you know there are coding books for children
(mumbles) business, what do you think
is probably the most realistic
I guess progression of ai (mumbles)
(mumbles) -: The question is either
how far will it go?
-: Is that and there's other part of the question
is given that--
Female Student: Progression in the next several decades
or so with the (mumbles) more insight
about product programmings and algorithms (mumbles)
I guess more people if able
to program these, how far do you think
it will likely progress in the next (mumbles)
-: That's hard to answer
but I think that if you're thinking about the use
and sort of being engaged at earlier ages in these processes
I think it sort of reflects back on Carrie's point
sort of education and so if we, I don't know
about how far are they gonna go
but actually what can they, how can we tailor
them to a more positive kind of way, right?
And so it's the what they can do and where they can go.
Hopefully, we as ethical society,
and people can sort of control that a little bit
through education and awareness in some way so.
In terms of where they're gonna go you know
that is a hard question (chuckling)
-: And I know I'm heartened that there are more and more
young people who are growing up,
understanding and being more sophisticated
consumers of information and understanding the technologies
that underlie so many of these things we're interacting
with on a daily basis but like Heather,
I don't know that I would prognosticate
about where things might end up
but, you're the future (chuckling)
Student: Thank you so much for coming out today
and hoping to see you all at the end of March
for a panel discussion
(mumbles) (clapping)