Lilienfeld Honorary Webinar
March 2 webinar - Honoring the professional career of Dr. Scott Lilienfeld
Toggle between list and paragraph view.
- [00:00:00.000]Hello, and welcome to this
webinar honoring Scott Lilienfeld.
- [00:00:04.500]I am Kurt Geisinger, the director
of the Buros Center for Testing,
- [00:00:07.920]which has now been at the University of
Nebraska-Lincoln for the past 41 years
- [00:00:12.450]after having been
founded by Oscar Buros.
- [00:00:15.510]I am very pleased to welcome the speakers
and attendees to our webinar in honor
- [00:00:19.500]of the life and work of
Professor Scott Lilienfeld.
- [00:00:23.520]They honor professor Lilienfeld today
as does Scott's wife Candice Basterfield.
- [00:00:28.230]Since 1980
- [00:00:30.810]when the center came to the University
of Nebraska-Lincoln, Buros has had a
- [00:00:34.740]National Advisory Council and its
history of members includes many,
- [00:00:38.910]many of the luminaries of educational
and psychological testing and
- [00:00:43.530]measurement. Traditional membership
on the council has been two
- [00:00:48.030]educational measurement scholars and
one in either school or counseling
- [00:00:51.960]psychology, an industrial psychologist,
and a university librarian.
- [00:00:57.120]However, because the biggest area,
- [00:00:59.160]historically within the
Mental Measurements Yearbooks,
- [00:01:01.500]our flagship publication, is in the realm
of personality and clinical psychology,
- [00:01:06.510]we decided that we needed to add a
clinical psychologist with a specialty in
- [00:01:10.260]clinical testing to our
advisory council. I asked
- [00:01:14.460]Dr. Janet Carlson,
- [00:01:15.840]our resident clinical psychologist and
Buros associate director to provide me
- [00:01:19.740]with names of the most appropriate
clinicians in the country.
- [00:01:23.400]It should surprise no one on
this committee or to hear today
- [00:01:28.320]that Scott Lilienfeld was the
top name on the list. To abstract
- [00:01:33.270]the rest of the story I called him,
asked him to join our group and he said, "Yes".
- [00:01:38.370]He spent four years on our National
Advisory Council and he was incredibly
- [00:01:44.250]and well-informed. Whether with PhDs,
- [00:01:47.880]graduate students or
staff, all would report
- [00:01:50.730]he was warm and remarkably friendly,
and as all of you know,
- [00:01:55.950]he was also unbelievably
smart and knowledgeable.
- [00:01:59.550]He shared his knowledge with all
of us generously. In some cases,
- [00:02:03.570]and speaking for myself,
- [00:02:06.000]he became our friend both
personally and professionally.
- [00:02:09.990]We miss him. Because Scott shared our
commitment to the science and practice
- [00:02:14.940]of testing and measurement,
- [00:02:16.560]we thought it was fitting to acknowledge
his contributions to our work in the
- [00:02:21.390]preface of the 21st Mental Measurements
Yearbook just out this month.
- [00:02:26.130]I would encourage those who are in
attendance today to learn about this
- [00:02:29.370]cornerstone resource,
- [00:02:31.020]as well as the many other resources
Buros provides for researchers,
- [00:02:34.890]practitioners, and educators.
- [00:02:37.080]But today is to honor Professor
Lilienfeld who was
- [00:02:42.000]taken from us far too early. So again,
- [00:02:45.300]welcome to our webinar and with
no further ado, Dr. Jonson.
- [00:02:51.170]and I'd also like to welcome you to the
Buros Center for Testing's Honorary
- [00:02:54.890]Webinar for Dr. Scott Lilienfeld.
- [00:02:57.440]We appreciate you joining us and this
distinguished panel of presenters who are
- [00:03:01.390]helping us recognize the eminent
career contributions of Dr. Lilienfeld
- [00:03:05.380]and we're pleased and honored
to start our program today with an
- [00:03:09.520]introduction from Ms. Candice
Basterfield. The presentation,
- [00:03:13.960]so the four panelists will follow her
introduction and as time allows each
- [00:03:18.460]presenter will answer one to two
questions after the presentation.
- [00:03:22.660]We do ask you submit those questions by
clicking the Q and A icon at the bottom
- [00:03:27.130]of the screen. We've shared
several links in the chat.
- [00:03:30.940]One is a link of listing
of Dr. Lilienfeld's
- [00:03:33.640]most important scholarly work that was
put together by a committee for the
- [00:03:37.240]Society for the Science
of Clinical Psychology.
- [00:03:41.230]A second link will provide you access to
a web page where you can enter your own
- [00:03:45.460]tributes and testimonies about
Dr Lilienfeld and his work,
- [00:03:49.570]and we will publish and share
that along with the recording of this
- [00:03:53.530]webinar. We've also included
links to articles about Dr. Lilienfeld's
- [00:03:57.520]life and career from
Emory University and the New York Times.
- [00:04:02.560]So we thank you again for
your interest in this program.
- [00:04:05.590]and I now will turn it
over to Ms. Basterfield.
- [00:04:17.040]I want to thank Buros for inviting me
to talk about my late husband, Scott Lilienfeld
- [00:04:24.270]who passed away on September 30th, 2020.
- [00:04:28.140]He was 59 years old.
- [00:04:31.410]Scott fought a battle against a
virulent form of cancer for several months,
- [00:04:37.050]with courage and dignity and continued
working up until his very last days.
- [00:04:43.470]The field of clinical psychology
and psychology more broadly lost a
- [00:04:48.060]pioneer, a brilliant scholar,
original and independent thinker,
- [00:04:53.850]beloved mentor, colleague and friend.
- [00:04:57.570]More than anything else though,
- [00:04:59.490]Scott was a kind, caring
and thoughtful person.
- [00:05:05.010]Scott was born in Queens, New
York Thelma and Ralph Lilienfeld.
- [00:05:10.260]His mother was a homemaker and his father,
- [00:05:12.690]a radiologist. Scott's interest
in science was developed
- [00:05:17.550]from an early age when his father would
frequently take him to the American
- [00:05:22.350]Museum of Natural
History in New York City.
- [00:05:26.400]Scott succeeded in school and
was the valedictorian of his high school.
- [00:05:31.320]In his youth,
- [00:05:33.870]he was an excellent chess player and
would compete in chess competitions.
- [00:05:39.510]Scott obtained his bachelor's
degree from Cornell University in
- [00:05:43.740]1982, where he initially was
enrolled to study astronomy,
- [00:05:48.930]but later changed to psychology.
- [00:05:52.470]One of Scott's intellectual heroes
was the astronomer Carl Sagan who
- [00:05:57.410]happened to be teaching at
Cornell when Scott was a student.
- [00:06:02.060]Scott was lucky enough to get a brief
encounter with him at Cornell and later
- [00:06:05.990]actually met him formally and
was able to attend his lectures
- [00:06:11.390]After Cornell University,
- [00:06:13.160]Scott enrolled for his PhD in Clinical
Psychology at the University of Minnesota,
- [00:06:17.540]and studied under
the supervision of David Lykken.
- [00:06:23.960]His dissertation developed and
validated a measure of psychopathic personality,
- [00:06:27.830]called the Psychopathic
- [00:06:33.110]which has become the most widely cited
self-report measure of psychopathy
- [00:06:38.660]Paul Meehl,
- [00:06:39.950]another one of
Scott's intellectual heroes
served as a committee member
- [00:06:44.120]on Scott's doctoral dissertation.
- [00:06:46.700]Scott kept a letter from Paul
Meehl on his office desk saying,
- [00:06:50.450]in sum that Scott's thesis was excellent.
- [00:06:55.130]Scott's first academic position was
as an assistant professor at the State
- [00:06:59.600]University of New York at Albany. In 1994,
- [00:07:04.100]he moved to Emory University
where he was a Samuel Candler Dobbs
- [00:07:07.680]professor. Scott authored,
co-authored and co-edited
- [00:07:12.680]more than 350 articles and
book chapters and 20 books,
- [00:07:17.630]including a popular psychology
textbook currently on the market.
- [00:07:22.640]He published on wide ranging topics,
including personality disorders,
- [00:07:27.260]dissociative disorders, anxiety
disorders, psychiatric classification,
- [00:07:32.240]pseudoscience and psychology,
- [00:07:35.480]and evidence-based practice. Considering
his broad range of interests,
- [00:07:40.340]Scott was a generalist.
- [00:07:43.430]This is incredibly rare in
today's hyper specialist academy.
- [00:07:48.380]Scott was especially interested in the
application of scientific thinking to
- [00:07:52.100]clinical psychology,
- [00:07:53.750]and he played a major role in
distinguishing evidence-based from
- [00:07:57.560]pseudo-scientific practices,
- [00:07:59.810]as well as writing about poorly validated
psychotherapeutic and assessment
- [00:08:04.250]methods, such as pseudo-scientific
- [00:08:08.870]projective tests, recovered
memory therapy, lie detectors,
- [00:08:13.580]and facilitated communication.
- [00:08:17.180]Scott pursued some of the
thorniest questions in psychology,
- [00:08:21.410]and he did so with intellectual
humility and courage.
- [00:08:26.030]His work was considered controversial
in many corners of academia
- [00:08:30.500]and at times he was
- [00:08:34.250]Scott was very much committed to
improving psychological science.
- [00:08:40.550]Scott was also deeply committed to
disseminating psychological science to the
- [00:08:44.780]general public and his work
was featured in USA Today,
- [00:08:49.880]The Washington Post, Los Angeles
Times, Chronicle of Higher Education
- [00:08:55.860]and the New York Times to name
just a handful of outlets.
- [00:09:00.840]From 2006 to 2015,
- [00:09:03.420]he was also a regular contributor to
the Scientific American Mind where
- [00:09:08.400]he had a column on facts and
fictions and mental health.
- [00:09:12.630]This was later turned into a
popular book under the same name.
- [00:09:17.580]In addition,
- [00:09:18.780]he co-authored another popular psychology
book called the 50 Grit Myths of
- [00:09:23.460]Popular Psychology, which was
translated into 20 languages.
- [00:09:28.890]On a personal note,
- [00:09:30.600]I was lucky enough to have met Scott in
Australia when he was giving a talk on
- [00:09:35.310]evidence-based practice, and
I was taken aback by his sharp wit,
- [00:09:39.600]his warmth, his kindness,
and his intelligence.
- [00:09:44.040]I will always be honored
to be Scott's wife
- [00:09:48.240]His extra ordinary intellectual
humble and inspire me.
- [00:09:52.140]and I'm sure many of you watching this
try to emulate the clear thinking that
- [00:09:56.970]he embodied. Thank you.
- [00:10:11.870]Okay, good morning everyone.
- [00:10:15.530]My name is Martin Sellbom.
- [00:10:17.210]I'm a professor of psychology
at the University of Otago in
- [00:10:21.740]Dunedin, New Zealand. It's
morning here in New Zealand.
- [00:10:25.070]Good afternoon to those of
you in North America.
- [00:10:28.610]I just want to start out with saying
just how honored I am to be part of this
- [00:10:32.540]webinar and,
- [00:10:33.770]it's quite the humbling experience
to be reflecting on Scott Lilienfeld's
- [00:10:38.510]legacy. And I'm going to
do my best to
- [00:10:42.350]present this in a correct
manner as possible.
- [00:10:46.300]I want to start out with
just a quick personal note.
- [00:10:49.160]So, I first started
communicating with Scott in
- [00:10:55.040]I was a graduate student
at Kent State University
- [00:10:59.270]under supervision of a
- [00:11:02.870]who's listed here on this article
that I'm showing you. Now
- [00:11:06.230]one of the things that I was doing
was trying to find out a measure of
- [00:11:10.850]psychopathy that I could use for a new
study that I was designing on assessing
- [00:11:15.410]psychopathic personality
traits with the MMPI-2,
- [00:11:19.580]and I found the PPI that Candace just mentioned
- [00:11:24.920]and Yossef had gone to graduate school with
Scott at the University of Minnesota,
- [00:11:29.720]and he recommended
- [00:11:30.620]I send Scott an email and ask him the
various questions that I have. So I did,
- [00:11:35.690]I had absolutely no
- [00:11:37.610]thinking that he would get
back with me anytime soon,
- [00:11:41.300]But to all of those who know Scott,
- [00:11:43.480]you would not be surprised to hear that
within an hour, I had an email reply
- [00:11:48.110]multiple paragraphs and went well beyond
the answering the questions that I
- [00:11:52.700]had, and that really started
my communication with Scott.
- [00:11:58.870]Shortly after that I met Scott and
we've pretty much have collaborated ever
- [00:12:03.790]since. He's very much served as
- [00:12:07.890]research mentor to me. And along
with some important others, you see,
- [00:12:12.940]for instance, Chris Patrick
on this one as well, and
- [00:12:16.330]there's some others as well, but,
- [00:12:17.960]Scott has pretty much
always been there since day one.
- [00:12:23.830]I also had the fortune to continue to
collaborate with Scott over the years.
- [00:12:28.510]For instance, when I
was an independent academic, we
- [00:12:33.400]published papers, some led by my students
like Becca Kastner here, Bryan Neo,
- [00:12:38.200]others by his students like
Joanna Berg, Shauna Bowes,
- [00:12:42.370]just to mention a few
examples. We also
- [00:12:46.090]collaborated on various
- [00:12:49.600]big issues and topics in
the psychopathy field.
- [00:12:53.410]And I was quite fortunate to
revise his original chapter
- [00:12:58.180]on the self-report assessment
- [00:13:01.300]for the second edition of Chris
Patrick's Handbook of Psychopathy
- [00:13:06.160]along with his original coauthor
Katie Fowler as well. So,
- [00:13:11.170]we pretty much worked
- [00:13:15.340]until his passing. So this
- [00:13:20.260]has essentially two loosely
- [00:13:24.850]They might perhaps appear
a little bit disjointed,
- [00:13:27.790]but there is some logic behind
here. The first part is that
- [00:13:31.640]I want to share some of Scott's
contributions to psychopathy assessment,
- [00:13:36.340]as well as some of his thinking
with respect to psychopathy.
- [00:13:40.270]And I'll say personality disorder
construct. But the second part
- [00:13:44.920]which represents some work, I
was doing was Scott
- [00:13:50.050]the addressing misconceptions
and fallacies about psychopathy
- [00:13:54.820]assessment, something that I
know Scott cared deeply about.
- [00:13:59.710]So, you know,
- [00:14:00.340]for those of you who are less familiar
with the construct of psychopathy understand
- [00:14:04.630]it's a pretty wide audience.
- [00:14:06.610]Some might wonder what is psychopathy
or at least
- [00:14:09.820]what is the definition of psychopathy?
- [00:14:11.920]And I wish I could tell you because
there's a significant debate in the field,
- [00:14:15.910]really debates have been going on for
the past couple of centuries in terms of
- [00:14:19.870]how to exactly define this disorder. Now,
of course, there's some consensus,
- [00:14:24.630]I think most experts in
psychopathy would argue that
- [00:14:27.850]there's at least characteristics
that fall into broad domains of affective,
- [00:14:31.660]interpersonal and behavioral functioning,
- [00:14:33.550]like individuals who score high in
psychopathy measures tend to be lacking in
- [00:14:38.200]remorse, empathy,
tend to be callous,
- [00:14:41.590]Interpersonally, they're deceitful,
- [00:14:45.580]but also potentially,
- [00:14:49.660]They hate really the impulsive,
risk taking, irresponsible.
- [00:14:54.830]But there is some significant debate about
what sorts of characteristics really
- [00:14:58.910]constitutes the core of psychopathy,
- [00:15:01.370]as well as which characteristics are
truly part of psychopathy versus others
- [00:15:06.380]might be potentially,
- [00:15:09.930]in terms of the
assessments of psychopathy,
- [00:15:12.290]I would say this started with Bob
Hare's Psychopathy Checklist. This
- [00:15:16.520]was a monumental
contribution to the field.
- [00:15:20.600]Bob Hare developed
- [00:15:22.250]the Psychopathy Checklist essentially
as a research tool at first to
- [00:15:26.780]unify the operationalization
of this construct,
- [00:15:30.470]which was pretty much in
shambles at the time.
- [00:15:33.860]The Psychopathy Checklist was
revised in 1991 for clinical use
- [00:15:39.230]and essentially constitutes
- [00:15:42.710]20 items that reflect different features
of psychopathy that was meant to
- [00:15:47.540]capture Harvey Cleckley's
original conceptualization of
- [00:15:52.370]psychopathy that he wrote about
in the 1940s after studying their
- [00:15:57.260]psychiatric patients, who he
believed were psychopathic.
- [00:16:02.060]And it's a clinical rating scale.
- [00:16:04.310]One makes ratings on these items after
interview and review of file information
- [00:16:08.780]and in an institutional context.
- [00:16:13.610]There's a score
ranging from 0 to 40,
- [00:16:16.250]where 30 has been denoted
- [00:16:20.030]reflecting a diagnosis of psychopathy.
Factor analysis of the PCL-R,
- [00:16:25.100]initially revealed two
broad factors of
- [00:16:28.880]affective interpersonal traits
versus more behavioral proclivities,
- [00:16:33.470]but later worked as focused on four
factors underlying these items that
- [00:16:37.760]represent affective,
interpersonal lifestyle and antisocial
- [00:16:40.310]characteristics. Now there's
been some concerns expressed about the
- [00:16:45.080]PCL-R. I should mentioned that
these concerns are certainly not
- [00:16:49.890]held by everyone, and
- [00:16:52.460]there's some significant
debate about these topics,
- [00:16:55.250]but I think these are concerns
that Scott would have shared.
- [00:16:59.330]One of them being that the PCL-R
had migrated away from Cleckley's
- [00:17:03.860]classical description in the
sense that there were certain
- [00:17:07.160]characteristics and traits that were,
- [00:17:09.020]that are not well embedded within the
PCL-R like fearlessness and stress immunity
- [00:17:14.960]Aspects I know Scott felt were important to
the conceptualization of psychopathy.
- [00:17:19.820]Others have also complained that
there's an excessive emphasis on criminal
- [00:17:23.690]and antisocial behaviors within the PCL-R, and
- [00:17:28.610]that you're essentially merging
- [00:17:36.020]also consequences of a personality
- [00:17:40.700]the personality
trait structure in itself.
- [00:17:45.680]There's again some significant
debate there. Candace mentioned that
- [00:17:49.740]one of Scott's initial
major contributions to the
field was a Psychopathic
- [00:17:54.540]Personality Inventory,
- [00:17:55.950]which was part of his doctoral
dissertation and was later
published in the Journal of
- [00:17:59.700]Personality Assessment and
revise for clinical use in 2005.
- [00:18:04.440]And originally out 187 items
- [00:18:07.120]it was later shortened to 154 items.
- [00:18:10.140]And this test was meant to capture
the personality traits associated with
- [00:18:13.980]psychopathy, the emphasizing some
of the more behavioral outcomes,
- [00:18:18.480]but also that it could be used
in institutional contexts.
- [00:18:23.100]He also canvassed the literature and
- [00:18:25.680]tried to capture all important
aspects of psychopathy
- [00:18:29.050]including some traits
that go back to Harvey
- [00:18:32.370]Cleckley like fearlessness and
stress immunity. For instance,
- [00:18:36.150]as I mentioned earlier,
- [00:18:37.980]and a lot of work with a PPI that's really
been on elaborating on its high order
- [00:18:42.990]So essentially you're subjecting
these eight scales to factor
- [00:18:47.850]analyses that have yielded,
- [00:18:52.050]broad factors or really two broad
factors of fearless dominance
- [00:18:56.400]that has been also
referred to as boldness
- [00:19:00.360]in the psychopathy literature that
capture some of these classic,
- [00:19:04.590]Clecklien characteristics like
stress immunity, fearlessness,
- [00:19:08.940]social influence and
- [00:19:12.180]One broad factor called in a
- [00:19:15.690]which has some of the
- [00:19:19.170]psychopathy traits that are
perhaps less debated.
- [00:19:23.040]And then cold hardiness always
ended up on its own factor. Now,
- [00:19:27.660]even this structure has been somewhat
debated with some finding that it doesn't
- [00:19:32.640]quite replicate in every single
sample, but now most of the work
- [00:19:37.650]on the PPI is really focused on
the two factors from that
- [00:19:42.660]high order structure, fearless
dominance, and self-centered impulsivity.
- [00:19:47.700]And a lot of this work has informed
- [00:19:50.310]Scott's theoretical thinking
- [00:19:53.880]He viewed psychopathy as not being a
classical syndrome in which all of the
- [00:19:58.650]symptoms and traits would
- [00:20:02.580]show the phenotype.
- [00:20:04.320]But rather, he argued that it was a
constellation of a range of personality
- [00:20:08.880]traits from multiple higher order domains,
- [00:20:11.370]some of which are not highly correlated
with one another. And he believed that
- [00:20:15.660]these individual personality traits
from common trait models can be quite
- [00:20:20.280]useful in understanding differences
in both psychopathy theories and their
- [00:20:25.200]various operationalizations, including
different manifestations of psychopathy.
- [00:20:30.360]There's been a lot of literature to
suggest there's not just one psychopathy
- [00:20:34.590]manifestation but rather research
on subtypes have suggested that there
- [00:20:39.570]was at least two as some of which I've
been called primary versus secondary
- [00:20:43.680]psychopathy although there's been
other names published as well.
- [00:20:47.590]And different trait constellations
can be used to understand
- [00:20:52.090]these different manifestations. Now,
- [00:20:55.630]Scott believed that the fearless
dominance or boldness is a key aspect of
- [00:21:01.960]And this has been under some significant
debates. Scholars like Don Lynam,
- [00:21:06.010]Josh Miller, Stephen Hart, among others,
- [00:21:08.570]tend to disagree with this perspective
and have certainly published some
- [00:21:12.130]empirical data to support their
- [00:21:15.550]perspective and disagreement
with the stance.
- [00:21:18.820]But in representing Scout's thinking, it's
important to highlight the fact that
- [00:21:23.980]he strongly believed that this was
an important part of psychopathy.
- [00:21:28.900]And interestingly boldness as I'll just
refer to it from here out is generally
- [00:21:33.670]modestly correlated with
other psychopathy domains
- [00:21:36.430]which of course would be inconsistent
with a psychopathy syndrome,
- [00:21:39.850]but he argue that boldness would interact
with other personality or sorry
- [00:21:43.300]psychopathy domains to form a particularly
impactful trade constellation.
- [00:21:48.460]But he also argued that boldness in
itself in isolation would be unlikely to
- [00:21:53.080]reflect psychopathy. It really needs
to interact with other psychopathy
- [00:21:56.800]traits for psychopathic
manifestation. And in fact,
- [00:22:00.850]some individuals are high in boldness,
- [00:22:03.040]but not those psychopathy
traits might actually be,
- [00:22:07.570]excuse me generally
successful in adaptive
- [00:22:11.740]in some domains of their
- [00:22:16.270]Scott argued that psychopathy
represents a particular personality trait
- [00:22:20.710]configuration that is especially
- [00:22:24.590]And what he meant by
this was that, you know,
- [00:22:27.400]a lot of scholars in personality
disorder field more generally,
- [00:22:32.140]have talked about personality
disorders being defined as extreme
- [00:22:36.400]manifestations of normal
individual personality traits
- [00:22:40.060]where this could essentially create
thousands of potential
- [00:22:43.270]constellations of traits that
would reflect personality disorder.
- [00:22:47.290]But why have not all of these different
constellations been observed
- [00:22:52.180]in clinical practice,
in other areas.
- [00:22:56.230]And Scott argued that
that's because there's some
particular constellations that
- [00:23:00.850]are particularly
- [00:23:04.390]And he wrote about this general
perspective quite recently,
- [00:23:08.110]representing it does an emergent
- [00:23:12.490]And he argued that this
- [00:23:14.920]trait constellation that includes
boldness would essentially
- [00:23:19.420]generate a manifestation that
can be adaptive on the surface.
- [00:23:23.720]Certainly look like they're leading
generally good lives
- [00:23:28.390]are superficially successful,
are free of mental illness and
- [00:23:33.160]other problems, but in the long
term very much
- [00:23:37.690]show maladaptive nature
and the harm to others
- [00:23:42.410]in society.
- [00:23:45.760]And some of this perspective
- [00:23:49.520]research into both adaptive and successful
psychopathy. For instance,
- [00:23:54.530]one study looking at psychopathy
in former U.S. presidents.
- [00:23:58.880]Or Waldman is a part of that study.
Maybe he will mention it in his talk
- [00:24:03.650]later on. But essentially in
- [00:24:06.860]the short amount of time
that I have hopefully
- [00:24:11.420]that does some of Scott's
thinking about this
- [00:24:14.870]very interesting construct justice.
Now the second part that I want to
- [00:24:19.880]share with you is
- [00:24:22.410]addressing misconceptions and fallacies.
- [00:24:25.430]I think Scott was the ultimate
mythbuster in psychology. He
- [00:24:29.570]clearly enjoyed writing on this
topic, not just on psychopathy,
- [00:24:33.740]but in psychology in general and
Candace spoke to this earlier.
- [00:24:39.050]And, it
was also represented in
- [00:24:43.100]the last work that I
authored with Scott
- [00:24:46.400]along with two other important colleagues,
Rob Latzman and Dustin Wygant.
- [00:24:52.180]And this was truly a team effort and
we very much enjoyed working on this
- [00:24:56.270]chapter that is about to
come out in psychopathy,
- [00:24:59.540]its uses, validity and status.
- [00:25:02.210]I believe the whole volume
is being dedicated to Scott.
- [00:25:06.080]And within here we addressed some
myths, misconceptions and fallacies.
- [00:25:10.040]I do not have time to talk
- [00:25:13.230]that we included in this
chapter, but I will
- [00:25:16.430]talk about some things I
know Scott in particular
- [00:25:20.360]felt strongly about.
- [00:25:22.190]And these were posed as questions and
we'll just dive into certain measures,
- [00:25:26.330]and standards for assessing
- [00:25:29.660]psychopathy, this major pet peeve
of Scott's especially since
- [00:25:34.520]frequently people would talk
about certain measures
- [00:25:37.700]being the gold standards for detecting
psychopathy with the PCL-R being the most
- [00:25:42.680]frequently heralded in this way.
- [00:25:44.960]For instance, Evans and Tully wrote
PCL-R as the sole tool of choice
- [00:25:49.910]for psychopathy measurement.
If you go to Google scholar,
- [00:25:53.180]you'll find plenty of hits
referring to PCL-R as a gold
- [00:25:58.010]standard. But also the
PPI, Scott's own measure,
- [00:26:02.870]had been referred to as the gold
standard self-report psychopathy measure
- [00:26:07.250]something that they also
strongly disagreed with.
- [00:26:10.190]And part of the reason for this is because
there are no genuine gold standards
- [00:26:14.390]for any construct in psychology
- [00:26:17.660]because psychological constructs by
definition are latent entities that we
- [00:26:21.710]can't directly observe.
So any approximation
- [00:26:25.280]of psychological construct
are by definition fallible.
- [00:26:29.390]So there's no perfect
- [00:26:32.600]this would also violate the principles
of construct validity that were
- [00:26:36.650]articulated by Lee Cronbach
and Paul Meehl for instance.
- [00:26:40.460]Paul Meehl as Candace mentioned,
certainly being one of
- [00:26:43.920]Scott's heroes in many
respects in terms of intellectual and
- [00:26:48.690]scholarly thinking. And also when
we think about construct validation,
- [00:26:53.250]it's an inherently ongoing
and self-correcting process,
- [00:26:57.000]meaning that we continuously learn
new things about our measures,
- [00:27:00.780]and it can never really be regarded as
final. It's not an absolute property.
- [00:27:05.460]And another important thing
is that construct validation
is also directly linked
- [00:27:09.120]to theoretical frameworks.
- [00:27:11.210]So you can essentially have two
scholars who have different theoretical
- [00:27:15.480]perspectives, look at the
same data for a measure,
- [00:27:18.750]and actually come to different
conclusions about construct validity.
- [00:27:23.220]And I think we see a fair bit of that
in the field of psychopathy assessments.
- [00:27:28.050]And there's also no meta analytic
evidence of the PCL-R for instance,
- [00:27:31.530]consistently outperforms other widely
used psychopathy measures with respect to
- [00:27:35.430]convergent and discriminant validity
- [00:27:37.890]with relevant external criteria.
- [00:27:41.310]A second question is antisocial
personality disorder an
- [00:27:44.640]adequate operationalization of
psychopathy. For instance,
- [00:27:48.420]Ben Karpman who wrote about psychopathy
early on wrote "It's about time that
- [00:27:53.370]we cease making psychopathy, criminal
and antisocial behavior identical and
- [00:27:59.370]which very much would have
implications for the States today. Now,
- [00:28:04.230]psychopathy of course
- [00:28:05.220]was always the target construct for
the criteria underlying antisocial
- [00:28:08.790]personality disorder. If you look
in DSM I and DSM II, for instance,
- [00:28:13.600]it's very much reflected the core
psychopathic personality traits. In fact,
- [00:28:17.430]it was referred to as sociopathic
- [00:28:21.570]but it was really with the DSM-III
and onwards that things changed.
- [00:28:25.920]A sociologist by the name of Lee
Robins had a lot of influence in the
- [00:28:30.420]shaping of the diagnostic criteria
of antisocial personality disorder,
- [00:28:34.560]and it dramatically shifted towards
a more behavioral operationalization.
- [00:28:39.330]In parts, they argued
for inter-rater reliability,
- [00:28:43.950]but also in part, because these were
deemed the most predictive
- [00:28:48.300]traits associated with a
disorder in terms of
- [00:28:52.830]outcomes. Now,
- [00:28:54.810]most experts in the psychopathy
field would likely argue that the DSM III
- [00:28:59.580]through DSM V criteria for
antisocial personality disorder are
- [00:29:03.780]certainly inadequate with respect to
operationalization of psychopathy,
- [00:29:08.910]at least in so far as the most
dominant psychopathy theory go.
- [00:29:12.780]I suppose if you adhere to Lee
- [00:29:16.890]on this disorder, you would argue
that the ASPD might
- [00:29:20.400]be associated with a good
- [00:29:23.850]But nevertheless, research has shown that
- [00:29:26.160]if you look at
prevalence rates in prison
- [00:29:28.020]samples of offenders for instance,
about 50 to 80% meet criteria for
- [00:29:32.130]anti social personality disorder.
If you use the PCL-R,
- [00:29:35.520]which as we have established before is
a certainly well-supported measure for
- [00:29:40.170]psychopathy and use that cut score of
30 that we'll get to in a minute,
- [00:29:44.680]about 15 to 25% would meet
criteria for the same
- [00:29:49.270]putative disorder. There's also been
some neurological differences shown
- [00:29:53.770]when you use
- [00:29:55.000]antisocial personality disorders
versus other psychopathy measures.
- [00:29:58.840]For instance, works by Luke Hyde
and colleagues have shown this.
- [00:30:02.920]Is it scientifically acceptable to rely
on psychopathy total scores? Certainly,
- [00:30:07.990]numerous research do
this. There was
- [00:30:10.510]a meta analysis by Poeppl and colleagues
that looked at fMRI correlates of
- [00:30:14.290]psychopathy without looking
at the self-dimensions.
- [00:30:18.850]There are also the brief measures of
psychopathy like the Dark Triad measures,
- [00:30:23.360]and the sort that don't provide
information about the sub dimensions of
- [00:30:28.810]But this is problematic because using
total scores can obscure and dilute
- [00:30:32.920]potential important associations
among the sub dimensions.
- [00:30:37.660]And we can look at the PCL-R
for instance,and trait anxiety,
- [00:30:41.320]where the old factor, the more
behavioral factors tend to positively
- [00:30:45.070]correlated trait anxiety whereas
affective interpersonal factor was
- [00:30:49.090]a weekly negatively correlated
with trait anxiety. And in fact,
- [00:30:53.020]in many cases,
- [00:30:53.740]a sub dimension psychopathy measures
do fractionate opposing directions in
- [00:30:58.240]terms of associations with
external criteria. And
- [00:31:02.030]this would of course,
- [00:31:02.830]result in near zero associations for
a total score and also potentially
- [00:31:07.720]theoretically important reciprocal
suppressor effects cannot be observed if you
- [00:31:12.340]rely on total scores.
- [00:31:15.130]Is it scientifically acceptable to
rely on psychopathy cutoff scores?
- [00:31:18.820]Certainly some manuals like the PCL-R
manual suggests a score of 30 out of
- [00:31:23.350]40 to suggest psychopathy. Now
the manual itself does
- [00:31:28.400]provide some caution about this.
- [00:31:31.360]But it's still potentially problematic
to use a score like this
- [00:31:35.880]to diagnose psychopathy cause it
does happen in clinical practice.
- [00:31:39.880]Some other authors use lower cut scores
for women or Europeans for instance,
- [00:31:44.020]and researchers
- [00:31:45.490]would often distinguish
between psychopaths and non-
psychopaths using this
- [00:31:50.260]sort of cutoff score.
- [00:31:52.900]Now there's little
- [00:31:56.140]for this sort of practice.
The PCL-R for instance,
- [00:32:00.940]the original score was based on one
standard deviation from the mean
- [00:32:03.910]on an offender population.
- [00:32:05.620]Nothing magical happens at a cut score
of 30 and there's no taxonomic
- [00:32:10.120]analysis that would look for
taxons or qualitative,
- [00:32:14.920]categorical constructs that would
support this idea that there is a
- [00:32:19.870]clear, qualitative
- [00:32:23.620]disorder underlying these measures.
- [00:32:26.230]And there's a number of issues with
dichotomization from a statistical
- [00:32:29.980]perspective that I'm listing here, but
because I'm quickly running out of time,
- [00:32:34.780]I'm going to move on to some other things.
- [00:32:38.740]And the final thing I want to talk
about are the PCL-R and allied measures
- [00:32:42.650]unparalleled predictors
- [00:32:45.860]Many assume that the PCL-R and
its variants are distinctively
- [00:32:48.920]if not uniquely suited for this
sort of prediction. One writer,
- [00:32:53.090]for instance,
- [00:32:53.630]recently described the PCL-R's ability to
predict violence as quote "unprecedented
- [00:32:58.040]and unparalleled". Now,
- [00:33:01.520]in saying that, the PCL-R
is indeed the consistent,
- [00:33:04.640]robust predictor of criminal
violence and recidivism.
- [00:33:07.250]We're not trying to argue otherwise,
- [00:33:09.140]but the evidence is uniquely
talented in this regard.
- [00:33:12.620]It's essentially non-existent. And in fact,
- [00:33:15.080]Singh and colleagues showed in the meta
analysis that the PCL-R ranked among the
- [00:33:19.190]lowest at predicting risk for general
offending and fell below a number of other
- [00:33:23.510]widely used measures and
variables for this purpose.
- [00:33:26.630]It's also been a recent
statement of concern. Experts
have questioned the utility
- [00:33:30.680]of using the PCL-R for risk assessment
in death penalty cases because of the
- [00:33:35.360]prediction of institutional
violence and aggression. Though,
- [00:33:38.570]there's certainly been some debate
about this issue as well
- [00:33:41.760]that that needs to be highlighted.
- [00:33:44.120]The most predictive utility
of the PCL-R for instance
- [00:33:46.700]resides with the anti-social facets
- [00:33:48.890]which is the most debated facet among
the PCL-R facets to begin with.
- [00:33:53.870]And might really just represent Meehl's maxim
that the best predictor of
- [00:33:57.980]future behavior tends to be past
behavior with other traits being less
- [00:34:02.000]predictive. So, in summary
- [00:34:05.090]there's many more myths about psychopathy,
some that were described, and
- [00:34:09.620]others that have been
described in the past.
- [00:34:12.320]I really encourage others to just rely
on psychopathy assessment measures that
- [00:34:17.790]robust construct validity for
the purposes and populations with which
- [00:34:22.730]they should be used.
- [00:34:23.630]And this recommendation is really no
different for any other psychological
- [00:34:28.250]And I think the field really needs to
move beyond the idea that one measurement
- [00:34:32.180]modality is inherently
superior to others,
- [00:34:34.790]and really focus on which measure best
maps onto the theoretical
- [00:34:40.100]perspective to which one prescribes.
- [00:34:42.830]Sorry for the rush there towards the end,
- [00:34:45.530]but hopefully this gives you
some ideas into a Scott's
- [00:34:50.720]thoughts about psychopathy
and some of the
- [00:34:53.150]concerns that he had expressed
about this literature. It's time for questions.
- [00:34:57.880]Thank you, Dr. Sellbom.
- [00:34:59.080]I don't have a lot of
time for questions and I'm
not sure if we have anything
- [00:35:02.950]that's come in yet. I'll just ask,
- [00:35:05.950]is there anything in particular
that you see as important next steps
- [00:35:10.600]worth the research on this,
- [00:35:12.130]in terms of what measures do you
use and how to use those measures?
- [00:35:16.570]Well, I think that Scott's
ideas of thinking about
- [00:35:21.400]psychopathy and any personality
disorder as a reflecting a
- [00:35:25.810]constellation of personality
traits, is really
- [00:35:30.550]the way to move forward.
Personality disorder and
- [00:35:34.090]science more generally,
in this assessment,
- [00:35:37.260]they're all certainly
moving in that direction.
- [00:35:39.930]There's lots of important scholars, not
just Scott who think this way and have,
- [00:35:44.760]certainly published a lot of
- [00:35:48.900]that would guide us in this direction.
- [00:35:51.090]But I do think that we need to
move away from psychopathy as some
- [00:35:56.070]unitary umbrella construct
- [00:35:58.560]and instead just focus
on the underlying components
and understanding those
- [00:36:03.360]some of which are relevant to other
personality disorders that we have
- [00:36:08.340]labeled and not just psychopathy. If we
look at disinhibition, for instance,
- [00:36:13.370]which is a broad trait
domain, it's relevant
- [00:36:16.600]to understanding psychopathy,
- [00:36:18.150]but it's also relevant to understanding
other manifestations of personality
- [00:36:22.020]disorder like I say,
borderline personality disorder.
- [00:36:25.740]So I think we need to kind of move
away from, from some of these labels.
- [00:36:29.640]And instead of trying to understand
- [00:36:32.220]maladaptive expressions of
these personality domains
much more so than
- [00:36:37.020]holding on to what I
think are becoming agent constructs.
- [00:36:41.970]Very good. Thank you. All right.
Well, we have a full program for you,
- [00:36:46.830]so we're gonna move on to
our next presentation. Thank
you, Dr. Sellbom and
- [00:36:51.510]Dr. Clark, you are up now.
- [00:37:00.320]See if I can get this going properly.
- [00:37:13.160]Did I do it right this
time? It looks good.
All right, thanks.
- [00:37:18.470]Well, first I want to thank the Buros
Center for testing for inviting me to
- [00:37:21.740]contribute to this webinar
honoring my late friend,
- [00:37:25.040]Scott Lilienfeld. I'm humbled
and honored to be able to
- [00:37:29.270]participate. Given that one
area to which Scott contributed
- [00:37:33.770]significantly, particularly
with regard to assessment,
- [00:37:36.740]as has already been made very clear,
was psychopathic personality,
- [00:37:40.880]I chose for my talk today to present
information regarding assessing
- [00:37:45.080]psychopathic personality
traits and behaviors using
the SNAP-2, the Schedule for
- [00:37:50.060]Non Adaptive and Adaptive
Personality, second edition.
- [00:37:53.630]This is partly because I am far
from a psychopathy expert as some of
- [00:37:58.640]those who are speaking today are so I
wanted to speak about something that
- [00:38:03.860]I at least had some knowledge rather
than going out in a whim and
- [00:38:08.630]hoisting myself by my own.
- [00:38:13.460]First, a conflict of interest
disclosure. I'm the author,
- [00:38:16.100]copyright holder of the
SNAP family of measures.
- [00:38:21.110]The various measures in the SNAP family
are all freely available for unfunded,
- [00:38:25.400]non-commercial research and
non-profit clinical work.
- [00:38:30.170]In all other cases,
- [00:38:31.250]I negotiate a mutually acceptable fee
with the user and use the funds to support
- [00:38:35.620]student research and in
all cases that require a user
- [00:38:40.300]license. So,
- [00:38:42.580]first I want to take a minute to
mention my relationship with Scott.
- [00:38:45.880]He and I have a number of connections
that suggest some similarities between us.
- [00:38:50.470]You heard from Candace, Scott
obtained his degrees from Cornell and
- [00:38:55.450]the University of Minnesota
- [00:38:58.690]I also obtained my bachelor's degree from
Cornell and my PhD from the University
- [00:39:03.370]of Minnesota.
- [00:39:04.120]We also both worked some with [inaudible]
and of course we were both interested
- [00:39:08.200]in personality pathology. However,
- [00:39:10.780]we never overlapped at either institution.
- [00:39:15.280]Related to our common interest,
- [00:39:16.960]The first article of Scott's that I
ever read was the relationship of
- [00:39:21.580]histrionic personality disorder to
anti-social personality and somatization
- [00:39:29.670]The comorbidity of histrionic personality
disorder, often abbreviated PD,
- [00:39:34.380]and anti-social PD is now well
documented as is the association between
- [00:39:39.240]what used to be called hysteria, now
histrionic PD and somatization disorder.
- [00:39:44.820]Although neither relation was established
when this paper was published,
- [00:39:49.020]this paper posited
- [00:39:50.100]the intriguing hypothesis that in
individuals with histrionic PD/hysteria
- [00:39:54.900]males were more likely
to have comorbid anti social PD
- [00:40:00.300]and females co-morbid
- [00:40:04.110]a hypothesis is that continues to be
of some interest to this day. Of note,
- [00:40:08.460]the senior author of the
paper was Hagop Akiskal,
- [00:40:11.670]a well-known psychiatrist and longtime
editor of the Journal of Affective
- [00:40:17.520]When Scott published this
paper in a premier journal,
- [00:40:20.790]the American Journal of Psychiatry with
well-known researchers as co-authors,
- [00:40:25.800]he was a graduate student.
- [00:40:28.710]I was an assistant professor whose only
first authored publication was a chapter
- [00:40:33.450]in my advisor's book.
- [00:40:37.050]Despite my lack of
credentials at the time,
- [00:40:40.110]I guess I was smart enough to recognize
Scott as someone who would rise to
- [00:40:44.190]eminence and perhaps needless to say,
I've followed Scott's work ever since.
- [00:40:50.820]So I'm going to transition here
to the main substance of my talk,
- [00:40:53.820]the Schedule for Non Adaptive
and Adaptive Personality,
- [00:40:56.970]second edition or SNAP-2,
- [00:40:59.490]and its relationship with psychopathic
personality traits, including of course
- [00:41:04.260]those assessed by Scott's
Psychopathic Personality Inventory,
- [00:41:07.950]as well as behaviors in the
psychopathy domain. First,
- [00:41:12.780]a quick introduction to the SNAP.
- [00:41:14.370]It assesses 15 traits relevant
to personality pathology,
- [00:41:18.150]including for example, mistrust.
- [00:41:21.210]The sample item is "I'm sure I'm
being talked about". Exhibitionism,
- [00:41:25.530]"I love to have my picture
taken" and impulsivity,
- [00:41:28.830]"I often act without thinking". The scales
were developed by combined content and
- [00:41:33.770]factor analysis. Factor analytically
derived measures have the advantage over
- [00:41:37.850]scales that were developed
using empirical keying
- [00:41:41.180]or that were rationally
derived. Now their items
- [00:41:44.360]generally cohere better and
form single-factor scales,
- [00:41:47.420]or have a hierarchical structure as
when a broad domain scale encompasses
- [00:41:52.010]narrower facets.
- [00:41:54.470]The full version of the SNAP has 390
items that were developed based on both
- [00:41:59.060]DSM and other PD criteria.
There are also short, brief
- [00:42:03.560]and very brief forms of the SNAP as well
as informant versions of all lengths.
- [00:42:09.500]In addition to the 15 traits,
- [00:42:11.870]the full version only has six validity
scales to detect various forms of
- [00:42:16.490]invalid responding and 10 diagnostic
scales keep to the DSM IV
- [00:42:20.930]PDs, or if you will
- [00:42:22.910]the DSM-V section two PDs
whose criteria are identical.
- [00:42:27.830]The SNAP's 15 traits form
three broad temperament domains.
- [00:42:31.790]The first being negative affectivity (AKA
neuroticism and negative emotionality)
- [00:42:37.280]with negative temperament being the name
of the scale in the SNAP that assesses
- [00:42:41.030]the affective core of
this broad domain. Second,
- [00:42:44.540]just positive affectivity (AKA
extroversion and positive emotionality)
- [00:42:49.580]with positive temperament being the
name of the scale that assesses the
- [00:42:52.730]affective core of this broad domain.
- [00:42:54.890]And third is disinhibition versus
constraint called psychoticism by Eysenck
- [00:43:00.110]and assessing both conscientiousness
and agreeableness at the opposite end
- [00:43:05.030]of the five factor model
of personality. Now,
- [00:43:08.500]many of you will recognize this as
Eysenck's three factor model with negative
- [00:43:12.650]affectivity being neuroticism and positive affectivity
being extroversion and disinhibition,
- [00:43:19.280]And some of you will also know that Eysenck's
psychoticism scale would be better
- [00:43:23.570]named psychopathy. Alternatively,
- [00:43:27.650]the SNAP can be thought of as measuring
four of the five scale domains of the
- [00:43:31.610]consensual five factor
model with disinhibition
- [00:43:34.280]combining agreeableness and
conscientiousness. In a later
- [00:43:38.690]development, scales keyed more directly
to positively appraised (inaudible) would
- [00:43:43.190]develop, except for openness, because SNAP
doesn't have the relevant item content.
- [00:43:49.190]some researchers have argued that
psychoticism is a pathological form of
- [00:43:52.940]openness. And if you buy into that theory,
- [00:43:57.890]you can assess that content
using SNAP's eccentric perception
- [00:44:02.210]scale. Besides negative temperament,
other scales in the negative affectivity
- [00:44:06.710]domain are mistrust,
- [00:44:10.070]self-harm, eccentric
- [00:44:11.750]perceptions and dependency.
Besides positive temperament,
- [00:44:15.740]other scales in the positive
affect domain are exhibitionism and
- [00:44:20.540]entitlement with detachment
on the other end.
- [00:44:24.590]And besides disinhibition, other scales
in the domain are impulsivity with the
- [00:44:29.330]constraint being
propriety and workaholism.
- [00:44:33.990]Very quickly, some
basic psychometric data.
- [00:44:37.080]The SNAP scales are internally consistent
with average alphas in the low to
- [00:44:40.740]mid eighties from mid to high seventies
at the low end and high eighties to
- [00:44:45.150]nineties at the high end. Interestingly,
- [00:44:47.670]a different scale was the
lowest in each of these samples
- [00:44:50.430]so there's no one scale that is generally
less internally consistent than the
- [00:44:56.370]whether that appears to be sample
specific. In the short to medium term,
- [00:45:01.170]scales are temporarily stable.
- [00:45:03.270]And even in a sample of patients in
treatment for three to six months,
- [00:45:06.870]they're reasonably stable.
- [00:45:09.570]A key property of the SNAP is that
despite having a higher order structure,
- [00:45:13.950]it's lower order scales are generally
independent with an average inner
- [00:45:18.750]scale correlation of around 0.2 and the
highest correlations topping out around
- [00:45:23.730]0.6. Thus,
- [00:45:25.290]the measure truly gives you 15 relatively
independent pieces of information
- [00:45:29.400]about individual's personality
- [00:45:32.790]regarding the SNAP's factor structure,
- [00:45:34.740]most of the negative affectivity factor
scales load on the first factor with
- [00:45:38.370]some cross loadings onto the other two,
- [00:45:41.550]especially the disinhibition
factors. By the way, C A T P here
- [00:45:46.440]stand for college, adult,
teens, and patients.
- [00:45:49.650]These data are from the same samples
that was showing earlier. Similarly,
- [00:45:54.450]the positive affectivity factor scales
load on the second factor with a few
- [00:45:58.590]cross loadings,
- [00:46:00.210]and finally, the disinhibition scales load
on the third factor with the strongest
- [00:46:04.560]cross loadings on negative
affectivity in patient samples,
- [00:46:08.250]in which there's a stronger representation
of the higher order dimension
- [00:46:11.550]originally termed simply alpha
more often now called stability,
- [00:46:16.560]which is composed of neuroticism,
low agreeableness, antagonism,
- [00:46:20.610]and low conscientiousness,
- [00:46:22.470]or in terms of the SNAP, negative
affectivity and disinhibition versus
- [00:46:28.320]Now on to relations between
the SNAP and Scott's
- [00:46:33.060]Psychopathic Personality Inventory.
- [00:46:36.630]Two large samples of undergraduates
completed the SNAP-2
- [00:46:39.840]or GTS, a form of the SNAP that contains
only the three core temperament scales,
- [00:46:45.120]the PPI,
- [00:46:46.590]and one or more other measures. I've
been asked to contribute a chapter to a
- [00:46:50.760]festschrift volume dedicated to Scott,
- [00:46:52.890]and I plan to use these data
in that chapter. Today, however,
- [00:46:56.580]given our time constraints,
- [00:46:58.230]I'm going to present data
mostly from the first sample.
- [00:47:01.230]I'm going to critique the popular
two factor scoring of the PPI
- [00:47:06.090]that Martin Sellbom introduced in his
talk in a way that I like to think Scott
- [00:47:10.830]would appreciate,
- [00:47:12.270]and I'm not going to discuss the
other measures that we collected.
- [00:47:16.650]We wanted to find a structural level
between the two factor higher order level
- [00:47:20.970]most commonly used, the one that Martin
introduced, and the original eight
- [00:47:25.680]scale structure. So we conducted
an item level factor analysis.
- [00:47:30.880]First, we replicated the two factors,
scale level structure using items,
- [00:47:35.770]convergent validity
correlations were 0.93
- [00:47:38.710]and 0.95 respectively in the two
samples and discriminate correlations
- [00:47:43.330]were similar to the 0.15 correlation
between the two level factor scales.
- [00:47:47.760]So we felt that we had a replicated the
two factor level and then continued to
- [00:47:52.570]extract factors and found the
four-factor level to be optimal.
- [00:47:57.940]Two factor fearless dominance
split into fearless non-conformity
- [00:48:03.460]and social dominance.
- [00:48:05.590]Whereas self-centered
impulsivity split into irritable
- [00:48:10.270]disinhibition and thoughtful
dependability with fearless
- [00:48:14.470]nonconformity having a secondary cross
loading on self-centered impulsivity.
- [00:48:20.320]We hypothesized that what was happening
at the four-factor level is that the
- [00:48:24.430]more pathological range and the more
normal range content in each of the higher
- [00:48:28.480]order scales was
splitting into two scales.
- [00:48:32.470]Incidentally, using absolute
- [00:48:36.490]inter scale correlations on the four
mid-level factor scales ranged from
- [00:48:40.990]0.07 to 0.31 so they're
- [00:48:46.390]Next, we examined relations of the two
higher order and four mid-level factor
- [00:48:51.350]scales with the SNAP or
GTS temperament scales.
- [00:48:56.260]S(1) is the smaller of the two
samples I mentioned earlier,
- [00:48:59.830]and S(2) is the larger. For clarity,
only correlations of at least
- [00:49:04.780]0.3 are shown.
- [00:49:06.910]You can see here that of
the four mid-level scales,
- [00:49:10.930]negative temperament correlates
only with irritable disinhibition,
- [00:49:15.400]whereas it correlates with both
of the higher order scales,
- [00:49:18.550]at least in one of the samples.
- [00:49:21.490]Conversely, positive temperament
correlates most strongly and positively
- [00:49:26.350]with social dominance at the four
factor level with moderate positive
- [00:49:30.760]correlations with
- [00:49:33.910]and only with fearless dominance
at the higher order level.
- [00:49:39.010]These results may then some support
to our hypothesis that social
- [00:49:43.360]dominance at least, reflects
more adaptive pro-social traits
- [00:49:48.280]than does fearless non-conformity,
- [00:49:51.040]which I think Martin was
alluding to in his talk.
- [00:49:55.360]It's less clear whether either irritable,
- [00:49:57.220]disinhibition or low thoughtful
dependability are more pro-social.
- [00:50:02.170]Turning to disinhibition
- [00:50:04.840]which we would expect to be more
strongly correlated with psychopathic
- [00:50:08.860]personality traits than either
negative or positive temperament,
- [00:50:12.640]we can see that it
correlates with everything,
- [00:50:15.100]but social dominance and relates more
strongly to fearless non-conformity
- [00:50:20.320]and negatively to
- [00:50:26.500]than with irritable disinhibition,
- [00:50:28.400]which you'll recall was related strongly
to negative temperament. At the higher
- [00:50:33.230]order level,
- [00:50:33.980]it also correlates more strongly with
self-centered impulsivity than with
- [00:50:37.640]fearless dominant,
- [00:50:39.050]suggesting that of those two
factors self-centered impulsivity,
- [00:50:43.880]maybe the more maladaptive.
- [00:50:47.750]Just for comparison sake,
- [00:50:49.760]there are correlations with five
factor model agreeableness and
- [00:50:53.600]conscientiousness in sample two,
- [00:50:55.970]showing that both of them are related
to both irritable disinhibition
- [00:51:00.680]negatively and thoughtful dependability
positively at the four-factor level.
- [00:51:05.780]And only with self-centered impulsivity
negatively at the higher order level.
- [00:51:11.570]So these data also suggests
social dominance per se
- [00:51:15.560]maybe best conceptualized is not
particularly strongly reflecting a
- [00:51:19.910]psychopathic personality trait.
- [00:51:22.070]Perhaps it does need to be combined
with other traits to reflect psychopathy.
- [00:51:27.680]Taken together,
- [00:51:28.370]these data show the advantage of
separating the two higher order factors,
- [00:51:33.830]fearless dominance in particular,
into two subtraits with fearless,
- [00:51:38.720]non-conformity more pathological
than social dominance,
- [00:51:43.340]and the jury is still out regarding
irritable disinhibition and low thoughtful
- [00:51:48.110]dependability. As I said earlier,
- [00:51:51.890]I plan to follow up on these analysis in
a chapter in a forthcoming festschrift
- [00:51:57.020]examining their relations
with the additional measures
and including behavioral
- [00:52:02.690]I hope you'll have the opportunity to
read that volume when it is published
- [00:52:06.740]so I'm sure it will contain a number
of chapters that will be worth
- [00:52:11.720]perusing. In some two factor,
- [00:52:15.170]fearless dominance can be split into
fearless nonconformity and social
- [00:52:20.030]which correlate respectively with
disinhibition positive temperament or
- [00:52:24.560]agentic extroversion, if you will.
- [00:52:27.920]Whereas self-centered impulsivity can
be split into irritable disinhibition
- [00:52:32.630]versus thoughtful dependability,
- [00:52:34.820]which correlate respectively with
negative temperament and constraint.
- [00:52:42.020]I want to end my talk today by
illustrating how the SNAP scales
- [00:52:46.970]relate to behaviors that are
relevant to psychopathic personality
- [00:52:51.440]traits using three different samples.
- [00:52:55.700]First is a sample of 170 college
students who prospectively
- [00:53:00.050]recorded on a daily basis
- [00:53:02.390]whether they had exhibited a set of
behaviors relevant to three trait domains
- [00:53:07.430]over a period of 10 to 14 days.
- [00:53:11.240]The domains we assessed were aggression.
- [00:53:14.150]Sample items include "yelled,
cursed, snap, or shouted at someone",
- [00:53:18.290]"got into an argument".
- [00:53:23.120]Impulsiveness. Sample items
included "skipped class on a whim"
- [00:53:29.070]"made.a list of things to do",
reversed scored of course,
- [00:53:34.530]and exhibitionism. Sample items
included "work clothes that drew attention"
- [00:53:39.240],"flirted with someone".
- [00:53:42.480]We then calculated the average
number of each set of behaviors
- [00:53:47.460]that participants had exhibited
daily over the sampling period and
- [00:53:52.230]correlated those with the corresponding
scales from the SNAP and other
- [00:53:56.790]measures that were assessed
at the beginning of the 10
day behavioral assessment
- [00:54:03.960]You can readily see that the aggressive
behaviors correlated with SNAP
- [00:54:09.000]trait aggression, and also the
Buss-Perry aggression questionnaire.
- [00:54:13.380]The impulsive behaviors
correlated with SNAP trait
- [00:54:17.400]impulsivity and the Barrett Impulsivity scale.
- [00:54:21.210]And the exhibitionistic behaviors
correlated with SNAP and
- [00:54:25.650]Narcissistic Personality Inventory
exhibitionism demonstrating that
- [00:54:30.300]self-reported personality traits
reflect corresponding real life daily
- [00:54:35.220]behavior. In the second study,
- [00:54:41.220]366 adolescents completed the SNAPY,
- [00:54:45.600]the SNAP for youth, and their
parents rated their teens using an
- [00:54:50.280]informant version of the SNAP.
- [00:54:53.250]We also gathered both academic data
and behavioral problem data from
- [00:54:58.200]their school charts.
- [00:55:01.620]Here are the correlates with the
academic data. Parent rated workaholism
- [00:55:07.350]correlated with students' grades as did
propriety, a scale assessing adherence to
- [00:55:12.300]traditional social rules as
rated by both parents and teens.
- [00:55:18.360]Grades also related negatively
to teen rated disinhibition,
- [00:55:22.830]impulsivity and aggression. Interestingly,
- [00:55:27.240]other than the two parent rated scales,
- [00:55:30.510]these personality traits were not
related to standardized test scores.
- [00:55:35.280]And even the parent ratings
correlated more weekly than with grades.
- [00:55:40.620]This indicates that grades largely
reflect the role of personality
- [00:55:45.960]in academic achievement,
- [00:55:48.180]whereas standardized tests
for the most part do not.
- [00:55:54.630]As for the problem behaviors,
the correlates were moderate
- [00:55:58.080]especially for the number of suspensions,
which were very low in frequency,
- [00:56:03.090]but higher for the number of times
that the school bus driver wrote up the
- [00:56:07.050]student for misbehaving on the bus
with aggression being the strongest
- [00:56:12.840]These results indicate that maladaptive
behaviors can in fact be self-reported
- [00:56:16.860]by adolescents as well as
discerned by school bus drivers.
- [00:56:21.900]Also know by the way,
- [00:56:22.800]the good agreement between parents
and teens on these relations was just
- [00:56:26.860]unusual. Finally,
- [00:56:30.880]in the third study,
- [00:56:33.010]561 community adults completed what
we call the demographic questionnaire.
- [00:56:38.770]Besides actual demographics,
- [00:56:41.440]the questionnaire asked whether they
were currently in treatment for mental
- [00:56:44.530]health issues, whether they
had a history of arrest or not,
- [00:56:49.360]and a number of questions regarding
frequency and quantity of alcohol and drug
- [00:56:53.440]use, including whether they had ever
been treated for drug or alcohol problems.
- [00:56:59.290]From the substance use questions,
we created a total score,
- [00:57:02.590]and for our purposes today,
- [00:57:04.450]I dichotimized that measure and we
examined the relations between the three
- [00:57:09.220]core temperament scales of the SNAP
and these three behavioral domains.
- [00:57:15.880]Regarding mental health treatment,
- [00:57:18.130]we see the expected difference in
negative temperament with those who are in
- [00:57:21.730]treatment higher in negative
temperament than those who are not.
- [00:57:26.260]A similar but smaller, and of course
- [00:57:28.780]opposite effect for positive
temperament was found,
- [00:57:33.040]but there was no significant
difference on disinhibition.
- [00:57:38.620]Severity of alcohol and drug use however,
- [00:57:41.680]shows a different pattern with no
significant difference on either negative
- [00:57:46.030]temperament or positive temperament,
- [00:57:50.890]but a significant
difference on disinhibition.
- [00:57:55.900]Likewise, a history of criminal arrest.
- [00:57:58.810]There was no significant difference
on negative temperament or positive
- [00:58:03.310]temperament, but a significant
difference on disinhibition.
- [00:58:11.050]In summary,
- [00:58:11.560]I've shown that the factors measured
with Scott Lilienfeld's PPI can be assessed
- [00:58:15.550]with a measure developed in the
three factor tradition of Eysenck,
- [00:58:20.140]and also Scott's and my common mentor,
- [00:58:23.020]Auke Tellegen who held to a three
factor model and that these scales do
- [00:58:27.790]indeed correlate with a variety of
behaviors in the domain of psychopathy.
- [00:58:33.430]I think Scott would have
been pleased to see how
- [00:58:37.660]broadly his Psychopathic
Personality Inventory reaches
- [00:58:42.490]into other measures of
- [00:58:47.830]At least I hope so. Thank
you for attending today,
- [00:58:51.730]and I guess we'll take questions now.
- [00:58:53.940]All right. Very good.
Thank you, Dr. Clark,
- [00:58:57.360]and we do not have any
questions I believe at this
- [00:59:01.590]time. And let me,
- [00:59:05.340]I was just double checking the time
to make sure we have some time.
- [00:59:11.130]So you mentioned early on that
- [00:59:14.940]Scott had mentioned some differences,
- [00:59:16.530]gender differences on such. Is that also
been some areas that you've explored
- [00:59:21.530]with the SNAP
- [00:59:22.310]or how it associates and aligns with
other measures such
- [00:59:26.180]demographic differences?
- [00:59:28.660]I have done some of that certainly,
and the biggest demographic
- [00:59:33.670]the biggest gender differences are
on negative temperament with females
- [00:59:38.650]typically scoring higher
- [00:59:42.760]with males scoring higher.
- [00:59:46.390]That's not been a major
emphasis of mine, but
- [00:59:51.100]yeah, those data certainly
are there and there's
- [00:59:53.600]surprisingly little difference
on positive temperament.
- [00:59:57.220]Very good. Very interesting. Yeah. It's
interesting to see how that plays out.
- [01:00:03.700]Let's assume that if
there are no questions,
- [01:00:05.560]that means that I was crystal clear.
- [01:00:09.610]I thought it came
across pretty clear,
- [01:00:15.670]so we thank you for your time. All
right. So Dr. Waldman,
- [01:00:20.470]you are up.
- [01:00:44.050]Thanks very much and thanks very
much for having me speak today.
- [01:00:49.060]So I'm going to be talking
about some work that I've done
- [01:00:53.560]in my lab recently with my senior
grad student Holly Poore on
- [01:00:58.240]mapping the externalizing spectrum and
youth from an assessment perspective.
- [01:01:04.450]So I had the great good fortune
to be friends with Scott,
- [01:01:08.980]close friends with Scott for 40 years
since we were undergrads together at
- [01:01:13.030]Cornell, and in fact,
- [01:01:14.740]I could give a whole separate
much longer talk about that,
- [01:01:18.550]but we don't have time for that today,
- [01:01:21.100]and I've been lucky to be a colleague
of his for 30 of those years.
- [01:01:26.110]And, I went back and after
just a little bit of digging,
- [01:01:29.150]found a couple of papers
that are relevant to
- [01:01:32.470]the topic of my talk today that we
published near the start of our careers.
- [01:01:38.140]Both here looking at the
relation of childhood ADHD to
- [01:01:42.760]oppositional defiant disorder
- [01:01:45.310]on the one hand and adult
antisocial behavior on the other.
- [01:01:49.480]And then also
- [01:01:51.760]looking at more conceptual and
statistical or methodological issues,
- [01:01:56.740]in this domain, and the
first paper on the left
- [01:02:00.880]in which we took on the
concept of comorbidity in
psychopathology research and
- [01:02:05.620]on the right, we
- [01:02:07.030]talked about statistical methods
on how they might be
- [01:02:11.140]better use to examine construct
validity of externalizing in childhood.
- [01:02:17.950]Aside from my
collaboration with Scott,
- [01:02:22.110]my own motivations for this study is
that I've been involved in research on
- [01:02:26.580]hierarchical structural models of
psychopathology for quite some time,
- [01:02:30.870]particularly of child
psychopathology. And most recently,
- [01:02:34.410]the past five years
through an organization,
- [01:02:36.930]a consortium called HiToP that
LeeAnna and Martin are also part of
- [01:02:42.390]and more broadly, this project
fits with research approaches
- [01:02:47.370]that have sought to reconceptualize
- [01:02:50.130]psychopathology moving from
- [01:02:54.030]into the use of continuous
- [01:02:56.760]And the way I think about this is
it represents a move from
- [01:03:01.530]commonly called transdiagnostic
approaches to more transdimensional
- [01:03:08.310]I've also more recently been involved
in genome-wide studies of childhood
- [01:03:12.630]psychiatric disorders,
ADHD, ODD, and conduct disorder,
- [01:03:17.640]and more recently
- [01:03:20.970]an externalizing GWAS consortium. And
this raises the question in my mind of
- [01:03:25.980]whether an externalizing higher-order
dimension might represent a better
- [01:03:30.000]phenotype for such studies in the sense
that it might increase statistical
- [01:03:33.990]power by increasing effect sizes.
- [01:03:37.170]So this work brings together my two
major interests in the classification and
- [01:03:41.790]causes of psychopathology, particularly
- [01:03:43.880]in child samples. So there are
- [01:03:47.630]a couple of specific questions we sought
to answer in this work. So first,
- [01:03:52.610]what is the evidence for an
Externalizing spectrum in kids?
- [01:03:56.360]I think we have come to know
- [01:03:58.250]quite a bit about this in
adults and late adolescence,
- [01:04:01.700]but the literature in youth
is lagging behind.
- [01:04:05.300]Can we distinguish a higher order
from a bifactor representation of
- [01:04:08.690]externalizing in structural
modeling and external validity analyses?
- [01:04:14.030]And rather than putting
these in competition,
- [01:04:17.450]might there be advantages of each of
those approaches to externalize it?
- [01:04:22.310]Another issue is whether the
factor structure for Externalizing
- [01:04:28.340]across clinically referred and
non-referred populations and what the
- [01:04:32.390]incremental value of these approaches
over a simple externalizing composite
- [01:04:37.700]which is still what's mainly used in
the literature that or factor scores to
- [01:04:42.470]represent externalizing,
- [01:04:44.390]what the incremental value of those
approaches might be? And in general,
- [01:04:48.530]how helpful a more granular approach
starting with specific symptoms
- [01:04:53.450]at the item level, rather than
symptom, dimensions or diagnoses might be?
- [01:04:59.570]So after
- [01:05:00.380]a little bit of background,
- [01:05:01.400]I'll go into the alternative models
that we contrasted and then the model
- [01:05:06.050]fitting results for those,
- [01:05:07.730]and then present some results
from external validity analyses
- [01:05:10.850]comparing the various models, and
then finish up with some
- [01:05:15.680]item response theory
- [01:05:17.320]results that have more to do with the
measurement of externalizing across the
- [01:05:21.970]various symptom domains.
So by way of background,
- [01:05:26.830]most people are familiar with
Tom Achenbach's seminal contribution in
- [01:05:32.740]in which he used factor analysis of
behavior problems that he gleaned from
- [01:05:36.970]review of clinic case files,
- [01:05:39.580]and in which he identified broad
externalizing and internalizing dimensions.
- [01:05:44.800]In adult late adolescent samples,
- [01:05:46.930]a lot of progress has been made
over the past 20 years
- [01:05:51.280]mostly I suppose by Bob Krueger's group
at Minnesota and Susan Young and her
- [01:05:55.540]colleagues at Boulder in which they
identified a broad externalizing dimension
- [01:06:00.160]that includes antisocial behavior,
- [01:06:01.870]substance use and personality traits,
in particular behavioral disinhibition.
- [01:06:07.660]In child samples, along with Ben Lahey,
- [01:06:10.360]we identified an externalizing
dimension using symptoms of
- [01:06:14.440]DSM-IV ADHD, ODD and conduct disorder,
- [01:06:18.310]but there are limitations in my
view of extant literature in youth,
- [01:06:22.570]and these include limited tests of
alternative conceptualizations of
- [01:06:26.410]externalizing. So what's the best
model for representing it?
- [01:06:31.330]Assessing fit relative
to a simpler model
- [01:06:35.800]which consists of correlated lower
order symptom dimensions rather
- [01:06:40.600]than any higher order factor.
- [01:06:43.660]Rudimentary measurement of
- [01:06:48.220]using composites
as I already mentioned.
- [01:06:51.970]And a failure to contrast
external validity results
across alternative models.
- [01:06:56.620]And in this work we're seeking
to address these limitations.
- [01:07:01.240]So I'm not going to spend a lot of time
on this slide. This is just capturing
- [01:07:05.020]some of the
- [01:07:07.360]results of previous relevant
recent studies in this domain.
- [01:07:11.860]And the main point to take away from
this slide is that across these studies,
- [01:07:16.120]not many have
represented symptoms of ADHD,
- [01:07:20.800]ODD, conduct disorder together, or even
symptom dimensions representing them,
- [01:07:25.150]and not many have started
at the item level.
- [01:07:29.410]So I'm going to be presenting today on
two samples, a non-referred sample that
- [01:07:34.000]comprises of 2,229 unselected
twins and siblings from Georgia,
- [01:07:39.580]and a clinic referred sample
consisting of 640 cases or control
- [01:07:44.510]children born in Georgia or
Arizona. Across both studies,
- [01:07:49.360]the participants ranged in age
from 4 to 19 years.
- [01:07:53.890]About 50% were male in
the twin sample, about
- [01:07:57.900]70% in the clinic referred
sample and across both samples,
- [01:08:02.410]about 70% were European ancestry
and 24% African ancestry.
- [01:08:07.810]As I said earlier, we assessed
symptoms of ADHD, ODD and conduct disorder,
- [01:08:12.910]and these were assessed by a parent
report on a questionnaire that we've
- [01:08:16.520]developed in my lab.
- [01:08:19.150]So we conducted
- [01:08:20.380]analyses of competing structural models
using confirmatory factor analysis and
- [01:08:25.210]did our external validity analyses
using structural equation models that
- [01:08:29.140]extended them, and then used item
response theory methods to examine
- [01:08:34.060]the measurement
properties of individual items.
- [01:08:36.370]So just to give a feel
- [01:08:39.910]for the nature of these symptoms,
for those who don't work in the child
- [01:08:44.140]psychopathology realm, ADHD
- [01:08:48.070]has three symptom domains,
- [01:08:50.920]inattention, involving being
easily distracted, impulsivity,
- [01:08:56.050]blurting out answers before questions
have been finished, hyperactivity,
- [01:09:00.910]involving fidgeting or trouble staying
seated when the child's required to.
- [01:09:06.280]Within ODD,
- [01:09:07.750]we follow recent trends that we've
contributed to where we distinguish
- [01:09:12.760]the what I'm going to
call negative affect symptoms,
- [01:09:16.720]but it's really a bit narrower than that
in the literature having to do with the
- [01:09:20.650]irritability component of
negative affect involving
- [01:09:24.910]losing one's temper,
- [01:09:25.990]being touchy or easily annoyed
from the defiant behavior aspects.
- [01:09:31.120]And then within conduct
disorder distinguishing the
aggressive symptoms from the
- [01:09:35.110]rule-breaking symptoms that involve
things like destroying property or running
- [01:09:39.220]away from home.
- [01:09:43.900]as is common with many studies,
- [01:09:46.060]we had to exclude a number of conduct
disorder symptoms due to low endorsement.
- [01:09:50.830]So in both samples, we
had to exclude "setting fires",
- [01:09:55.630]"breaking and entering", "mugging"
and "forced sex", and in the clinic sample in addition,
- [01:10:00.700]we had to exclude "running away from
home overnight", "using a weapon in a fight"
- [01:10:04.780]and "being cruel to people".
- [01:10:06.310]So I guess I would summarize
this by saying bad for science,
- [01:10:09.100]but good for society that there are so rare.
- [01:10:13.750]So what are the models that we
- [01:10:15.220]contrasted? So first is this
correlated factors model. Here
- [01:10:19.990]you see the seven symptom dimensions
that I referred to and all the
- [01:10:24.250]correlations among them. And then
- [01:10:28.420]two models that represented
- [01:10:31.990]The first being a very commonly
used bi-factor model where all those
- [01:10:35.980]correlations among the seven symptom
dimensions are captured by this general
- [01:10:40.750]externalizing factor. Now, in this work,
- [01:10:43.690]we realized that that's a pretty tall
order that general externalizing
- [01:10:48.160]factor may not be sufficient to capture
the entirety of the correlations among
- [01:10:53.140]the symptom dimensions.
- [01:10:54.670]So we tested a modified version
that also had residual correlations
- [01:10:59.920]among the symptom dimensions
within each disorder
- [01:11:04.210]as shown here. And then the alternative
model is a higher order
- [01:11:09.190]externalizing model
- [01:11:10.810]where the externalizing factor influences
seven symptom dimensions rather than
- [01:11:15.780]the symptoms directly. And
as with the bi-factor model,
- [01:11:20.010]we tested the modified version as well.
- [01:11:22.250]So what did we find?
- [01:11:26.150]Before I go on to the structural
models, the results for those,
- [01:11:31.070]this is the correlation matrix for the
clinic sample in red and the non-preferred
- [01:11:34.670]sample in blue.
- [01:11:35.990]And what you could see is that all the
correlations among the seven symptom
- [01:11:39.470]dimensions are moderate
- [01:11:41.330]except for the correlation between the
two ODD dimensions of negative affect
- [01:11:46.580]and deviant behavior which
is quite high 0.96 in both
- [01:11:53.630]In the clinic referred sample
first, we found that
- [01:11:58.520]the two models for externalizing
shown at the bottom
- [01:12:03.230]but including the modification of the
domain specific correlations among the
- [01:12:09.500]fit as well as the seven
correlated factors model
- [01:12:13.490]which is noteworthy because they're
more parsimonious representations
- [01:12:18.530]of the relations among
the symptom dimensions.
- [01:12:23.540]You could also see though, if you look
at the bottom two lines that are bolded,
- [01:12:27.830]that the fit between these two
models is totally interchangeable and
- [01:12:32.810]indistinguishable. And in fact,
- [01:12:35.000]one of the only things that distinguishes
these two models from each other is
- [01:12:38.750]that as often is the case,
- [01:12:40.940]there were some concerns with the
bi-factor model. In this case,
- [01:12:44.660]there were some non-significant or
nearly so loadings. The same is true
- [01:12:49.610]in the non-referred sample. Again,
- [01:12:51.800]indistinguishable fit of
the two externalizing
- [01:12:56.780]models from each other
- [01:12:58.850]as well as from the seven
correlated factors model. Here again,
- [01:13:02.780]there was a concern with
the bi-factor model
- [01:13:05.450]in that as you'll see in a few slides,
- [01:13:08.330]the correlation between the
aggressive and rule-breaking symptoms
- [01:13:14.210]is non-significant.
- [01:13:14.630]This is what the results look like.
This is for the clinical sample.
- [01:13:18.140]You could see that all seven symptom
dimensions have moderate to high loadings
- [01:13:22.310]on the externalizing factor,
- [01:13:24.410]and there are moderate residual
correlations among the ADHD,
- [01:13:28.790]ODD, and conduct disorder
- [01:13:32.510]The same is true for the non-referred
sample of twins and sibs with the
- [01:13:37.130]exception as I mentioned,
- [01:13:38.930]that the correlation between aggression
and rule breaking is really quite low
- [01:13:42.650]and was non-significant.
- [01:13:46.160]So we can go on and ask using the
results of the higher order factor
- [01:13:50.960]model of externalizing
- [01:13:52.790]what percent of the correlations among
these seven symptom dimensions is
- [01:13:57.230]explained by that higher order factor?
And in bold in the off diagonal,
- [01:14:02.480]we're showing the relations
- [01:14:05.720]across symptom domains, across disorders.
- [01:14:08.840]And you could see that
they average to 100% percent.
- [01:14:12.340]You could also see that some of
these go above 100% percent.
- [01:14:15.880]That's not a mathematical error.
- [01:14:17.440]It's just that these are correlations
that are implied by the model rather than
- [01:14:22.240]the correlations that we observed.
- [01:14:25.150]In the off diagonal elements in italics,
- [01:14:28.900]you can see that we did not capture 100%
percent of the correlations
- [01:14:33.790]for the within disorder
- [01:14:37.810]by the externalizing higher
order factor loadings.
- [01:14:40.900]And that's why indeed
we needed the residual
- [01:14:43.380]correlations among them. So looking at
- [01:14:47.950]the factor loadings on the higher order
factor, we can see a couple of things.
- [01:14:52.950]First, the pattern and indeed
- [01:14:54.570]even the magnitude of these loadings
is very similar across the two
- [01:14:59.880]the clinical and non-referred samples.
And the highest loadings are for the
- [01:15:04.800]deviant behavior dimension of ODD
followed by the negative affect
- [01:15:09.150]dimension of ODD and the
two conduct disorder
- [01:15:12.000]dimensions of aggression
and rule breaking.
- [01:15:14.850]And then finally the ADHD symptom
dimensions at the far left
- [01:15:20.250]the lowest. So what
- [01:15:22.470]about external validity? So these
analyses are quite important
- [01:15:26.910]not just for the obvious reason
that we want to look at
- [01:15:30.720]the external validity of these
factors structures in predicting
- [01:15:35.040]external correlates and outcomes, but
because within the context of
- [01:15:39.510]the literature on the general
- [01:15:43.140]what some have argued is that
- [01:15:46.830]while one cannot distinguish a
general factor from correlated
- [01:15:51.480]factors using
conventional model fit
- [01:15:55.650]indices, one can use external validity
- [01:16:00.030]indices such as R squared to
distinguish among those models.
- [01:16:04.050]So we wanted to see the degree to
which that was true with regard to externalizing.
- [01:16:07.650]We use a number of measures
- [01:16:11.160]in these external validity analysis.
- [01:16:14.850]They're listed here and
for the sake of time,
- [01:16:18.180]and also because we haven't
finished all of them yet,
- [01:16:20.820]I'm going to limit myself to the
first five to the impulsivity,
- [01:16:25.290]narcissism, callousness and
emotionality scales from
- [01:16:29.550]the APSD, the Antisocial Process
Screening Device which is
- [01:16:34.470]a measure of psychopathic traits in kids
and then reactive and proactive
- [01:16:42.780]In this slide, on the Y axis,
I'm showing the R squares
- [01:16:46.470]with their 95% confidence
intervals for three models.
- [01:16:51.960]From left to right,
- [01:16:53.550]the hierarchical and the bi-factor models
that include an externalizing factor
- [01:16:58.410]and then an externalizing
composite shown in red.
- [01:17:01.950]And as you can see here,
- [01:17:03.780]the hierarchical and bi-factor models
always explain exactly the same percentage
- [01:17:08.220]of variance, completely
interchangeable or indistinguishable
- [01:17:12.140]whereas the composite is always explaining
about 5 to 8% of the variance less.
- [01:17:19.130]Here, now I'm comparing the
correlated factors model.
- [01:17:23.180]The first model in sequence in
teal to the hierarchical model in
- [01:17:27.860]green and the bi-factor model
in red, and what you can [inaudible]
- [01:17:48.790]It's about 10% less than the models
containing an externalizing factor,
- [01:17:53.890]but if you look at the far right
with regard to proactive aggression,
- [01:17:57.490]you could see that it explains
about 10% more of the variance.
- [01:18:01.270]So we wanted to drill down on this a
bit further to understand it better,
- [01:18:05.920]and so we compared the results and
the results here are standardized
- [01:18:11.110]coefficients, betas that are
partialing out externalizing
- [01:18:15.970]along with their 95% confidence
intervals on the y-axis for
- [01:18:20.650]reactive aggression on the left and
proactive aggression on the right. And
- [01:18:25.030]starting with proactive aggression
what you could see is,
- [01:18:29.140]the filled in dots here mean that these
results are significantly different from
- [01:18:32.920]zero is that for
- [01:18:37.270]the strongest predictor were the
aggressive symptoms of conduct disorder,
- [01:18:41.050]rather unsurprisingly,
but more surprisingly,
- [01:18:44.020]no other scales were related. And
in fact, in the positive direction,
- [01:18:49.270]the rule-breaking symptoms of conduct
disorder were significantly negatively
- [01:18:53.710]related, reflecting suppression effects.
- [01:18:57.370]but you see very different results
for reactive aggression on the left.
- [01:19:01.300]Aggressive CD symptoms again
we're a significant predictor,
- [01:19:04.510]but nowhere nearly as strongly
as for proactive aggression,
- [01:19:08.530]but impulsivity in red
and the negative affect
- [01:19:13.240]symptoms of ODD in dark blue also
were significantly positively
- [01:19:18.910]And this makes a lot of sense when you
think of what reactive aggression is,
- [01:19:23.170]which is hot tempered,
- [01:19:27.370]often in reaction to a perceived threat
to the self in contrast to proactive
- [01:19:32.290]aggression, which is cold,
- [01:19:35.140]planned, premeditated aggression.
- [01:19:37.810]And this also highlights the
utility of not only limiting our
- [01:19:42.400]attention to the higher order dimensions
of psychopathology in this case
- [01:19:47.800]but looking at the residualized
lower order dimensions as well in our
- [01:19:53.890]So now I want to finish up by talking
about some results from item response
- [01:19:57.550]theory analyses of externalizing,
- [01:20:00.070]and as is common in the IRT literature,
- [01:20:03.760]we were interested in
two aspects information,
- [01:20:07.500]which captures the precision
of measurement at different
points along the latent
- [01:20:10.950]trait in this case,
externalizing and item difficulty,
- [01:20:14.760]which captures the probability of
endorsing a symptom, a binary symptom at
- [01:20:19.410]different points along the latent trait.
- [01:20:21.960]The one difference here is that
given that many of our symptoms,
- [01:20:25.560]the majority of them are
polygamous rather than dichotomous,
- [01:20:28.890]we present something called Location
Index Item Response Function Values,
- [01:20:33.810]and these capture the level of the
latent trait at which respondents have an
- [01:20:38.040]average score that's equal
to half the maximum of the
- [01:20:41.640]scale. So this
- [01:20:43.950]is the total information
plot of externalizing
symptoms and the twin sample,
- [01:20:48.600]and what you could see here is that
it was measured fairly precisely
- [01:20:53.130]ranging from minus one standard
deviation below the mean to
- [01:20:57.990]about five standard
deviations above the mean,
- [01:21:00.720]but with the majority of the precision
residing between the mean at zero
- [01:21:05.640]and three standard deviations
above the mean. Similar results
- [01:21:10.470]were found for the clinic sample,
- [01:21:12.120]except here the distributions are
narrower where the majority of
- [01:21:17.850]the information,
- [01:21:18.900]the precision of measurement is
provided from the mean up to about two
- [01:21:23.460]standard deviations above the
mean so narrower interval
- [01:21:27.780]at which we had precision
of measurement in
- [01:21:29.640]the clinic sample.
- [01:21:32.160]Looking at the
partial information plots,
- [01:21:36.390]this looks at the contribution of each
of the seven symptom domains to the total
- [01:21:40.860]precision of measurement. We
see that the peak is for
- [01:21:44.680]the inattentive symptoms at about two
standard deviations above the mean,
- [01:21:49.410]and that the ODD symptom dimensions
of negative affect and deviant
- [01:21:54.000]behavior are a bit below that,
about one standard deviation
- [01:21:59.550]above the mean, whereas
- [01:22:04.620]and impulsive and aggression and
rule-breaking especially symptoms
- [01:22:09.780]are providing greater
precision at the higher level.
- [01:22:18.080]Similar results were found for the
clinic sample though these were
- [01:22:21.600]more of a piece of that
- [01:22:25.810]is at the lowest along now with
- [01:22:29.000]negative affect and deviant
behavior. And again,
- [01:22:32.120]the conduct disorder symptom dimensions
along with hyperactivity and impulsivity
- [01:22:36.410]now provide more information
at the higher levels of the
- [01:22:41.210]latent trait.
- [01:22:44.720]Looking at the LI-IRF
values, we see that
- [01:22:49.490]the negative affect symptoms
and deviant behavior symptoms
- [01:22:54.290]provided the best
- [01:22:56.870]aspects of item
difficulty at the lower part of
- [01:23:01.850]the latent trait at about one standard
deviation above the mean. The ADHD
- [01:23:06.010]dimensions were next in the middle
- [01:23:08.470]followed by the conduct disorder
dimensions of aggression and rule-breaking.
- [01:23:12.400]And you see that these are flatter
- [01:23:14.170]and what that is representing is the
fact that these symptom domains seem
- [01:23:19.120]to be a bit more heterogeneous than
those of ADHD and oppositional defiant
- [01:23:24.040]disorder. Similar results
- [01:23:26.440]were found in the clinic
sample except here,
- [01:23:28.960]as you can plainly see the results
are a bit more clumped
- [01:23:32.800]such that they're a little harder
to disentangle but
- [01:23:37.840]the inattentive symptoms and
the ODD symptoms seem again to be
- [01:23:44.290]most of the item difficulty
at the lower end followed
- [01:23:49.000]by impulsivity and hyperactivity
and aggression, and then rule-breaking
- [01:23:53.830]at the top.
- [01:23:56.080]So I want
- [01:23:57.070]to end by saying a couple of
conclusions and then future directions.
- [01:24:01.240]So in our CFAs and
external validity analysis,
- [01:24:05.560]the fit of both the hierarchical or
higher order and bi-factor models
- [01:24:11.170]identical to each other in
from that of a correlated
- [01:24:14.770]factors model.
- [01:24:16.510]The pattern and magnitude of the loadings
of the seven symptom dimensions were
- [01:24:20.350]very similar across the non-referred
and clinically referred samples,
- [01:24:24.940]such that the deviant behavior
- [01:24:28.510]items had the highest loadings on
the higher order externalizing factor
- [01:24:33.430]followed by the negative affect,
aggressive and rule-breaking
- [01:24:36.760]symptom dimensions, and
then the ADHD dimensions.
- [01:24:41.020]And as I showed in the bar
graph, the loadings
- [01:24:44.830]of these seven symptom
dimensions on the higher order
- [01:24:47.650]externalizing factor explained
100% of the cross disorder
- [01:24:51.700]correlations, but only about
75% of the within-disorder associations.
- [01:24:59.010]As in the
- [01:24:59.880]structural models and the
external validity analyses,
- [01:25:03.240]the higher-order or hierarchical,
- [01:25:05.040]and bi-factor models showed identical
relations with correlates
- [01:25:09.940]and outcomes. And this was often,
- [01:25:12.420]but not always true for the
correlated factors models,
- [01:25:15.570]the most notable exception
being for proactive aggression.
- [01:25:21.210]From the item
- [01:25:21.780]response theory analyses,
- [01:25:24.300]different symptoms measured externalizing
more precisely at different levels,
- [01:25:28.500]in the non-referred sample,
the ODD symptom
- [01:25:32.790]domains captured
- [01:25:38.580]best at low levels followed by the ADHD
symptoms and then the aggressive and
- [01:25:43.290]rule-breaking symptoms.
- [01:25:45.630]And then a similar finding
was in the clinical sample,
- [01:25:48.960]but here the inattentive symptoms
were best at capturing the lower end.
- [01:25:53.700]And the results were really quite similar
looking at the average endorsements as
- [01:25:58.170]these symptoms relative to
externalizing at different levels
- [01:26:01.320]so I'll skip over that.
And then finally
- [01:26:04.790]what came out of these IRT analyses,
- [01:26:07.040]but also would come out if you looked
at their factor loadings in the CFAs
- [01:26:12.000]is that the aggressive and
- [01:26:15.260]symptom domains were more heterogeneous.
- [01:26:18.350]So what are some of the future
directions we intend to conduct
- [01:26:22.820]or to examine? So we
- [01:26:25.280]are going to go on and conduct formal
analyses of measurement and structural
- [01:26:28.850]variance across sample
type, more formally
- [01:26:33.770]contrasting the clinic
and non-referred samples,
- [01:26:37.220]but also looking at age and sex
- [01:26:39.080]since there's such a large
age range of our sample.
- [01:26:42.890]We're going to finish the external
- [01:26:45.800]obviously with the remaining criteria
and look at the variance explained by
- [01:26:50.570]the higher order externalizing factor
versus the lower order dimensions.
- [01:26:55.100]And again,
- [01:26:55.850]look for instances where we do find
interesting suppression effects
- [01:27:00.650]with the lower order dimensions
once we partial out externalizing.
- [01:27:05.540]We've also just launched a followup
study of these two samples in which we
- [01:27:10.100]intend to examine the relations of
externalizing and the lower order
- [01:27:14.990]dimensions with adult psychopathology,
disordered substance use,
- [01:27:19.340]and other outcomes. And,
Holly Poore in her dissertation,
- [01:27:23.660]which she has her defense scheduled
in three weeks I'm happy to say,
- [01:27:28.640]and I'm sure she has as well is
- [01:27:31.040]we're going to examine extensions of
externalizing spectrum in youth using
- [01:27:35.810]the relevant personality traits and
behavioral characteristics that were the
- [01:27:39.890]external validity outcome
variables in this study.
- [01:27:44.090]And of course we want to replicate
our findings in larger samples
- [01:27:48.050]for its own sake, but also
- [01:27:49.400]so we can retain more of the
conduct disorder symptoms
- [01:27:53.300]rather than losing them. Okay, thanks.
- [01:27:56.300]I'm going to stop there
and take any questions.
- [01:27:59.710]Thank you, Dr. Waldman, and
we do have a few questions,
- [01:28:02.800]but unfortunately are a bit short on
time. I'll just ask one briefly.
- [01:28:07.150]So what extent do you believe
- [01:28:08.890]or as research identified healthy or
prosocial counterparts to externalizing
associated with ADHD, ODD or
- [01:28:21.370]We have not looked at that,
- [01:28:25.000]but traits relevant to those could
be looked at that way. So risk-taking
- [01:28:29.710]for example, not always a bad thing.
- [01:28:34.330]And of course that goes into
some of the symptoms.
- [01:28:39.850]Very good. All right. Thank you
very much. All right. And Dr. Wood,
- [01:28:45.220]we are ready for
you now. Thank you, Dr. Waldman.
- [01:28:49.690]Thank you. Okay.
- [01:28:53.520]Let's see what happens here.
- [01:29:05.700]All right. Now, can you see that?
- [01:29:10.700]Yes, we can.
- [01:29:12.260]You may want to start your slideshow
that may provide us a better view.
- [01:29:19.220]Okay. Let's see.
- [01:29:24.740]Here we go. Let's see if that
works. There we go. Looks good.
- [01:29:28.850]Thank you. Hi, my name
is Jim Wood, and
- [01:29:32.720]today I'm going to give a little talk
on the current status of the Rorschach
- [01:29:36.590]Inkblot Test.
- [01:29:37.550]My co-authors for the talk are
Theresa Nezworski and Howard
- [01:29:42.500]Garb. On behalf of Teresa,
Howard, and myself,
- [01:29:47.360]I'd like to begin by saying how much
the three of us already miss the sage
- [01:29:51.860]comments and helpful insights of
Scott Lilienfeld our fellow Rorschach
- [01:29:56.870]critic and foxhole friend
for more than two decades.
- [01:30:01.580]My presentation today would surely be
better if he were giving us input and
- [01:30:06.410]editorial advice as a
coauthor. As I learned,
- [01:30:11.210]well, preparing dozens of
articles and one book with Scott,
- [01:30:15.110]he was a great collaborator.
- [01:30:16.880]His knowledge of the scientific
literature on clinical psychology,
- [01:30:20.900]social psychology and
assessment was encyclopedic.
- [01:30:25.400]He was a fluent writer who explained
complex issues with clarity and
- [01:30:29.870]grace. He provided excellent
- [01:30:34.220]always with modesty and a light touch.
- [01:30:38.720]But what I admired most
about Scott was his
- [01:30:43.340]willingness to speak out
calmly and courageously on
- [01:30:48.440]many occasions regarding
ill-conceived, harmful notions about
- [01:30:53.000]psychology that have gained widespread
acceptance from time to time in our
- [01:30:57.620]profession or our society.
When multiple personality
- [01:31:02.510]disorder and facilitated
communication became fads,
- [01:31:06.770]he stepped forward to critique
the unscientific claims
that were being advanced.
- [01:31:11.720]When the United States Congress voted to
condemn a meta-analysis by Rind and his
- [01:31:16.610]associates in Psychological Bulletin
and the American Psychological
- [01:31:21.050]Association caved to
the political pressure,
- [01:31:24.500]Scott advocated for scientific
integrity and the peer review
- [01:31:31.130]I could compile a long list
of similar instances in which
- [01:31:35.960]Scott was willing to speak out
in his distinctively calm,
- [01:31:41.000]courteous, and good humored way
- [01:31:43.310]when a voice of reason was urgently
needed to represent the cause of good
- [01:31:47.480]science and depose popular
myths and fallacies.
- [01:31:52.610]Of course,
- [01:31:53.660]I observed Scott's opposition to
myths and fallacies most closely and appreciatively
- [01:31:58.520]when he joined Teresa, Howard and me
- [01:32:03.940]in the Rorschach controversy.
- [01:32:07.600]So that brings me to my talk today.
- [01:32:10.930]I originally planned to focus on the
importance of norms for test validity,
- [01:32:15.160]but one of my coauthors suggested that
it would be more interesting to give a
- [01:32:18.970]little elevator talk on the current
status of the Rorschach Inkblot
- [01:32:23.800]Test. So I'm gonna hop a bit from
topic to topic, and I hope you find it
- [01:32:29.650]I'll start with one of the most
interesting and unanticipated
- [01:32:34.180]Rorschach developments
of the past 10 years.
- [01:32:38.740]Once again,
- [01:32:40.390]there are two competing
- [01:32:44.140]The first is the comprehensive
system developed by John Exner,
- [01:32:50.080]and the second is the Rorschach
Performance Assessment System,
- [01:32:54.970]the R-PAS developed by
Gregory Meyer, Joni Mihura,
- [01:32:58.750]Donald Viglione and Phillip Erdburg.
- [01:33:01.870]Now to appreciate the
significance of this development,
- [01:33:05.020]it will help to give a little
history. From the 1940s until the
- [01:33:11.470]there were several competing Rorschach
systems in use by American psychologists,
- [01:33:16.060]the most prominent were the
system developed by Samuel
Beck and a much different
- [01:33:20.920]system developed by Bruno Klopfer.
- [01:33:23.680]There was considerable controversy,
- [01:33:26.320]even hostility between Beck and
Klopfer regarding their systems,
- [01:33:30.280]but in 1974, after Beck
- [01:33:33.700]and Klopfer had both died, John
Exner introduced a new system,
- [01:33:40.270]the comprehensive system for the Rorshach
that combined what he considered the
- [01:33:44.680]best features of the Beck
and Klopfer systems.
- [01:33:48.850]This idea of combining both
systems was appealing to many
- [01:33:53.680]clinicians. Furthermore, Exner won
- [01:33:56.170]the admiration of empirically
minded assessment experts
- [01:34:01.120]such as Anastasi because
he included extensive norms
- [01:34:05.860]for American adults and children,
- [01:34:08.020]and provided detailed reviews of the
scientific literature to support the
- [01:34:12.580]validity of the scores in his system.
- [01:34:16.150]By the time my colleagues and I began
publishing articles on the Rorschach in
- [01:34:20.920]1995, Exner's system had
become by far the most
- [01:34:25.630]popular approach to the Rorschach.
- [01:34:28.750]There seemed to be no likelihood that
another system could seek to replace it
- [01:34:33.130]within the foreseeable future. However,
- [01:34:35.620]two events opened the door for a
new system. First in the years
- [01:34:40.270]between 1995 and 2005,
- [01:34:43.750]it became clear that the comprehensive
system had serious scientific and
- [01:34:48.430]psychometric difficulties. Second,
- [01:34:51.610]John Exner passed away in 2006. After
- [01:34:56.200]Exner's death,
- [01:34:57.560]the group of Rorschach scholars that I've
already named established a new
- [01:35:02.060]Rorschach system,
- [01:35:02.840]which they named the Rorschach
Performance Assessment System,
- [01:35:07.220]or (R-PAS).
- [01:35:09.080]The first technical manual for
(R-PAS) was published in 2011.
- [01:35:14.600]During the 10 years since then,
- [01:35:17.780]the comprehensive system has continued
to be the most popular Rorschach system.
- [01:35:24.050]R-PAS has steadily gained adherence
and may eventually become the most
- [01:35:28.820]popular Rorschach system,
- [01:35:30.710]particularly because Exner is
no longer alive to update and
- [01:35:35.690]promote his own system.
- [01:35:40.340]Now I've alluded to serious
problems with Exner's comprehensive
- [01:35:45.230]system that were identified
between 1995 and 2005.
- [01:35:50.360]And the rest of this talk, I'll
briefly describe some of them.
- [01:35:54.530]The first problem concerns the
inaccurate literature reviews presented
- [01:35:59.450]in Exner's books. From
1985 until the present, many
- [01:36:04.430]psychologists have been introduced
to these books in graduate school.
- [01:36:09.380]They are often treated as
authoritative textbooks or test
- [01:36:13.850]manuals in clinical assessment classes.
- [01:36:18.560]However, in 1995 and 96,
- [01:36:22.640]my coauthors and I began publishing
articles to point out that Exner's
- [01:36:27.560]literature reviews on
the Rorschach omitted
- [01:36:30.230]many important research findings
and substantially exaggerated
- [01:36:35.360]the empirical evidence favorable
to the Rorschach. In these
- [01:36:40.370]same articles,
- [01:36:41.660]we pointed out that Exeter's claims
regarding Rorschach validity were
- [01:36:46.370]often based on brief descriptions of
unpublished studies that he and his
- [01:36:51.050]colleagues had carried out,
- [01:36:54.380]but when we wrote to Exner and
requested copies of these studies
- [01:36:58.610]so we could examine their
methodology and analyses in detail,
- [01:37:02.960]he had an assistant write back to us that
the studies had not been written up in
- [01:37:07.730]a form that could be shared
with us. For many years,
- [01:37:12.590]our criticisms of Exner's
literature reviews were more or less
- [01:37:17.270]ignored by Rorschach researchers.
However, in a new development,
- [01:37:21.950]Joni Mihura, a prominent proponent
of the Rorschach and coauthor of the (R-PAS)
- [01:37:26.910]has recently expressed
views very similar to
- [01:37:35.080]are some of the comments of Mihura and
her colleagues on Exeter's book,
- [01:37:39.160]which they refer to as the
Comprehensive System (CS) test manual.
- [01:37:44.920]The Comprehensive System (CS) test
manual contains a notable number of
- [01:37:49.840]reporting errors and
inaccurate study descriptions.
- [01:37:54.460]The degree which the Comprehensive
System (CS) manual portrays a more
- [01:37:58.770]consistently positive picture of its
test variables than the published
- [01:38:03.240]literature is due
- [01:38:04.560]at least in part to the reporting errors
in the Comprehensive System (CS) manual.
- [01:38:10.530]The extent of the inaccuracies in
the Comprehensive System (CS) manual
- [01:38:15.150]make it impossible for test user
users to rely on its report of
- [01:38:20.130]the validity literature
for its test variables,
- [01:38:23.700]especially for the test
developers' unpublished studies
- [01:38:27.240]which are not available for
review to correct any errors.
- [01:38:36.740]Now I'm often asked, is
the Rorschach valid?
- [01:38:39.740]And my answer is the Rorschach
includes hundreds of different scores.
- [01:38:44.480]Some are genuinely valid
for limited purposes,
- [01:38:47.930]but most have little or no
validity for any purpose.
- [01:38:53.150]Since 1995, my colleagues and I,
- [01:38:55.730]including Scott have been evaluating
the validity of Exener's Rorschach's scores
- [01:39:00.980]for measuring traits relevant to mental
health, including psychiatric diagnoses.
- [01:39:05.870]Our conclusions stated in our book,
- [01:39:09.770]"What's Wrong with the Rorschach?" are that
some comprehensive system scores are
- [01:39:14.540]validly related to cognitive ability,
- [01:39:17.600]that is to G and to cognitive impairment.
- [01:39:21.500]Other scores are validly related to
schizophrenia, psychosis, thought
- [01:39:26.120]disorder, and personality disorders
that involve thought disorders such as
- [01:39:31.250]schizotypal personality disorder
and borderline personality disorder.
- [01:39:36.740]Otherwise, comprehensive system scores
- [01:39:40.160]bear little or no demonstrated
- [01:39:42.650]relationship to
diagnoses of anxiety, depression,
- [01:39:46.250]antisocial personality
- [01:39:49.610]or other psychological disorders.
- [01:39:54.860]These conclusions have
received substantial though
- [01:39:58.430]not total support from
a massive Psychological
Bulletin article published in
- [01:40:03.410]2013 by Joni Mihura,
- [01:40:06.110]Greg Meyer and their colleagues.
- [01:40:09.170]The article included meta analyses
of what the authors identified as the
- [01:40:14.120]65 most important variables in
Exner's Comprehensive System.
- [01:40:19.610]I'll quickly summarize what we think
can be learned from the Mihura et al. meta
- [01:40:24.590]analysis. First, I'd like to
discuss the top rows in this table.
- [01:40:29.960]This is a summary of the 65 studies.
You can see the number of studies there,
- [01:40:35.060]and it tells
- [01:40:36.890]how they were ranked as far as
their evidence of the validity. Mihura
- [01:40:42.200]and her colleagues found that 12
of the 65 comprehensive system
- [01:40:46.670]variables have no research support at all.
- [01:40:50.510]The validity of these variables has
simply never examined by researchers.
- [01:40:56.200]Another 13 variables have been examined
by researchers, but
- [01:41:01.420]have validity lower than 0.15.
- [01:41:05.050]Ten additional variables
have validity between 0.15
- [01:41:10.060]and 0.20. So to sum up these numbers,
- [01:41:13.600]the Mihura meta analysis or
meta analyses actually found that
- [01:41:18.520]more than half of the
Comprehensive System (CS) variables,
- [01:41:21.910]that is 53.8% have no evidence of validity
- [01:41:26.890]or validity of 0.20 or smaller. Now,
- [01:41:31.360]many of these variables in the past
- [01:41:33.280]have been given substantial importance
in the Comprehensive System (CS). However,
- [01:41:36.850]it's hard to imagine that variables
with validity this low could be
- [01:41:41.530]clinically useful.
- [01:41:44.440]Now, next
- [01:41:44.950]I want to call your attention to the
group of 13 variables at the bottom of the
- [01:41:50.230]that according to Mihura and her
colleagues have excellent validity,
- [01:41:54.610]that is validity greater than 0.33.
- [01:41:58.840]Now this is the group of Rorschach
scores that my colleagues and I are
- [01:42:03.560]most excited to see. What
are the top performers
- [01:42:08.410]among Rorschach scores? As you recall,
- [01:42:12.790]our book said that some Rorschach
scores are validly related to
- [01:42:17.290]cognitive ability, cognitive impairment,
thought disorder or psychoticism.
- [01:42:22.180]It turns out that none of these top
performing Rorschach variables are
- [01:42:27.900]of that type.
- [01:42:30.900]These nine Rorschach variables reflect
the number of responses given by the test
- [01:42:35.880]taker, the complexity of those responses,
- [01:42:39.600]whether the objects described
actually fit the shape of the blocks,
- [01:42:43.920]whether the test taker shows
odd thinking or language,
- [01:42:48.180]or there is a combination
of psychotic features.
- [01:42:53.760]The high validity coefficients
for these nine scores confirm our
- [01:42:58.230]conclusions from 20 years ago. However,
- [01:43:02.340]there are four other Rorschach variables
in this group of top performers that we
- [01:43:06.960]had not predicted. We published a
comment in Psychological Bulletin,
- [01:43:12.030]which showed that two of these four
variables actually have very low validity
- [01:43:17.130]once you include data from
- [01:43:21.600]The Mihura et al. meta analyses looked
only at the published literature,
- [01:43:27.180]but there still remained two Rorschach
variables on the list of top performers
- [01:43:31.650]that we never expected to
see there and
- [01:43:35.250]so far their validity coefficients
- [01:43:38.580]do not seem to be explained by
- [01:43:41.730]publication bias.
- [01:43:43.950]So it may very well be that there are
some Comprehensive System (CS) variables,
- [01:43:49.200]maybe two that were not named in our book,
- [01:43:52.280]but have a useful level of validity.
- [01:43:59.410]As I said earlier, many people have
asked me is the Rorschach valid?
- [01:44:03.520]But no one has ever asked the
really important question to
- [01:44:08.290]me, does the Rorschach have good norms?
- [01:44:11.230]But if clinicians want to be
accurate and avoid errors
- [01:44:14.500]when using test scores to assess patients,
- [01:44:17.440]then it's important to pay at least as
much attention to norms as to validity
- [01:44:23.740]Using a test with low validity
will lead to serious clinical
- [01:44:29.320]But using a test with inaccurate norms
can lead to errors that are much more
- [01:44:33.400]severe and render
- [01:44:37.960]The Comprehensive System (CS) is a good
illustration of the point. As you recall,
- [01:44:43.270]when the system first appeared,
- [01:44:45.190]it gained approval from some
rigorous empiricists because it was
- [01:44:49.900]thought to be the first and only
Rorschach system ever to have good
- [01:44:56.050]Exner's books were full of normative
tables for both adults and children,
- [01:45:00.730]and these norms were used for more
than 30 years with tens of thousands or
- [01:45:05.530]even hundreds of thousands
of patients. However,
- [01:45:09.460]starting in the late 1990s,
study after study began to
- [01:45:14.350]show that Exner's
- [01:45:16.000]adult and children's norms
were seriously in error,
- [01:45:20.320]they tended to overpathologize. That is,
- [01:45:23.860]they tended to make normal individuals
appear psychologically disordered.
- [01:45:28.660]I won't catalog the stories here, but
they're discussed in depth in our book.
- [01:45:33.040]And I'll mention just
one as an illustration.
- [01:45:38.050]In a 1999 study published in the
Journal of Personality Assessment,
- [01:45:43.210]Shaffer and his colleagues reported giving
the Comprehensive System for the Rorschach
- [01:45:47.680]to 123 non-patient community
- [01:45:55.690]Their scores on the WAIS and
- [01:46:00.460]the MMPI-2 were average,
- [01:46:05.770]but on the Rorschach,
- [01:46:07.030]17% of participants had
scores suggestive of psychoticism
- [01:46:12.010]or thought disorder.
- [01:46:14.380]And 30% were identified as narcissistic.
- [01:46:20.140]Now, as evidence like this mounted,
- [01:46:22.060]it became clear that the Comprehensive
System (CS) norms were seriously in error
- [01:46:26.950]with a tendency to overpathologize.
Exner adamantly denied
- [01:46:31.630]there was any problem
with the norms. However,
- [01:46:34.570]his position was seriously
eroded when in 2001,
- [01:46:39.400]he reported that his
adult normative sample,
- [01:46:42.610]which supposedly contained 700
individuals and had been used by
- [01:46:46.960]psychologists since the 1980s
had in fact consisted of
- [01:46:51.570]only 479 individuals with the other 221
- [01:46:56.460]records consisting of
- [01:47:02.370]The Comprehensive
- [01:47:03.180]System (CS) has remained popular
since Exner's death in
- [01:47:07.290]2006, but psychologists who still
use it are running a grave risk,
- [01:47:12.420]if they use his norms.
- [01:47:14.790]There's a very substantial danger of
identifying psychologically normal people
- [01:47:19.470]as disturbed.
- [01:47:25.880]The problem is with these norms
have gradually become clearer even
- [01:47:30.380]to some of the most stalwart
Rorschach defenders. For example,
- [01:47:35.510]in 2001, when evidence of
these problems first emerged,
- [01:47:39.710]and we pointed them out in a review
article, Gregory Meyer, published a
- [01:47:45.290]arguing that it should be clear that
the Comprehensive System norms do
- [01:47:50.060]not overpathologize.
- [01:47:54.410]More recently though,
- [01:47:56.000]Meyer has adopted a position
similar to what we argued in 2001.
- [01:48:01.040]So here's what he has said
and we are in agreement.
- [01:48:06.770]Clinicians in the United States and
- [01:48:08.390]inother countries using the standard
Comprehensive System (CS) norms will
- [01:48:13.130]incorrectly infer that non-patients
are prone to perceptual distortions,
- [01:48:18.320]see the world in an atypical and
idiosyncratic manner, tend to be
- [01:48:22.700]simplistic, lack affect resources,
- [01:48:25.190]lack coping resources in general, are
prone to affective disturbances and
- [01:48:29.660]dysregulation, and misunderstand
others and perceive relationships.
- [01:48:34.910]Of additional concern,
- [01:48:36.380]these inferences will be incorrectly
applied in clinical and research settings
- [01:48:41.150]as well, where they are likely
to have serious ramifications.
- [01:48:49.130]Yeah, so you've noticed that I've said
very little about (R-PAS),
- [01:48:53.300]the baby in the Rorschach system. In fact,
- [01:48:56.120]there's not a lot to say. There's
very little independent research
- [01:49:01.280]on basic topics concerning
(R-PAS), such as validity,
- [01:49:05.060]temporal stability, and the
representativeness of norms.
- [01:49:09.380]So I'm going to end by providing
just six swift observations that
- [01:49:14.330]may provide some stimulation to your
own thinking about the (R-PAS).
- [01:49:21.830]the administration and scoring procedures
for (R-PAS) were intentionally
- [01:49:25.670]designed to modify the administration
scoring and interpreted procedures
- [01:49:30.470]of the Exner system.
- [01:49:32.270]The two systems differ
substantially in these respects.
- [01:49:40.190]One of the most important changes for
the (R-PAS) was its introduction of new
- [01:49:44.780]scoring rules for form quality.
- [01:49:47.350]It is still unclear whether these new
scoring rules are effective or yield
- [01:49:51.610]results that are as reliable and
valid as the comprehensive systems
- [01:49:56.080]rules for form quality.
- [01:49:59.560]It is sometimes said. This,three.
- [01:50:02.290]It is sometimes said that (R-PAS)
retained the valid scores in the extra
- [01:50:07.240]system and tossed out the invalid ones.
This claim is only half true. The
- [01:50:12.190](R-PAS) authors did discard
many invalid scores from
- [01:50:17.050]Exner's system,
- [01:50:18.640]but they also incorporated a substantial
number of variables of unproven or
- [01:50:23.410]limited validity. So when, for instance,
- [01:50:26.710]deciding whether to teach this into the
assessment class, one cannot assume that
- [01:50:31.000]most (R-PAS) scores have
good empirical support.
- [01:50:38.190]Number four. At present,
only a very small number of
- [01:50:42.510]studies have examined the
validity of (R-PAS) scores.
- [01:50:47.160]Most assertions about (R-PAS) validity
- [01:50:50.010]rest upon research using the
Comprehensive System (CS) for the Rorschach,
- [01:50:54.750]which I indicated has
substantially different administration and
- [01:50:59.490]scoring rules or based on
other Rorschach approaches.
- [01:51:06.390]The adult norms of (R-PAS) are highly
problematic and have been criticized by
- [01:51:11.130]several authors. And, if you would
like to references, just let me know.
- [01:51:15.660]There are two main problems
with these norms. First,
- [01:51:20.010]the normative sample is a composite
based on combining normative samples
- [01:51:24.600]from a fairly large number of different
countries, socioeconomic groups,
- [01:51:29.100]language groups, and cultures.
They were mainly,
- [01:51:35.520]they were not selected in any
way as a random sample of
- [01:51:39.570]the countries they came from
or the people that they came from.
- [01:51:44.550]Therefore, the
- [01:51:45.270]norms do not represent
a representative sample
from the United States
- [01:51:50.370]or from any identifiable population.
- [01:51:55.860]the Rorschach protocols for the (R-PAS)
were gathered and initially scored
- [01:52:00.540]using the administration and scoring
procedures of Comprehensive
- [01:52:05.220]System (CS), which as I have said,
- [01:52:07.470]are substantially different
from the procedures of (R-PAS).
- [01:52:11.280]After protocols were gathered,
- [01:52:13.410]the (R-PAS) norms were
calculated by estimation.
- [01:52:16.770]Specifically, the (R-PAS) authors
performed what they called a modeling
- [01:52:21.540]procedure to estimate what
participants Rorschach scores would
- [01:52:26.280]have been
- [01:52:27.480]if the task had been administered
using (R-PAS) administration
- [01:52:37.710]When the (R-PAS) was developed
in or introduced in 2011,
- [01:52:42.510]it was widely expected that the
(R-PAS) authors would soon gather
- [01:52:46.460]representative American norms using
(R-PAS) administration and scoring
- [01:52:51.260]procedures. Ten years later,
- [01:52:54.110]adult norms have still not appeared.
- [01:52:58.100]So there I'm done.
- [01:53:00.500]And I want to say it's been my pleasure
to address you and to join this
- [01:53:05.030]very special occasion in honor of Scott.
- [01:53:10.030]Great. Thank you, Dr. Wood. And we
just have maybe a few moments
- [01:53:14.500]for a couple of questions.
One question was you mentioned,
- [01:53:19.240]and I don't think you
referred to it that,
- [01:53:23.110]there were two Rorschach indices
that had some validity
- [01:53:27.340]were non-cognitive?
- [01:53:29.740]Yes. One of them is called
the suicide constellation.
- [01:53:35.710]Okay. And
I've forgotten the other one. Okay.
- [01:53:40.630]Okay. Very good. Another
question that was asked,
- [01:53:44.230]and you may have been commenting on
this because it came in before your
- [01:53:47.560]conclusion, but I wasn't entirely sure
if this is what you're referring to,
- [01:53:52.360]but the
- [01:53:53.620](R-PAS) authors developed a
technique to use during the Rorschach
- [01:53:57.730]administration to optimize the number
of responses offered and in part to
- [01:54:01.750]strengthen norms.
- [01:54:03.070]So could you offer your perspective
on that response optimization?
- [01:54:07.360]Well as you said,
- [01:54:11.060]they developed basically three
procedures. They give
- [01:54:14.320]instructions at the beginning that
they expect maybe two or four
- [01:54:18.580]responses per card.
They kind of prime the client.
- [01:54:23.410]They then
- [01:54:26.620]give the cards to the client and
if the client does not respond,
- [01:54:30.760]they prompt, if they
only get one response,
- [01:54:33.040]they prompt for at least one more.
And then they also will pull a card.
- [01:54:37.000]In other words, if a
subject has four responders to a card,
- [01:54:41.890]they take it back.
Plus, there's a procedure
- [01:54:44.320]if someone comes to the end
without enough Rorschach responses,
- [01:54:47.680]they circle back,
- [01:54:49.240]and re-administer the cards a second
time and keep all the responses.
- [01:54:53.680]This was an attempt to
- [01:55:02.110]and the question is
was it worth it?
- [01:55:07.150]And recent studies indicate,
- [01:55:10.680]if I understood the findings
published by Gregory Meyer,
- [01:55:13.810]it was that they have the
- [01:55:17.140]result of the (R-PAS) administration
slightly reduces the mean level
- [01:55:22.120]of R.
- [01:55:23.830]And it reduces the
standard deviation of R by
- [01:55:28.300]about maybe a quarter to one
half of the standard deviation,
- [01:55:33.220]but that has no impact
on other scores in the
- [01:55:37.840]system except for one
- [01:55:42.730]So surprisingly, the
aim was to
- [01:55:47.550]keep the R from contaminating
all these other responses,
- [01:55:51.090]but the results as I read them,
indicate that it didn't succeed in that,
- [01:55:55.950]it just reduced R
- [01:55:58.710]without the desired effects
on other variables.
- [01:56:09.390]I do want to say
one other thing. There was
- [01:56:17.010]probably the most amazing Rorschach
study of the last 15 years was done by
- [01:56:21.390]Phillip Erberg
- [01:56:22.830]and his colleagues where they gathered
data from all these different countries.
- [01:56:27.420]Now, in my talk, I made it clear.
- [01:56:29.070]I don't think it was that that data
is appropriate for forming norms for
- [01:56:32.880]Americans, right. But nevertheless,
- [01:56:35.790]it's an amazing study because it
began to show us where are the central
- [01:56:39.240]tendencies of the Rorschach.
- [01:56:40.920]And there's quite a bit of similarity
between different nations and even
- [01:56:45.900]language groups.
- [01:56:48.210]The trouble that I see with the (R-PAS)
administration is that we can no longer
- [01:56:53.880]compare for instance.
Let's go back.
- [01:56:59.330]The existence of these many datasets
from around the world allowed us to
- [01:57:04.140]see very distinctly what
we were already suspecting
- [01:57:06.780]that Exner's norms
were way out of line with
- [01:57:11.730]what you would find in
any other country or
- [01:57:15.330]in other samples
within the United States.
- [01:57:18.600]But now with the (R-PAS)
norms, which I've indicated,
- [01:57:22.350]I think may be problematic, we just
don't have had a chance to see them.
- [01:57:26.670]But now we have no comparison again,
since the administration has been changed,
- [01:57:31.080]we can't compare them to
thousands of Rorschachs
- [01:57:34.680]which have been administered with the
(R-PAS) and other countries and see if
- [01:57:38.340]the (R-PAS) norms are indeed
getting at the
- [01:57:43.140]central distribution
of these variables.
- [01:57:46.130]All right. Very good. Well, I
want to thank the presenters.
- [01:57:50.420]We are running out of time.
- [01:57:52.580]But I want to thank the
presenters for their informative,
- [01:57:54.890]interesting presentations.
- [01:57:56.250]I think it was a great sampling
of how you spend time carefully
- [01:58:01.220]defining your constructs,
determining how to best measure those,
- [01:58:05.900]and then bringing those
- [01:58:07.550]measures and how constructs
are defined into question
- [01:58:11.330]that represents the kind
of work that Scott did.
- [01:58:14.960]So thank you the
presenters, the panelists.
- [01:58:18.050]I really appreciate your
time and effort and
- [01:58:21.680]joining us in this honorary
webinar. And of course,
- [01:58:26.090]Ms. Basterfield for
sharing more with us about
- [01:58:30.500]Dr. Lilenfeld's life and career.
Just as logistics,
- [01:58:35.360]if you applied for
continuing education credit,
- [01:58:38.300]you'll receive a follow up email or
everyone attending will receive a
- [01:58:43.240]follow up email to an
- [01:58:46.960]that you're required to complete if
you want continuing education credit,
- [01:58:51.130]but we would like everyone to complete
to help us plan and improve our future
- [01:58:57.490]And we appreciate your interest
in this program and honoring Dr. Lilienfeld's
- [01:59:01.600]professional legacy.
- [01:59:04.630]Please don't forget to visit
the kudo board link
- [01:59:08.440]that's the area where you can put down,
- [01:59:10.240]share your own tributes and
testimonies about Dr. Lilienfeld.
- [01:59:14.290]Any final words from
our presenters at all?
- [01:59:17.170]Or anything you'd like to
say before we say goodbye?
- [01:59:22.800]Thanks to everybody for attending.
- [01:59:27.030]It really shows how
many people have valued
- [01:59:31.980]Scott's contributions to
the profession. Very good.
- [01:59:36.660]Thanks Dr. Clark.
- [01:59:37.920]I will just echo those same words.
- [01:59:40.260]Thanks everyone so much for
attending this webinar today.
- [01:59:45.690]Thank you. Yes.
- [01:59:48.680]Thanks for attending. And thank
you for organizing this, Jessica and
- [01:59:53.550]Kurt. It was a real pleasure.
- [01:59:58.350]Thank you.
- [01:59:59.730]You bet. Well, thank you again. We
appreciate your time and effort.
- [02:00:02.490]I know you put a lot into your
presentation so we appreciate that. So,
- [02:00:06.600]all right, well, thank you.
- [02:00:09.920]We were glad you were here and
- [02:00:12.510]we hope you have a
pleasant rest of your day.
Log in to post comments