Is More Better When Embedding Ethics in CS Courses?

In this episode I unpack Horton, Liu, McIlraith, and Wang’s (2023) publication titled “Is more better when embedding ethics in CS courses?,” which investigates the impact of one and two embedded ethics modules within undergraduate computer science courses.

Quote as more Computing departments
begin to deploy embedded ethics modules
we Face important questions about scale
and costs what is the ideal number of
embedded ethics modules and how should
they be spaced across an undergraduate
program how do we assess the
effectiveness of these modules is more
better end quote that's from page 657 of
is more better when embedding Ethics in
CS courses which is written by Diane
Horton David Liu Sheila a mcilrath and
Nina Wang apologies for mispronounced
any names we're going to unpack this
paper in relation to Computer Science
Education particularly in the K8 or K-12
space Here's the abstract for this paper
quote embedding ethics modules in
computer science CS courses is an
approach to post-secondary ethics
education that has been gaining Traction
in contrast to Dedicated courses on
ethics and CS embedding ethics modules
into CS courses supports tight
connections between ethical
considerations and CS Concepts as well
as enabling repeated exposure to ethics
across multiple courses initial studies
of the effectiveness of such modules
suggests that that this approach can
increase both student interest in ethics
and technology and student self-efficacy
towards incorporating ethical
considerations in their Computing work
departments wishing to deploy embedded
ethics ee modules need to decide how to
invest resources including class time to
maximize Effectiveness while on
maintaining curriculum objectives such
considerations include the number of ee
module experiences a student has
throughout their degree program as well
as the spacing of those experiences
research to date has focused on the
effect of a single embedded ethics
module in this paper we report on a
study examining the impact of
experiencing ee modules in multiple
courses among our findings our results
suggest that more is not necessarily
better that a modest number of periodic
exposures to eat modules over the course
of a degree program may be sufficient to
achieve sustained positive attitudes and
self-efficacy among students while
pictures beginning to emerge these
results highlight the need for further
research on the effectiveness of
embedded ethics programs as a whole end
quote far to summarize the study into a
single sentence I'd say that this study
invest investigates the impact of one
and two embedded ethics modules within
undergraduate computer science courses
now today's podcast kind of builds off
of what I've been talking about in the
previous two episodes at least if you
skip the April Fool's episode that
released on Saturday so these are
episodes 180 which is integrating ethics
into Computer Science Education
multi-enter and transdisciplinary
approaches and episode 179 which is this
applies to the real world colon student
perspectives on integrating ethics into
a computer science assignment if you
don't know who I am my name is Jared
Leary and I have a background working
with every single grade kindergarten
through doctoral student in a variety of
contexts like music education and
computer science education so I'm going
to draw from my varied experiences to
kind of unpack this paper in relation to
Computer Science Education in K-12 or K8
space so in the introduction to the
paper the authors talk about how there's
an increasing need for including ethics
within computer science education
degrees and I would argue within the
K-12 space as well so as I mentioned in
the previous episodes there are many
ways that you can go about embedding or
integrating or just working on ethics
you could do this as a standalone class
asset is disconnected from the degree
itself a standalone class that is
specific to the degree you could do it
embedded throughout a particular course
or embedded throughout the entire degree
program
Etc this same idea can apply to the K-12
space that I've kind of unpacked that
more in some of the prior episodes so I
won't rehash that here much like
previous authors who mentioned this in
the our embedded Ethics program Section
about how they are focusing on how to
think and not what to think so they're
focusing on making discussions safe and
having conversations around ethics
informed design choices and they
elaborate that on that a little bit more
on page 653 also on that page they
include their research questions quote
rq1 what is the impact of participating
in multiple ee modules sequentially
across semesters rq2 what is the impact
of participating in multiple ee modules
concurrently within the same semester
are Q3 what is the impact of
participating in ee modules robust over
different contexts including different
courses module topics course levels and
delivery modes rq4 does the positive
impact seen in CS2 in this 2022
University of Toronto study replicate
end quote so the study that they're
referring to is they did like a pre and
post they're just kind of trying to see
okay in 2022 what happens if you
introduce an Ethics module within a
class and how does that impact like
self-efficacy and whatnot and so they
found an increase and so they're trying
to kind of explore that a little bit
more and see okay well we introduced one
module what happens if we introduce two
modules or if we do this over multiple
different semesters or what happened
between when they engaged in that model
previously and then they're going into
it like a year later or a semester later
was there any kind of like learning loss
or anything in terms of Changes in
Attitude so the method section on page
this was conducted at University of
Toronto which is a large R1 a research
one institution which means they like
have a high research output and if the
study took place over fall of 2021 and
winter of 2022 semesters if you want to
see a breakdown of like the number of
participants and gender identities Etc
you can check that out on page 654 which
is just a little friendly FYI as a
non-binary individual the other category
can come across as a literal form of
othering all right so let's talk about
the results so this starts on page 654
so they've broken this down by the
different research questions so 5.1 is
titled positive impact decreases and is
restored by a subsequent module end
quote so they basically did a little
pre-post and on the the post-test from
the first time that they were doing a
module to like a following semester
there was a decrease in the overall
scores however the decrease was still
higher than the pre-tests on the very
first time so even though there was like
between the initial treatment which is
like the introduction of the FX module
to the next semester when they are
actually going to go back and like do
another pre-test to see okay has there
been any change since they've last done
this so over a period of time have they
decreased and yes they did decrease in
terms of like their attitudes and
whatnot but it was still higher than it
was on that initial pre-test now the
next thing that they tested was a
post-test after a second round of
treatment and what they did find is that
there was a higher increase from the
second round so they basically described
this almost as like a booster shot in
terms of making it so that the initial
treatment like helped to introduce
ethics and kind of like helped increase
self-efficacy and attitudes and whatnot
it decreased over time because they
weren't engaging with this but then when
they engaged in the ethics module again
a little bit later then it increased it
even higher than it was previously so
then one might ask okay well what
happens if you do even more instead of
just one a semester so the second
section which is on page 6055 which is
titled a second module in the same
semester provides a little added benefit
they indicate on page 656 quote we found
no significant difference between
students who received one module and
those who received two or even between
students who receive zero modules and
those who receive two a little bit
further down in other words receiving
either one or two modules appears to
confer benefit to self-efficacy but two
modules did not confer significantly
more benefit than just one end quote as
I mentioned in recent episodes on ethics
it's important to figure out at what
point is zero soundtrack duration point
when is there diminishing returns for
focusing on ethics within Computing
classes and while this appears to kind
of give some suggestion that well maybe
you only need to do it once a semester I
would argue that we can't necessarily
make any generalizable conclusions from
this maybe the second module wasn't very
enjoyable maybe it didn't connect to
students in a particular way that the
first one did or maybe something
happened in like society that made
ethics less valued or less important
than it was previously or maybe
something happened in the school like
some kind of a traumatic event and that
traumatic event somehow related to
ethics all this could kind of have an
impact on it so we can't necessarily
make any generalizable conclusions but
it does lead to some interesting
questions again what point is the
saturation point for the number of
ethicus courses or embedded modules that
we should include in a class so section
ee modules is robust across many
contexts basically explored how there
are many different classes that could
embed ethics and that they tried it
across these and that it had a positive
impact so it doesn't need to just be
embedded within an introduction to CS
course so this might raise some
questions about well do we need to just
have a standalone ethics course or are
there ways that we can embed this
throughout different types of courses so
this study might suggest that yes we can
include this across different types of
classes so if you're like a high school
educator maybe include it in your app
development course maybe you include it
in your introduction to CS course maybe
you include it in your game design
course Etc but this leads into the next
section which is 5.4 which is on the
replication of results showing positive
impact from CS2 module so they wanted to
basically see well can we replicate what
happened previously which I really
applaud the authors for this because
there's not enough research that
actually looks back at prior research
and says hey can we have a replication
of these results can we duplicate this
in some way to make sure that this was
not just a one-off success yay we found
this positive thing but then if we would
actually try it again we'd find out no
it actually didn't have that same impact
and so they did find that if you
embedded the ethics then it had a
significantly higher scores or ratings
than if you did not which makes sense
now the next section which is the
discussion section kind of highlights or
rephrases or summarizes some of the
prior findings that I just kind of
mentioned but here's an interesting
quote that I'm read from page 657 quote
reflecting on the results of these two
investigations as it relates to EE
program design we surmise that one ee
module is sufficient to achieve a
certain level of attitude and
self-efficacy but it appears that an
additional module is unlikely to further
increase scores furthermore since
attitude and self-efficacy scores were
observed to diminish with time but can
be restored by a subsequent module
experience the results suggest that an
effective delivery of ee modules is to
provide an initial ee module followed by
time delayed subsequent module
experiences this should help to sustain
student attitudes and self-efficacy
levels Across Time end quote if you want
to read more of the study I do include a
link to it in the show notes I highly
recommend taking a look at it but this
is the point in the podcast where I kind
of like to talk about some lingering
questions and thoughts when I read
through these papers so the first
question or thought that I have is is
there a point where you don't think
computer scientists need to focus on
ethics and Computing so as I mentioned
previously is there a saturation point
for understanding ethics and is that
saturation point the same frequency with
which someone should engage in ethical
Computing the two can be very different
one is an understanding and one is
actually applying that understanding
within a context that is real world
let's make this even more concrete so
rather than just focusing on ethics
let's think about this in relation to
accessibility or Equity at what point is
there a saturation point where students
understand how to write accessible and
Equitable programs but should that
understanding stop in terms of the
application of that understanding I
would argue no but maybe you disagree
with me on that and that's okay I'm
curious what your perspective would be
so if we take that same rationale the
same understanding of yes we've reached
an understanding of what ethics is and
how to use it within Computing but
should we stop utilizing that in the
programs that we create or the products
that we design especially when we think
about the the assumptions kind of like
embedded within studies like these so
yes there is an increase in terms of
like the baseline or like the the
control group that did not receive any
kind of treatment did not do any kind of
ethical uh training Etc so while the
group that did receive that treatment
had an increase in self-efficacy and
attitudes at what point is that enough
of an increase so like let's let's take
it away from like a discussion on ethics
so if like on a scale from zero to 100
somebody is like 90 racist and we
engaged in some kind of a module that
helped them to be only 80 racist is that
enough if we found that a second module
only got them to 78 racist should we
stop at the first module because well
there wasn't enough of a decrease in the
amount of racism of the students who
participated and again this is not a
critique of the authors or anything I
enjoyed reading through this paper and I
think it's very important questions to
explore I'm just posting some questions
to the field that we can't just look at
the increases or decreases and go well
we've reached the the saturation point
with this like it's going to have some
diminishing right turns what we really
need to think about is okay but what is
the impact that we want to actually have
on the students that we work with and
the field and you know the people who
use the products and things that are
designed by computer scientists so if
you think that ethics or again if we
think broadly like accessibility Equity
Etc should be embedded at what point
should it be embedded within a unit or a
class Etc so for example one of the
things that I talked about in Prior
episodes is in like the elementary space
in particular there's often a discussion
on using physical Computing so a
question can be asked at what point
should use physical Computing should you
use it to introduce a CS concept should
you use it to reinforce the Cs concept
so like if we have like a beginning
middle and end should of lessons should
it be like a middle lesson of a group of
three or should it come at the end to
like reinforce the computational
thinking or the concept that's being
taught like there's no right or wrong
way to do it I tended to do it towards
the middle so that way you'd have an
introduction you have this need to know
you dive deeper into it by looking at
from a different angle through like
physical Computing and then cool you
move on to applying that like newer or
deeper understanding into the project
you previously started same thing can be
explored with the ethics discussions so
we'll point in a course do you actually
apply ethics and explore it so if you're
learning like a a new
um concept or you're creating some kind
of like a new app or new design do you
do this before you learn the concept do
you think about ethics would that cause
overwhelm in that you're trying to learn
a new thing while also learning how to
apply it in an ethical way it's not
going to be too many things from the
cognitive load standpoint or do you put
it towards the middle of a semester or a
unit or Etc or do that at the end or is
this like a standalone thing in between
different units and different projects
Etc so I don't have a necessarily an
answer where I think that you as an
individual should do it but I do think
it's something important that we should
think through as a field and as
individuals within the field especially
if you're only going to do it one time
for a semester or for a course it's
important to think of when you do that
because you don't want to overwhelm
people but you also don't want to make
get this like tangential add-on after
you've learned the real stuff oh here we
go we're going to sprinkle in some
ethics do a little Salt Bay with the
ethics don't want to do that but another
question that I have is how might a
person's understanding of Ethics in
Computing impact their careers in
Computing so one of the things that I
mentioned previously is that there are
many different ways of thinking of
Education from a design standpoint or
from like a student standpoint some
people go into CS through a boot camp
approach where they spend I don't know
like six weeks or something just doing a
deep dive into one language one very
specific thing to get a very specific
type of job at the end of that other
people go through four plus year degrees
in order to learn computer science more
broadly speaking and then they are going
to be able to apply to many different
types of jobs to do many different
things should the boot camp focus on
ethics or should the only the broader
degrees focus on ethics now how does
that have an impact on the different
careers in Computing so if we make a
decision that okay people who take the
Cs courses are going to do some ethics
but the people who do the boot camps are
just going to focus on like the skills
and Concepts needed to be able to
program and not necessarily the broader
implications or impacts of computing
will that impact then who actually gets
to focus on the design side of things so
are there going to be a lower tier of
like let's say programmer who is going
to just focus on executing the design
decisions of somebody who is a higher
tier programmer who's actually thinking
through the ethical implications but not
necessarily spending the majority of
their time doing programming that is
instead divvied out to people who are
have a lower paycheck than the people
who are actually focusing on the design
does that then create kind of like a
glass ceiling between those who are
making those broader decisions and
focusing on equity and those who are not
and will it get to a point where like
companies are going to hire Consultants
who specialize in ethics as opposed to
hiring generalists who have kind of
dabbled in ethics and Computing if you
just kind of think of it as like again
sprinkling it into like your program as
opposed to from the ground up it is a
core part of what it is that you are
creating that's going to require
different kinds of expertise or
different kinds of thinking that's going
to occur within that company or within
that product development again I don't
have a right or wrong answer for this
but I think it's something that we
should all kind of think through as a
field which is why I've been doing this
little mini series on ethics and
Computing and you can check out the
other episodes on that in the show notes
at jaredollary.com and you can just
click the link in the app that you're
listening some will take you straight to
that thanks so much for listening to
this episode I hope it raised some
questions for you to consider with the
Cs classes that you work with and if
you'd like to talk about your thoughts
on when and how to embed ethics within a
K-12 or even higher education class
happy to have you join for an interview
there's a contact me button on my
website at jaredelier.com stay tuned
next week for another episode till then
I hope you're all staying safe and are
having a wonderful week

Article

Horton, D., Liu, D., McIlraith, S., & Wang, N. (2023). Is More Better When Embedding Ethics in CS Courses? Proceedings of the 2023 ACM SIGCSE Technical Symposium on Computer Science Education - SIGCSE ’23, 652-658.

Abstract

“Embedding ethics modules in computer science (CS) courses is an approach to post-secondary ethics education that has been gaining traction. In contrast to dedicated courses on ethics in CS, embedding ethics modules into CS courses supports tight connections between ethical considerations and CS concepts, as well as enabling repeated exposure to ethics across multiple courses. Initial studies of the effectiveness of such modules suggest that this approach can increase both student interest in ethics and technology, and student self-efficacy towards incorporating ethical considerations in their computing work. Departments wishing to deploy embedded ethics (EE) modules need to decide how to invest resources, including class time, to maximize effectiveness while maintaining curriculum objectives. Such considerations include the number of EE module experiences a student has throughout their degree program, as well as the spacing of those experiences.

Research to date has focused on the effect of a single embedded ethics module. In this paper, we report on a study examining the impact of experiencing EE modules in multiple courses. Among our findings, our results suggest that more is not necessarily better — that a modest number of periodic exposures to EE modules over the course of a degree program may be sufficient to achieve sustained positive attitudes and self-efficacy among students. While a picture is beginning to emerge, these results highlight the need for further research on the effectiveness of embedded ethics programs as a whole.”

Author Keywords

Ethics education, embedded ethics, impact of technology on society

My One Sentence Summary

This study investigates the impact of one and two embedded ethics modules within undergraduate computer science courses.

Some Of My Lingering Questions/Thoughts

Is there are point where you don’t think computer scientists need to focus on ethics in computing?
- Is the saturation point for understanding ethics the same as the frequency with which someone should engage in ethical computing?
Where should ethics be embedded when learning something new?
How might a person’s understanding of ethics in computing impact their careers in computing?

Resources/Links Relevant to This Episode

Other podcast episodes that were mentioned or are relevant to this episode
- Integrating Ethics into Computer Science Education: Multi-, Inter-, and Transdisciplinary Approaches
  - In this episode I unpack Goetz’s (2023) publication titled “Integrating ethics into computer science education: Multi-, inter-, and transdisciplinary approaches,” which unpacks three approaches to integrating ethics with computer science education.
- “This Applies to the Real World”: Student Perspectives on Integrating Ethics into a Computer Science Assignment
  - In this episode I unpack Jarzemsky, Paup, and Fiesler’s (2023) publication titled “‘This Applies to the Real World’: Student Perspectives on Integrating Ethics into a Computer Science Assignment,” which explores student perspectives on an undergraduate ethics assignment in a CS class.
- More episodes related to ethics
- All other episodes
Find other CS educators and resources by using the #CSK8 hashtag on Twitter