The intersection of videogames, music, and computer science education

In this episode I unpack my dissertation, which explores the intersections of videogames, music, and computer science education.

  • Back when I used to teach general music in a K-6 district, I had some teachers who would often ask me to like, Hey, can you create a song about math facts? That way we can do some interdisciplinary collaborations together. I problematize these forms of integration and many other podcast episodes that I have listed in the show notes for this episode.

    They don't use what Bresler might describe as a co-equal design, where the two disciplines are kind of position equally with each other. Instead, it's often positioning one higher than another, where the other is in a subservient relationship. And quite frankly, a lot of these integration ideas don't really engage in practices that kids could actually do outside of school settings.

    They're too artificial or contrived. Now, one of the reasons why I actually jumped into computer science education is because during my doctoral work, I happened to audit a course that was on coding musical apps, and I kept asking myself, When is this music? When is this coding? And when are the two inseparable? Those questions led me back into the classroom, but this time as a K-8 computer science educator and I are specifically trying to focus on art space programing and programing that just integrated a bunch of different topics.

    Domains, practices, languages, etc. I talked about that in a bunch of different other podcast episodes. I'm not going to elaborate them on here. Instead, today's episode is actually going to talk about a holistic form of integration that uses computer science, music, entrepreneurship, visual art practices and so many more. And this is through a practice known as Chip Tunes, which is a type of music that is made from or in order to sound like retro video game computers and consoles.

    So all the little bleeps and bloops that you might hear in some of the old school video games are often described as chip tunes. When I investigated a chip tune discussion forum to find out what kind of practices were discussed when creating this kind of music, I found some intriguing themes and subthemes. The main themes that I discovered were composition practices, performance practices, maker practices, coding practices, entrepreneurial practices, visual art practices and community practices all existing within the same space.

    Because his dissertation is hundreds of pages long. I'm not going to go in everything in detail. Instead, what I'm going to do is give you a brief overview of some of the main findings as it relates to computer science educators. And then just kind of gloss over some of the things that might be less interesting to you. Okay.

    So let's very quickly just go over what you would find in chapters 1 to 3. If you're interested in reading more about it in the introduction chapter, I talk about, well, what are chip tunes? What do I mean by making and multidisciplinary practices? And a concept that I introduced known as music centered making. I then discuss the discussion forum that I investigated for this, which is chip music dot org.

    Now Chip Music dot org is a discussion forum that at the time that I was doing this research had over 11,000 registered members and it looked at every single discussion forum post from December 30th, 29th through November 13th, 2017. This was 245,098 discussion forum post, which resulted in 10,892,645 Words to Analyze and in chapter three to talk about how exactly did I analyze that much data without reading every single word.

    However, the remainder of the introduction just kind of talks about, well, what are the central questions and phenomena that I'm looking at? What do I mean by discourse and discourses? Corpus This is the discourse, analysis and other nerdy stuff, like what is my position ality on all of this? If you're interested in reading more about it, it's a fairly short chapter, so take a look at it if you want to.

    It's a link in the show notes and it's for you. Chapter two is a review of literature. In this chapter I talk about, Well, what are chip tunes? In case you want to learn more about it, nerd out about it. It's really fascinating. How does this relate to the mod scene and maker culture? And then what are some critical perspectives on maker culture?

    I've actually done a podcast episode that unpacks some of those perspectives and a link to that in the shownotes. And then finally, in chapter two, it talks about the like online music spaces, because this was for a PhD in music education. So the target audience is music educators, but there's a ton of it as relevant to educators outside of music education.

    Now, Chapter three is where we get extra nerdy and we talk about the method in design for stuff. So if you want to hear about like, what do I mean by the definition of discourse? And check out the beginning of that as well as like, what are some different ways that you can analyze discourse? I happen to use an approach that was perfect for analyzing nearly 11 million words of data, but there are many other ways that you can actually analyze discourse that would be relevant for other sets of data, context, etc..

    And so you can learn more about those different approaches in chapter three as well as just like the overall research design, like an overview, what is the setting, How did I do data selection, collection and cleaning and then data analysis. So there's two distinct phases, basically. So one was kind of like analyzing patterns of discourse using word list, lexical frequency analysis, dispersion, concordance, collocation keenness, etc. And then the other phase was using discourse analysis techniques to talk about or explore the significance, practices, politics and connections within the practices that were being uncovered in the data.

    Also talk about, well, how do they organize these into different themes? And just like what were the research relationships and ethics involved with this kind of data analysis? Again, chapter three gets pretty nerdy, so feel free to explore that one if you want to learn more about this. But now we get into the largest chapter. This is the revealed practices.

    This chapter is over 100 pages long because again, I'm looking at 11 million words of data. There's going to be a lot of stuff to talk about. The way that it organizes Chapter four was to present in order of most frequently discussed. So the most frequently discussed topics was actually composition practices, which for music educators is like mind blowing.

    It's like, Wait, what? They weren't just talking about performing. They were talking about how to compose most music. Educators, quite frankly, don't do that enough in their classes. They're teaching students how to perform the works of others rather than how to create their own compositions for other people to perform, whether it be a human or an algorithm. So if you want to learn more about that, check it out.

    As there are some really interesting discussions on different chip tune appropriations, sound synthesis composition, tools like music theory, whether or not that was relevant and a concept known as fake bit, which I'll just give you a teaser for. And if you want to learn more about it, check out chapter four. The next most frequently discussed practice was actually performance practices, so they talk about using a Gameboy as a performance instrument That was the most frequently discussed instrument that was actually mentioned, which is fascinating.

    Literally going up on stage with a Gameboy and either performing with it or triggering some pre composed samples of like different compositions that they've like kind of sequenced together, etc.. There's a lot of really cool things that are discussed in that. So check out that section of chapter four as not only do they perform with electronic instruments like a Gameboy, but they also like did it in like small chamber ensembles with acoustic instruments, etc..

    So it's a really interesting little subculture to explore if you're interested in learning more about it. And now let's get into the first section that is most relevant to computer science educators. So this is the maker Practices Maker practices engaged in something known as hard mats, which are hardware modifications. So it could be a something as simple as like changing the way a device looks using esthetic mods.

    So like they're going to paint their Gameboy to make it like resemble some of their favorite characters related to gaming or whatever. So these changed the way a device looks, but not necessarily how it functions when they actually change how a device function. This was known as a functionality mod, so a very common example was something called a pro sound mod.

    So when somebody is performing or recording with the Gameboy, the original headphone jack that you might have used with like these little like three and a half millimeters didn't necessarily sound great when you plug them into large speakers like a P.A. system, there would be all this like extra noise and it does not have a good capture or a good projection of whatever it was that was being played on the Gameboy.

    So people actually modified the devices by bypassing the internal sound system and adding in their own audio ports so that way they could like plug in two different channels into it and it would get rid of that noise and actually amplify the sound. It sounded better than the way it was designed, and this was one of the most common modding practices that was discussed.

    However, modding practices ranged from modifying devices that were already preexisting all the way to manufacturing and designing individual components or devices. So as an example, somebody created a little chip that would plug into a Sega Genesis or mega drive that would allow a digital audio workstation on a modern computer to actually control the sound chip of the Sega Genesis or Mega drive.

    This person designed it, manufactured it and sold it. So there's quite the range of hardware modifications going on. What was really interesting to see is there's a lot of sharing, reading and even creation of circuit diagrams. Now some of these were created on the computer and some of them are like literally just kind of like drawn out like on a napkin and just like taking a picture of it and shared it on the discussion forum.

    Another really interesting finding that related to Maker practices was a concept known as circuit bending. That circuit bending is like when you open up some kind of a toy that makes music or whatever, usually not something that you plug in because you want to have a low voltage, maybe like a stuffed animal or something. When you open up that circuit, what you do is you take a wire and you touch one part of the circuit and then you touch another part and you make a sound like squeezing the animal or whatever, and you see if it changes it.

    And then you try it again with another part and you just keep touching different metal parts on there. Again, low voltage and to see how it adjusts a sound, then if you find something that adjusts a sound, you can actually add in some switches, some potentiometer errors, all sorts of things to modify the music or the sound to make it so that you could go faster or slower, etc. There's a lot of really unique things that can be done with this.

    So check out this section on circuit bending if you want to learn a little bit more about that. Now on page 139, I talk about some perspectives of mining. Many of the people who engage in mining practices said that part of the enjoyment of making chip tunes was actually modifying some kind of device. They thought of it as like an inseparable process that was needed in order to make chip tunes, and others were actually so interested in modding that they weren't interested in the actual music making side of things.

    They just were there to discuss the mining practices. So there's quite a range of different perspectives. But they all felt that these like hardware practices were relevant to the music making. Now, one of the interesting things to note is that this kind of leads into the next most disgusting, which was actually coding practices. So a lot of these devices that were made required some kind of programing.

    So software mods weren't as common as hardware mods. However, they were frequently discussed enough that people were sharing source code or even engaging in collaborative development, not because source code was shared so much when a developer would abandon a project, there were actually people who would pick it up and just go in and read the comments and like learn how to like add on to or try and expand the work of an original developer.

    So for example, like creating a tracker, which is basically like software to be able to compose music on like a Gameboy or like a Sega Genesis or mega drive, whatever, you're able to sequence together a bunch of different beats, notes, all sorts of things using the sound chips in the devices. These had to be programed because there's no native way to do that with like in any case or whatever Nintendo entertainment system, that when developers were actually working on these projects, they would sometimes work on them on their own or in small teams or even large teams.

    And this often included some kind of feedback with the community where they'd share updates and ask like, Hey, what did you think of this little update that we added? And some of these actually grew into some pretty large projects where there were many people all working on some kind of piece of software that related to CHIP tunes, whether it was like to make music or even to like enhance the visual design on the stage that was being projected behind the musicians.

    And it's interesting to actually read some of the developer perspectives on this because while some developers felt appreciated, others felt underappreciated and like, why did I spend so much time working on this stuff? And all people do is complain about it. So if you want to learn more about that, check out the section on coding practices. Now, what's fascinating with these coding and the maker practices is that people learned by curating or creating tutorials to help other people or by simply engaging in a discussion forum like asking question, Hey, how did you do this thing?

    And somebody might write up something about the steps that they took to be able to do that so somebody else can replicate it. But when it came to the code because they were sharing source code, they were also adding in different comments and whatnot that people were then going into and actually learning how to program by reading the comments and then reading the code and going, if I change this parameter or this variable or whatever, what will happen and how will this change?

    And they'll just kind of tinker around with it, modify some things, and it might break it or it might make it do something new or different. So this is actually how some people learned how to code with the goal of making music for some kind of device like a Gameboy. Now, on page 164, there's a little subsection on questioning the relevancy of coding practices.

    This was fascinating to read through some of the discussion. Forum members actually posed the question of whether or not discussing coding was relevant for chip tunes or making music, and people responded saying, Yeah, this is more than relevant. This is like really essential to what I'm doing. I'm learning a lot by seeing people posting source code or by reading these discussions that people are having on coding.

    And this relates to other research that I cite in here that kind of found some similar things with other intersections of music and coding in holistic ways. So again, this is not this contrived like, Hey, make a song about math facts. This is no, this is people who are actually making music and performing and selling their albums, etc., using music practices combined with computer science practices in a way that is actually relevant to people both inside of and outside of school settings.

    In fact, some people pose the question like, Is there a place that I can go to like study this at university? I want to like, I'm in high school right now and I want to be able to like, get a degree in chip tunes. Can I do that somewhere? Which actually kind of leads into my Chapter five, which is on multidisciplinary.

    So individual topics, posts and even media often demonstrated a variety of interdisciplinary practices and discourse that was either relevant or needed for the discussion. These discussions weren't isolated into little disciplines or even like sub forums or threads where it's like in this sub forum you can only talk about computer science and in this sub form you can only talk about composing.

    A single post would often jump around between first. I did this hardware practice and then I engage in the software practice which led me to this composition practice, which led me to a performance practice, and then I went and sold it and that might be all in one paragraph. Or am I being a topic where people are jumping back and forth and talking about how they're doing some maker practices here and then some coding practices here and then some composition stuff right there, and it's all within the subtheme that all relates to making music in some way, whether as a process or as a result of the process that they are currently working on.

    So in other words, they're either making music by going in, performing or composing, or they're making music by thinking through how they're going to do design some kind of a hardware or software to be able to make music. In this chapter, I raise some questions that I think are worth considering as an educator, if you're going to engage in computer science, it's going to connect with other disciplines and how it does that is important to consider.

    I've got several podcast episodes that talks about integration now link to those in the show notes, but this kind of leads into the final chapter, which is chapter six, which is on the implications. So in this chapter, I talk at the very beginning about how I struggled searching for a framework to describe this kind of music making, or as I describe, music centered making.

    I pulled a little bit from curricular scholarship, from maker culture, scholarship, from music education, from computer science all over the place. And I was finding stuff that worked well for some aspects, but not in others. As I mentioned earlier, there were also entrepreneurial practices, visual art practices and community practices. Where do all of those fit in this intersection of computer science, music, etc.?

    What framework can encompass all of that in a way that is meaningful? Make your culture scholarship was the closest in multidisciplinary, happened to work for what I was trying to describe, but it can also be described as integrated or an interdisciplinary or interdisciplinary, which is different than interdisciplinary, transdisciplinary, etc. But in the subsection on Chapter six titled Music Centered Making, I talk about what is no curricula and how does it relate to music centered making.

    So all curricula are the things that are left outside of a curriculum. So as a former curriculum developer, whatever I put into the curriculum was the thing that you would work on. The stuff outside of that curriculum would be considered in all curricula. The stuff that is not discussed, the stuff it does not engage with, with any single topic or lesson or unit or course or degree.

    There's going to be a ton of stuff that you learn and even more stuff that is left out of it. It's so important for us to consider the whole curricula that I actually have a podcast episode that discusses a chapter by Eisner that I will link to in the show notes that kind of unpacks this a bit more.

    The thing about no curricula is that concept can be applied to standards. As I've discussed before, the standards that are written in stone in some kind of a discipline like computer science are what teachers are going to tend to focus on, which is what students are going to learn. Anything that is not considered a standard is going to be like the normal standards or the normal curricula.

    What is not discussed now, when we start to look at these practices that are at that intersection or nexus of multiple disciplines, how do we account for standards that can't account for this stuff? So when it came to composing and performing, there were some music standards that might have related to that. But what about all the hardware and software practices when it came to building and manufacturing devices or coding those devices?

    If we look at it from a computer science perspective, which I talk about to the core standards for computer science in this chapter, it's the other way around. So we might have some standards that can talk about the maker and coding practices, but what about all the composition practices and performance practices? What about the entrepreneurial practices? Those are not in there.

    So our teacher is supposed to kind of combine those practices together and combine those standards into one set, one giant set of like music standards and computer science standards and entrepreneurial standards, etc.. Well, discussion for members considered many at least, that this was part of the enjoyment of making music on Gameboys and was to be able to engage in all these different like practices and discourses, etc..

    But if you are to do this into a school setting, what would that look like? Who would do it? Would it be a teacher that is a music educator? Would it be a computer science educator? Would it be a collaborative course? And would you create an interdisciplinary that is just on chip tunes and then that would create its own set of standards, its own curricula, and then it would be codified into something that would have its own novel curricula.

    Would you create a new set of problems? Because again, things would be left outside of that. So I problematize that in the section on ship tunes as interdisciplinary. Again, you can't separate the music from the computer science in these kinds of examples. It is too embedded within it. It is part of it. You have to do it on page 215.

    Here's an example of that quote. Consider a simplified example of what a person would need to know to enable user controlled volume within a tracker. From a music education perspective, a person would need to understand that sounds and music can change volume and they might label that concept dynamics, volume or even amplitude. From a computer science perspective, a person would also need to understand that a symbolic label can keep track of a value that changes through user interaction, and they might label that concept a variable without understanding that volume can change.

    Music education concept and that a variable that represents the numeric value of the volume can change through user interaction. Computer science concept A person would be unable to create the simple interface. This simplified example demonstrates a person would apply concepts from both disciplines to enable user controlled volume within and tracker input. So again, you could create an interdisciplinary with this intersection of computer science and music.

    But what about for the next intersection of computer science and something else like art? Are you going to be creating more and more courses for every single one of those? I certainly hope not. And we're going to offer a thousand courses on this, like who's going to be qualified to teach all that? You're going to find an expert in every one of those intersections.

    Good luck. As somebody who has three degrees in music education, who's been teaching computer science education for years, I can say there's very few people out there like me, which again is one of the reasons why I talk about the importance of being comfortable with facilitating multiple different types of projects in the same class, even on a topic that you might not understand or know.

    It's all linked to some episodes in the show notes. I kind of unpack that more now instead of trying to create a new class for every type of intersection. Instead, what we can think of this as chip tunes, as an exemplar of transdisciplinary engagement, where learners bring together a multitude of disciplinary practices to solve the problem of making music through computer and video game hardware and software.

    In quotes from page 218. And if you want to learn more about that particular topic, take a look at that section. So only a couple of pages long, but it's pretty interesting if I do say so as the author of it, and I unpack some of these concepts a little bit more in episode 121, which is titled Intersections of Popular Musicianship and Computer Science Practices, which I'll link to in the show notes, as well as some other episodes that kind of unpack this nexus of music and computer science.

    Now, the next main section of Chapter six has a lot of stuff that might not be of interest to you. It's a multifaceted music making. However, one key takeaway that might be of interest to computer science educators is the idea that members were able to freely move between disciplinary practices, different hardware, software, engagement or the lack of etc. They are not forced to finish a project.

    They are not forced to discuss something that might not be of interest to them. I talk about this more in the Affinity Space podcast episode, which is episode number 89. In is titled Applications of Affinity Space Characteristics in computer Science Education. So learn more about how you might actually create a space that allows for all of that all within your computer science classes.

    And then a final little subsection that might be of interest to computer science educators is talking about incorporating collective learning practices. So these are practices that are supposed to try and sustain engagement or participation within some kind of a community. So in this case, it might be a learning community. If you want to learn more about this, take a look at that particular section towards the end of the chapter.

    Now, this was a very quick overview of couple of hundred pages of my dissertation. Hopefully it kind of teases you to make you want to actually go and read a little bit more of it or to at least check out some of the other podcast episodes that unpack some of these ideas a little bit more. Now, you may have noticed that I haven't posted an episode in a while because I actually got a sinus infection or some kind of a cold or whatever that like really messed with my congestion and made it so my throat sounded really funny for like a month, so I wasn't able to record a podcast episode.

    Now during that time, I was feeling well enough to spend a lot of time writing music and then like scripting out some drumming and scratch videos that will be coming out on my YouTube channel and website. And during that time I realized that, quite frankly, I wanted to spend more time doing that, creating more drumming, gaming and scratch content, which means that I need to take away time from something else.

    And in this case, the podcast I've created over 200 podcast episodes, had some interviews with some phenomenal guests and just loved it. But at some point I have to realize that I don't necessarily have time to do everything. In fact, a lot of people ask me, How do you have time to do all this? Now another thing is I honestly just want to read outside of the scope of scholarship and possibly even share what I'm learning from like philosophers or whatever, some topics that are just outside of computer science education or just education in general.

    I want to be able to make time for that without taking away from all the gaming, drumming and scratch videos that I create. So when I guess I'm saying is I'm putting an indefinite pause on the podcast episode, Maybe if I see a paper that like really interest me and it relates to computer science, I'll do another episode in the future or like feature presentation at a conference which I haven't been submitting for either because again, I'd rather spend my time working on gaming and drumming and scratch stuff rather than just doing a presentation.

    So stay tuned for more content that I'll be creating. But whether or not it's going to be a CSK ed podcast, I don't know. I'm really looking forward to sharing with all of you all the things that I create and you can find all that on my website. Editor Jared O'Leary, icon, where you can also find all the different socials that I'm on and all of the gaming drumming and computer science education content neatly organized into different tabs at the top of the screen.

    Thank you so much for listening. I hope you all stay safe and are having a wonderful week.


Abstract

“This study examined discussion forum posts within a website dedicated to a medium and genre of music (chiptunes) with potential for music-centered making, a phrase I use to describe maker culture practices that revolve around music-related purposes. Three research questions guided this study: (1) What chiptune-related practices did members of chipmusic.org discuss between December 30th, 2009 and November 13th, 2017? (2) What do chipmusic.org discussion forum posts reveal about the multidisciplinary aspects of chiptunes? (3) What import might music-centered making evident within chipmusic.org discussion forum posts hold for music education? To address these research questions, I engaged in corpus-assisted discourse analysis tools and techniques to reveal and analyze patterns of discourse within 245,098 discussion forum posts within chipmusic.org. The analysis cycle consisted of (a) using corpus analysis techniques to reveal patterns of discourse across and within data consisting of 10,892,645 words, and (b) using discourse analysis techniques for a close reading of revealed patterns.

Findings revealed seven interconnected themes of chiptune-related practices: (a) composition practices, (b) performance practices, (c) maker practices, (d) coding practices, (e) entrepreneurial practices, (f), visual art practices, and (g) community practices. Members of chipmusic.org primarily discussed composing and performing chiptunes on a variety of instruments, as well as through retro computer and video game hardware. Members also discussed modifying and creating hardware and software for a multitude of electronic devices. Some members engaged in entrepreneurial practices to promote, sell, buy, and trade with other members. Throughout each of the revealed themes, members engaged in visual art practices, as well as community practices such as collective learning, collaborating, constructive criticism, competitive events, and collective efficacy.

Findings suggest the revealed themes incorporated practices from a multitude of academic disciplines or fields of study for music-related purposes. However, I argue that many of the music-related practices people discussed within chipmusic.org are not apparent within music education discourse, curricula, or standards. I call for an expansion of music education discourse and practices to include additional ways of being musical through practices that might borrow from multiple academic disciplines or fields of study for music-related purposes.”


Resources/Links Relevant to This Episode



More Content