Videogames in Empirical Aeshtetics
Videogames are kind of a big deal by now.
Jenkins [3] refers to them as “the new lively art”: An art form scrutinized by the established art world but, in fact, the art form most consumed by the public. While I would argue that we have come a long way since 2005, with videogames being a staple of mainstream culture now more than ever, videogames definitely did not yet arrive in the field of empirical aesthetics.
They did arrive in many other fields of academia. Videogames are often discussed in design focused areas like Human-Computer-Interaction (HCI) and obviously also in psychology. The loudest output of the latter, sadly, focuses on the looming link of videogames and violence. A discussion not quite done, exemplified by [1] and [2].
But are videogames a fruitful subject for empirical aesthetics?
I believe so, yes.
For three reasons:
1. Videogames are, as mentioned above, one of the most consumed forms of art. The videogame industry is huge, not even counting the cottage industries that have grown around it, like YouTube and twitch.
2. Videogames might be a great tool for laboratory studies on the aesthetic experience.
3. Videogames engage perceivers on every modality, making them probably the most complex and involved form of media to consume.
Here I want to talk about via which modalities videogames engage a player, two perspectives on could take when studying them, and how a videogame might be turned into a studiable stimulus.
On Videogames
From a cognitive psychologist’s viewpoint, videogames are complex beasts. Comprised of a myriad of stimuli with different modalities, usually experienced over hours. In order to study the experience of videogames, we should therefore first try to divide said experience up by the senses they address.
Visual
Like artworks, videogames feature a strong visual component. In style as diverse as traditional art. From the art deco architecture of Bioshock (Irrational Games, 2007), the three dimensional recreation of M.C. Escher’s works in Fragments of Euklyd (NuSan, 2017), the Art Nouveau world design in Abyss Odyssey (ACE Team, 2014), the watercolor style of Child of Light (Ubisoft Montreal, 2014), to the pixel art of Owlboy (D-Pad Studio, 2016): The current videogame landscape uses the computing power of modern systems to create elaborate visual art. However, videogame art does not solely serve aesthetic demands. As the player has to interact with the visuals, said visuals have to stay readable and quickly assessable. An intricate wall texture, for example, will infuriate a player if an enemy, or even UI elements, blend into it, hindering the action of the player.
Auditorial
Videogames further feature soundtracks equally as diverse in style on top of rich soundscapes. Setting scenes or communicating information to the player. For example, a player of Hollow Knight (Team Cherry, 2017), will hear the music of the soundtrack, as well as ambient sound effects like crickets or leaves rustling in the wind. They will be able to determine by sound alone that an enemy jumps out of the bushes. The player will hear their sword swing and the enemy die in a distinct ‘puff’ noise. Similarly, a seasoned player of Battlefield 5 (Dice, 2018) will be able to tell by hearing alone, if they missed or hit an enemy and if their hit dealt regular damage or if they hit a vulnerable spot, doing extra damage and usually immediately dispatching the enemy. Like the visual aspect, the soundscape has an aesthetic/world building aspect (soundtrack, leaf rustling) and an information aspect (enemy is missed, hit, dispatched).
Kinaesthetics
While players can usually not feel the happenings in a videogame per se, videogames do feature a pronounced sense of feeling. On a direct level, game controllers possess small motors to create vibrations. Like smartphones, which usually rumble when the touch screen is pressed, vibration feedback via the controller gives the player additional information: A controller might violently shake when the player’s character takes damage or give a quick bump when the character lands after a jump to signal that the character can start running again. Even without rumble functions, perceivers feel the keys of the keyboard or the joystick in their hand. Their key presses, button-pushes and joystick-tilts translate directly and reliably into reactions on screen (E.g: A player’s finger presses “w” and their character will always walk forward). The tactile experience of videogames does not end here, however. With subtle tricks good videogames can use visual, auditorial and tactile feedback to induce a sense of virtual kinaesthetics in the player. For example displaying the movement of the player’s character as having weight.
Dark Souls is a good example for this: Compare the two videos below: the first showing a light character, carrying a light weapon the second showing a character who is and carries neither (Dark Souls III, From Software, 2016).
Both characters are given more or less similar inputs (attacks, movement and dodge rolls) but react to them quite differently. While the light character is quick and nimble, the heavy character not only winds up their attacks and movement, they also dodge roll way slower. In addition, the screen shakes when the heavy character hits the ground to convey the impact of the body and the club. It is important to note, that the heavy character doesn’t react slower. When an input is pressed, both characters immediately start their action so that the player has a sense of controlling the character and not just giving it “suggestions”. This is a perfect example for Game Feel [6], which is a positive quality of a game and describes this feeling of immediacy, weight and being part of the game world. Swink (2009) calls these parts Real-Time Control, Spatial Simulation and Polish. I, however, would go a step beyond videogames just causing this game feel and argue, that videogames actually engage our sensory modality of kinaesthetics: We feel the push of the “Dodge” button, see ourselves (read: The character on screen) dodge and feel like feeling the weight of our body hitting the ground. Similar to empathy, which is argued to activate our own pain pathways when seeing other people in pain [5], I believe a good videogame can activate our sense of physical presence. See also the concept of spatial presence [7] which is discussed further down.
Story and Motivation
In addition to the moment to moment game play, videogames feature narratives and complex semantic information which put the players actions into context and gives them a concrete goal. This goal may be as simple as ‘Get the most points’ in a puzzle game like Tetris (Paschnitnow, 1984), or as complex as ‘Escape the objectivist nightmare that is the underwater city of Rapture’ in Bioshock. Whatever the goal, videogames usually make them clear, quantifiable and obtainable: Reach the next level, defeat this enemy, beat this high score. The framing of the player’s actions is something game designers can play with. For example, the Call of Duty series tend to frame the player characters as patriotic heroes
Interactivity
The last big element of videogames is the interactivity.
A videogame is a work of art that reacts to the player and gives feedback whether it is engaged correctly or not. For example: A beholder of Duchamp’s Fountain (1917) failing to understand the message of the artwork will still be able to look at the whole installation. The failure of grasping the artwork is implicit. A player failing to understand the game play of Dark Souls (From Software, 2011) will not be able to experience the whole work as they are barred from progressing. Depending on the videogame, a player will need to become proficient in, if not master, the videogame before they will able to completely experience it. Videogames feature implicit failure states as well. Dark Souls, besides being hard, is especially famous for veiled storytelling that allows players to completely miss out on the bigger picture of their characters actions, even on consequent play-throughs.
On Studying Videogames
The experience of videogames can be viewed in a dual way: First, treating the videogame as the art (and therefore as the stimulus). Second, looking at the art presented in the videogame. I.e: Am I looking at the experience of Bioshock or am I looking at the experience of the art deco advertisements hung on the walls of Rapture, the city Bioshock is set in. The first perspective should look at all the parts I dissected earlier. When a study ignores, for example, the quality of the game play the experience of the artwork cannot be gauged. Imagine playing a game that, while featuring “beautiful” graphics, has extremely boring game play. The initial liking of the visual experience will suffer after hours or even minutes of boring, uninteresting game play. At the same time, “ugly” graphics can start to be overlooked after a view minutes when the game play is engaging enough. Doom (id Software, 2016… and 1993, too) would not be the same if the shooting of hell demons would be removed. At the same time, however, if a player would be bad at the shooting of hell demons, they would probably have a bad time playing the game as a whole [4], which in turn would change the perception (or at least rating) of, for example, the visual aspects of the game. Simply choosing an “easy” videogame might not be enough either, as many games that lack engaging second to second game play often derive their incentive to play from the overarching story (e.g. Gone Home (Fulbright, 2013) or Firewatch (Campo Santo, 2016)).
The second perspective, studying the experience of art inside the world a videogame proposes can equally be valid. It will, however, study a vastly different experience. The first, “videogames as the art”-perspective sees playing a videogame as the same level of experience as looking at an artwork. The second perspective treats looking at an artwork in the videogame as looking at an artwork in, say, a museum. In the second perspective, a player is not playing a videogame, but standing inside the virtual space and experience the artwork as if they were really standing in front of it. All other ignored aspects of the videogame (i.e. sound, virtual kinaesthetic, interaction) are not part of the experience of the art, but part of the experience of the world. If one is not interested in the visual aesthetics presented in the videogame, the constellation of what is the aesthetic experience and what the “world experience” changes of course.
But how can videogames be a world to be experienced? I argue videogames do this by virtue of eliciting spatial presence. [7] postulated a model of spatial presence, which proposes that videogames (or media in general) can capture players attention and switch their primary ego reference frame (PERF) from the real world to the mediated world. I.e: A player focuses on the videogame and (to an extend) only perceives the stimuli and action possibilities in the videogame. An interesting example for game developers playing with this is the “Psycho Mantis” boss fight in Metal Gear Solid (Konami, 1999) in which players actions are always countered by the boss until the players unplug their controller from the console and plug it in the second controller port in the real world . This fight purposely forces players, which see all the action possibilities they have in the game fail, to mentally break out of the mediated space, switch their PERF back to the real world to even acknowledge the possibility of replugging their controller.
The two, here discussed perspectives serve different purposes. I believe that if one want to study videogames as art, they have to first take the “videogames as the art”-perspective in order to gauge the general experience of videogames. Especially the tactile, virtual kinaesthetic experience should not be overlooked. The second perspective is not without merits, however. On the on hand, the content perspective could allow for simulating aesthetic experiences in a highly controlled, but still ecologically valid environment. On the other hand, the study of perception in the game world is highly interesting for both game studies and the psychology of perception. This in turn should mean, that aesthetic stimuli in the videogame world are not perceived as occupying a different space, but the same space as the perceiver.
Game play Loops as Stimuli
Finally, taking all these different facets together, how could one potentially study a videogame in the scope of empirical aesthetics where the controlled exposure to stimuli is paramount for many experiments?
I believe, for many videogames the trick to study them efficiently lies in identifying and defining suitable game play loops. Let’s take Doom (2016) as an example.
Doom has an overarching plot unfolding in a circa 10 hour campaign, which is mostly irrelevant. The avatar of the character (The Doom Marine) makes this clear at every time they encounter the story. The players investment in the story might be there, they, however, have to pause the game in order to read it in little story segments hidden away in a menu outside the game play. The story in the game proper is barely present. Most of the game play is spent by the player always moving through a linear series of rooms, their progress only temporarily halted by hell demons. The game play is separable by these combat encounters in which a player moves into an area, fights all demons present (or spawning in) and then moves on. The length of these loops have a duration of usually a few minutes, depending on the specific room and the ability of the player to efficiently dispatch the demons.
It is, however, not the only game play loop that could be defined. On a micro level, for example, the game play loop could be defined as the player analyzing a situation, choosing an action, executing the action, and evaluating the result. Another game play loop, usually found several times in a single (longer) encounter room is defined by the weapon usage and following scavenging for ammunition. During a combat encounter, the player is driven to use their weapons arsenal by the specific combat situation (shotgun for close quarters, plasma rifle for mid-range etc.). As ammunition is, however, limited, the player has to switch through their weapons, change their game play style to get in advantageous combat situations for this new weapon, and finally refill their ammunition with either engaging a demon in melee or by searching the combat area for pick-ups.
The choice of game play loop to study is neither easy, nor absolute, nor guaranteed to accurately depict the “true experience” of a specific videogame. Still, when the stimulus is well known and the game play loop chosen well, it should allow for a good compromise to make videogames studiable in an experimental, laboratory context.
Conclusion
I think videogames can and should be studied by empirical aesthetics. However, the multifaceted nature of videogames has to be accounted for and taken into consideration.
References
Slideshow code is mostly taken from https://www.w3schools.com/w3css/w3css_slideshow.asp
[1] Anderson, C.A. et al. 2010. Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta-Analytic Review. Psychological Bulletin. 136, 2 (2010), 151–173. DOI:https://doi.org/10.1037/a0018251.
[2] Ferguson, C.J. and Kilburn, J. 2010. Much Ado About Nothing: The Misestimation and Overinterpretation of Violent Video Game Effects in Eastern and Western Nations: Comment on Anderson et al. (2010). Psychological Bulletin. 136, 2 (2010), 174–178. DOI:https://doi.org/10.1037/a0018566.
[3] Jenkins, H. 2005. Games, the new lively art. http://web.mit.edu/cms/People/henry3/GamesNewLively.html. (2005), 175–189. DOI:https://doi.org/10.1109/MASCOT.2002.1167092.
[4] Przybylski, A.K. et al. 2014. Competence-impeding electronic games and players’ aggressive feelings, thoughts, and behaviors. Journal of Personality and Social Psychology. 106, 3 (2014), 441–457. DOI:https://doi.org/10.1037/a0034820.
[5] Rütgen, M. et al. 2015. Placebo analgesia and its opioidergic regulation suggest that empathy for pain is grounded in self pain. Proceedings of the National Academy of Sciences. 112, 41 (Oct. 2015), E5638–E5646. DOI:https://doi.org/10.1073/pnas.1511269112.
[6] Swink, S. 2009. Game Feel: A Game Designer’s Guide to Virtual Sensation. Elsevier Inc.
[7] Wirth, W. et al. 2007. A process model of the formation of spatial presence experiences. Media Psychology. 9, 3 (May 2007), 493–525. DOI:https://doi.org/10.1080/15213260701283079.
