«

»

Nov 12

Lessons Learned on Scientific Field Work in Esports

Scene: Mandalay Bay Esports Stadium, a 12,000-seat arena in Las Vegas, Nevada. Two young men sip water and wipe their sweaty hands as they wait for a cue that it’s time to perform for the filled stadium seats and the two hundred thousand viewers online. Digital versions of themselves pose in sponsored gear on loop on the supersized screens above them. The cue is given – their faces harden. They don padded headsets to drown out the crowd. Their hands become a blur. The next 8 minutes will determine the Evo World Champion in Super Smash Bros Melee and the winner of $8,000 for a weekend of competitive gaming.

To perform at the level of a world champion, these players must perform at the peak of human cognition: based on knowledge they’ve accrued over a decade of play, they make high-level strategic and predictive gameplay choices by executing sensorimotor action combinations at the level of frames-per-second. Think ultra-high-speed chess with extreme, precise motor demands and you’ve begun to grasp esports.

As a scientist, I’ve made it my goal to understand the cognitive, physiological, and social components that ultimately determine who gets up on that stage. The psychological study of expertise has made clear that both practice and talent support cognitive and sensorimotor skill acquisition and performance. However, to understand the complete picture, we must also look at emotion regulation skills that allow for performing under pressure and the role of identity and stereotypes in granting access to the game at all. My work aims to take each of these components into account to find out what makes a Top 100 player.

Over the summer, I attended four national tournaments to work with 20 of the Top 100 players in the Super Smash Bros Melee competitive scene. My protocol involved pre-tournament surveys, cortisol sampling (a 45-minute commitment each morning for participants), a wrist-based heartrate monitor to be worn for the whole tournament, a 45-minute interview on skill and social support, and a 20-minute cognitive testing session. My plan is to combine the variance from each of these measures to explain both performance in that tournament and overall Top 100 ranking. Many of these measures had to be taken in the field, a setting often neglected in traditional psychology graduate student training. My pilot testing taught me three important lessons about field world which I explore below.

 

  • No matter how organized you are, things will go wrong and you will be stressed.

 

Be prepared to fail and MacGyver your way through travel and data collection.

For data collection, you can prepare for the worst by having access to everything both digitally and hard copy and keeping a checklist of all materials. Important tidbits like laptop chargers and labels tend to take a mental backseat to the primary study materials like questionnaires, but they’re just as important. Compartmentalizing helps too: I had a carryon dedicated to study materials and organized it so that I could look into it and immediately know what I was missing.

But even when you’ve prepared, some things will be out of your control. Different tournament venues provide different testing environments, so I had variation in screen visibility, ambient noise, and subject comfort for every cognitive test and interview. Despite a lengthy verbal explanation and multiple printouts on how to do saliva sampling to measure daily cortisol rhythms, subjects still took their samples at the wrong time or not at all. A heartrate monitor is still MIA from a subject who forgot to return it. Despite my plans, every single aspect of the study was challenged by the environment and the subjects themselves, factors outside of my control.

Even things within your control will go wrong. The stress of travel and data collection will compromise your executive functioning, especially if you’re working solo to save grant money. I had to retest my very first subject because I forgot to unmute the computer when audio cues are integral to the reaction time task – not to mention the multiple, small, silly mistakes I made from fatigue by the end of the month-long data collection marathon. Externalizing everything possible will unburden your working memory and as a result bolster your mental energy. When things go wrong, a master sheet of notes will help explain funky gaps in data and make judgments about which data to discard. Every detail belongs on paper so it won’t weigh you down: who was retested, the order of presentation, the testing environment, who wore what color watch, even what real names match up with gamertags.

In field work, so many factors are outside of your control that you cannot achieve perfection. Prepare the best you can to collect high-quality data, but also be forgiving of yourself of the mistakes and issues that will pop up along the way. My perfectionist tendencies in data collection only compounded my stress. Do maximum prep work so you can fall back on it when things go wrong or you feel overwhelmed.

 

  • Know the unique demands of your population.

 

In an ideal situation, you’ll be doing field research within your own community. Being a community member means you’re familiar with domain-specific skills and measure them better in your work; you’ll have a network from which to recruit, split rooms, and socialize; and you’ll have a gut sense of ethics as far as what is exploitative versus uplifting to the community. But even if you aren’t a member of the community you’re studying, it’s worth it to do your research on the unique demands within it. Otherwise you risk compromising your science or even failing to collect data at all.

For example, knowing the schedules of your participants (and potential participants) is crucial. Perhaps at a military base or boarding school subjects do not have 45 minutes after awakening to complete a saliva sampling procedure. Similarly, I had many subjects wake up 20 minutes before competing, meaning I couldn’t measure their cortisol awakening response. Schedules are important for recruitment too: I don’t approach people who are competing in 10 minutes, and on the flip side, I would be comfortable recruiting someone knowing they were done for the day and just hanging out. (Also, as a rule of thumb, I never asked people to participate right after they lost.)

I also knew walking in that I would have trouble with subjects showing up. Although I had many people do surveys beforehand and commit to times to meet, actually scheduling someone at a tournament is impossible because their schedule will vary with their performance and a thousand other factors, like who wants to hang out right then. Plus Smashers tend to party at national tournaments, meaning lots of oversleeping, hangovers, and lost materials. Communicating with my subjects via text instead of email was a good way to keep up with ever-changing availability. Furthermore, my regular attendance at national tournaments meant more opportunities to fill in cells for each person rather than absolutely needing to complete everything in one weekend.

Finally, sexual assault is a problem in the Smash scene. I as a woman was ready to navigate differently interactions with players who have a history of assaulting women – testing in public places rather than hotel rooms, limiting the information they know about me, that sort of thing. The only way I could know who to take caution with came from being embedded in the community’s network.

 

  • Your social identity intersects and determines your ability to do research and the content and quality of the data you do collect.

 

Most gaming scenes, especially for competitive and fighting games, are majority male. I do not exaggerate when I say that women make up less than 5% of Smash tournament attendees and about 2% of competitors. My lay theory on that percentage difference is that although many of us wish to be respected and feared top players, competitors and commentators have an especially bright spotlight of the male gaze put upon them, making them the most vulnerable to sexism and abuse – so a lot of us engage with the community by organizing tournaments and players, making art, or creating photo and video content. Whatever role we occupy, it’s safe to say that in gaming spaces, women are The Other, not the standard player, and with that minority status comes stereotypes. Unavoidable stereotypes about my gender changed the way I thought, felt, and worked on a daily basis, energy that could have gone to another hour on the ground at a tournament, another dinner out to network, or another reminder text for cortisol samples.

A huge amount of my energy during my summer tournament run was spent on first-impression management. Every morning was a struggle to handle the girl gamer stereotypes I may be slotted into, to balance impressions of my competence with my femininity. When I wore a skirt and makeup, I worried that people labeled me an “egirl” with the accompanying stereotypes of social climbing, promiscuity, and two-facedness. And indeed, in those clothes, I had better success cold-recruiting players and making friends but I was assumed to have no game competence – no one wanted to talk about the game, no one probed my scientific theories, but they were happy to feel out my social network and ask about my plans for the afterparty. If I presented more masculine, a suit jacket and loose shirt, many top players hardly gave me a second glance after I introduced myself, but people listened longer when I talked about my theories and directed conversations in more academic directions. (I always relied on the authority of a suit jacket in formal research presentations at events.) In the end, no outfit could reconcile the competing identities of scientist, dry and intelligent and unsexual and masculine thus competent, with a gamer girl, comfortable in her body and approachable and granted access to important figures of the community. My attempts to dress and act in between these two extremes only led to assumed incompetence of both social and research domains in my first meetings with people.

Even beyond first impressions, I spent so much energy navigating conversations to establish who I am, to research subjects and community members alike. Sometimes it was more useful to talk about my Smash commentary first, to establish my in-group gamer status, and in subsequent conversations bring up my research. Other times, people were clearly engaged in the science but assumed I knew nothing about the game, an assumption that had to be corrected over multiple interactions filled with jokes and references to in-depth game knowledge or my commentary gigs before they would even talk to me in an interview on a natural (rather than dumbed-down) level.

Besides my appearance, I managed my social impression in other ways. Players take a video game sexism survey as part of my initial questionnaires and I have no illusions that my status as a female researcher and gamer influence their likelihood of reporting their true thoughts. I do what I can to manage that: I recruited online rather than face-to-face as much as possible. And although I have never concealed my gender, the ambiguity of being a woman with a man’s name has afforded me much within the Smash scene.

If it isn’t obvious already, I’ll explicitly state it: every worry I’m expressing is directly linked to the integrity of my research. When I ask them about practice habits, I need them to speak to me using the highly domain-specific knowledge that may set them apart from one another – every Top 100 player is going to say they practice a certain number of hours, but how many use a CPU 20XX Fox set to approach for their chain-grab practice versus studying frame data on a computer before picking up the controller? When I ask them in surveys and in person about the role of women in the Smash community, I want them to be as comfortable as possible telling me the truth rather than sugar-coating their answers because they want to seem progressive to a scientist or because they want to date me. To even access members of a special population like the Top 100 requires networking and trust-building, all dependent on the above impressions. In the end, the identities that subjects assign to me are inextricable from the answers they give me and from accessing their expertise. Thus, it’s not something I can ignore.

In the end, my identity as a woman, scientist, and gamer all intersect to create a unique set of circumstances that, in the field and in the data, affect my research profoundly. I have a unique, diverse, and valuable perspective compared to the average gamer and games researcher, one reason I continue on despite challenges. Arguably, I have better access to top-level players to study. In other ways, I am permanently an out-group member who may never learn the truth of gender-gaming stereotypes or high-level game-specific knowledge on my own. The lesson here is that every scientist in search of truth, in the lab or otherwise, needs to remember how their identity is affecting their access to and interpretation of truth in specific ways, every step of the way.

 

Field work requires a balance of priorities: the integrity of the research, reducing noise that comes from uncontrollable circumstances, and your wellbeing as a researcher. Hopefully, these lessons I’ve learned are valuable to other researchers hoping to dive into field work themselves.

 

 

Related readings:

  • Taylor, T.L. (2012). Raising the stakes: E-sports and the professionalization of computer gaming. Cambridge,
    MA: MIT Press.
  • Taylor, N. (2018). I’d rather be a cyborg than a gamerbro: How masculinity mediates research on digital play. MedieKultur: Journal of media and communication research34(64), 21.
  • Consalvo, M. (2012). Confronting toxic gamer culture: A challenge for feminist game studies scholars. Ada: A Journal of Gender, New Media, and Technology, (1).

1 comment

  1. Kevin Schmidt

    Nice write-up. Thanks for posting.

Leave a Reply