Slapfights, foot cars, and confusion - an analysis of the long-form survey answers

In the first installment of the analysis mini-series of the results of the playable prototype survey, I looked at the multiple-choice answers and tried to draw conclusions about what the players did, and whether that was what I wanted them to do. In part two, I will examine the written answers to the questions about what the players liked, disliked, and were confused about.

Good news:
None of the respondents seemed completely and utterly confused by the game and what is going on in it; In fact, a few of them said they weren't confused at all! This is actually a big deal: before I made a few minor modifications to this same prototype, people had a lot of trouble understanding what's going on and what they are supposed to do next. I think the most important modification I made was changing the duration of each cutscene and instruction frame from 1 second to 2 seconds. This allows people to take in everything that's being thrown at them and it still maintains a frantic, chaotic pace (which is one of my goals for the game). I also added more cutscenes, instructions, and clarifications before and after mini-games, which seems to have helped a lot, but there is still room for improvement on that front (more on that later)

More good news:
A lot of people liked a lot of the things I was hoping they would like! The crowd favorite was the slapfighting, with 7 people mentioning it as a favorite. The footprint-following mini-game was a close second, with 5 votes. This is really cool to me, because these two games are quite different: a really simple but funny game with really straightforward outcomes, and a game that offers strange new gameplay and outcomes that may or may not be desirable based on the context of the game (e.g. are you trying to chase someone or trail them?). And yet they both worked really well in this setting!

People also enjoyed the pac-man and searching for exit mini-games, the persistent items as exemplified by the sword you could find and use, and the whole "story progression and overworld made out of discrete minigames" deal.

The not so good:
There was a lot less agreement and a lot more variety in what people people found confusing or not enjoyable. One of the common sources of confusion seems to be the goal of the game, where one is supposed to go, and what is going on in general. This is partially by design - this is, after all, supposed to be a hectic, "controlled chaos"-style game with a fairly abstract storyline. But I feel like the game does still need more cutscenes, instructions, and cues in place so that it remains fun. I think that we need to have cutscenes at the beginning and end of each mini-game to create a frame of reference about what you just did and how it affected the world and story. We also need more clear instructions for each game, and more moment-to-moment feedback, like progress bars in mini-games and a "next goal" indicator on the overworld map.

The other big thing people were confused about or displeased with was the foot-car mini-game. It has two major problems, both of which I was already aware of, but did not have the time to fix. The first problem is that unlike all the other mini-games, there is not instruction screen before the foot-car game that prepares you and tells you what you're supposed to do. Instead, at the last moment, I put an animation of alternating right and left keys right into the game. (The way that I've hacked together the sequence of events in the demo prevented me from easily putting in a proper instruction screen)This actually helped a lot (before that, players had no clue at all what they were supposed to do there!), but there's still no consistency or time for players to get ready and find the right keys. For the final game, we'll make sure to add instructions to the mini-games themselves so that all of the games will have their own instructions before the game starts.

The second problem with that game is that there are no consequences to the game whatsoever. There is nothing the game accomplishes, and you move to the map location you were traveling to regardless of whether you actually get to the finish line. One of the players called it an "interactive loading screen", and that's pretty accurate. My plan for how travel is supposed to work is a little more elaborate than just making people press left and right for five seconds, though: if the player does not get to the finish line in time, he will have to make camp for the day in the middle of the road. This means playing another mini-game. The "camp" mini-games would include things like sword-sharpening, reading books, and inventory tetris. Playing these games should give the player some kind of bonus, so even though he took more time to get to his destination, he got something else out of it. I'd also like to add other events to the traveling mini-games itself, like being waylaid by enemies or finding random items in your path.

Surprisingly, only two people caught on to (or at least cared about) the fact that every time you fight a slime in the dungeon, you get placed in a new random dungeon afterwards instead of going back to the spot you were before. This was another hack to speed up the development of the prototype, since I didn't want to implement the functionality to pause an existing mini-game (the maze) and start a new one on top of it (the fight), so I just got rid of the old maze completely whenever you had to do something new like fight a slime. Sorry! We are putting that functionality in the "real" architecture.

So overall, I'm pretty happy with the feedback I got from the game. I feel like the proof-of-concept stage was pretty successful, and people are understanding and enjoying this weird "mini-game story" format. I got some valuable feedback about what works, what doesn't, and the kinds of things I need to be careful about to keep the game from getting too confusing.
In the next post, I will look at the story aspect of the prototype. How well this format works for storytelling, how people interpreted the story, and what kinds of things help or hinder the player's understanding of the narrative.

Research Hypothesis

As part of the capstone process, each graduate student (which for this project is just me!) has to have a research topic that is associated with the game. I've already talked a little bit about my general topic: dynamically generated stories using the Micro Missions architecture. Now, I think, the time has come to define more clearly my research hypothesis - what am I attempting to prove or disprove with my research?
So, without further ado, my research hypothesis is:

With a base architecture of modular, independent mini-games connected into cause-and-effect chains through discrete output, we can generate engaging and understandable semi-abstract stories using a "just-in-time" story generator which will choose and populate the next piece of content on-demand based on current story context and a basic model of narrative structure.

And here is what it means in the context of Micro Missions:

Most video game stories rely on the story designer anticipating and authoring every choice a player can make with respect to the story, and the consequences of these choices. With this method, the quality of the interactive aspect of the story depends in large part on whether the author anticipated a particular player's desired story-related choices. This method also places a great burden on the story designers, since they must author and test all the possible combination of player actions and story consequences.

The Micro Missions architecture allows a designer to create stories out of mini-games by connecting these mini-games through their inputs and outputs. Mini-games can take in a variety of variables that define the current context and environment; gameplay is adjusted using these variables, and after a mini-game ends, a series of values is output. Each of these values is in a discrete, pre-defined range. For each possible output value of each output, the designer of the story specifies what the next Interaction (mini-game, cutscene, or special event) should be.

The prototype developed at the end of Winter quarter served as a proof-of-concept to demonstrate that the described architecture can be used to hand-author stories that are understandable and fun to play through. The research question is whether we can build a story generator on top of that architecture that would generate stories which are comparably understandable and at least as engaging as the hand-created stories.

Pie Charts! - A quantitative analysis of the prototype playtest survey results

Since I posted the playable prototype and survey to the blog a couple of weeks ago (and shamelessly plugged it on facebook), I've gotten over 100 views on the page, and 17 people filled out the survey, which is pretty awesome! Thanks, everyone!

Now that I have a sizable amount of data, I'd like to analyze it and see what worked well, what needs improvement, and what is just plain broken. So, this is the first post in a 3-part series analyzing all the data I got out of the surveys. Today, I'd like to analyze the most tangible, quantitative, low-hanging data: the multiple choice answers about the gameplay experience.

Completion rate

Google Docs provides handy charts for the multiple choice answers

88% of respondents finished the game, and 24% finished it more than once! Yay! It's particularly encouraging that a significant number of people were intrigued enough by the game to try it again (and presumably try out different ways of going through it).

Swords

75% of the players found the sword, which to me is a good percentage: the sword is not strictly necessary to complete the game, but is visible (and desirable) enough that many players do get it.

The fact that only one person couldn't tell whether they ever got to use the sword is also encouraging: there is enough context provided about the sword and the situation it's used in (i.e. combat) that people can generally tell if it actually came in useful. I was worried that the game might be too confusing and chaotic for people to be able to tell what the sword is supposed to be for, especially if you never actually see it used in combat.

However, about 1/3 of the people who did get the sword never actually got to use it, which is a shame. Ideally, this mission would be longer and give people more chances to whack things with a sword.

Slimes

About half of the players ended up fighting the slimes, which makes sense, since people playing a clone of pacman would probably try to avoid the clones of the pacman ghosts. I'm pretty happy with the 50/50 breakdown; if this mission were longer, or if there were several missions where the pacman/ghost fight bit recurred, I'm guessing most players would eventually (accidentally or out of curiosity) bump into a slime and find that you can fight them.

Jail

Going to jail (perhaps "prison" might have been a more appropriate word?) after being defeated in a fight, then getting out of jail to continue your adventure, seems to be the default way of playing the game. 

On the other hand, 6 people finished the game without losing a single slapfight! And yes, I do know that they all finished the game, since all of the respondents who gave up part of the way through did go to jail at one point. 

Actually, this brings us to the most interesting part: collating different answers and looking for correlations and dependencies. For example:

• All of the people who never finished the game never found the sword, never fought a slime, but did go to jail. 
• All of the people who never went to jail played through the game exactly once
• Which means that 4/9 of the people who did go to jail, and did nevertheless finish the game, played the game more than once.
• 5/6 of the people who never went to jail did find the sword, but only 2 actually got to use it. 
• 3/4 of the people who found the sword but never used it also never went to jail.
• 2/3 of the people who went to jail but never found a sword did not finish the game.
• 2/8 of the people who never got to use a sword fought a slime monster; 3/8 of the people who did get to use a sword did not fight a slime monster.
• If you never went to jail, and you never fought a slime monster, (and you did respond to the survey) then you did find the sword! None of the respondents avoided *all* of the optional stuff.
• Going to jail does not seem to affect your chances of fighting a slime.

Drawing conclusions from these results can be tricky without further data. For example, while I can be pretty certain that going to jail without getting a sword is a bit of a bummer and is liable to make people quit, I can't really tell whether going to jail overall makes people more likely to try the game again, or trying the game several times makes people more likely to go to jail one of those times. 

Tune in next time for an analysis of the paragraph answers: what people liked, disliked, and were confused about!