Pie Charts! - A quantitative analysis of the prototype playtest survey results

Since I posted the playable prototype and survey to the blog a couple of weeks ago (and shamelessly plugged it on facebook), I've gotten over 100 views on the page, and 17 people filled out the survey, which is pretty awesome! Thanks, everyone!

Now that I have a sizable amount of data, I'd like to analyze it and see what worked well, what needs improvement, and what is just plain broken. So, this is the first post in a 3-part series analyzing all the data I got out of the surveys. Today, I'd like to analyze the most tangible, quantitative, low-hanging data: the multiple choice answers about the gameplay experience.

Completion rate

Google Docs provides handy charts for the multiple choice answers

88% of respondents finished the game, and 24% finished it more than once! Yay! It's particularly encouraging that a significant number of people were intrigued enough by the game to try it again (and presumably try out different ways of going through it).

Swords

75% of the players found the sword, which to me is a good percentage: the sword is not strictly necessary to complete the game, but is visible (and desirable) enough that many players do get it.

The fact that only one person couldn't tell whether they ever got to use the sword is also encouraging: there is enough context provided about the sword and the situation it's used in (i.e. combat) that people can generally tell if it actually came in useful. I was worried that the game might be too confusing and chaotic for people to be able to tell what the sword is supposed to be for, especially if you never actually see it used in combat.

However, about 1/3 of the people who did get the sword never actually got to use it, which is a shame. Ideally, this mission would be longer and give people more chances to whack things with a sword.

Slimes

About half of the players ended up fighting the slimes, which makes sense, since people playing a clone of pacman would probably try to avoid the clones of the pacman ghosts. I'm pretty happy with the 50/50 breakdown; if this mission were longer, or if there were several missions where the pacman/ghost fight bit recurred, I'm guessing most players would eventually (accidentally or out of curiosity) bump into a slime and find that you can fight them.

Jail

Going to jail (perhaps "prison" might have been a more appropriate word?) after being defeated in a fight, then getting out of jail to continue your adventure, seems to be the default way of playing the game. 

On the other hand, 6 people finished the game without losing a single slapfight! And yes, I do know that they all finished the game, since all of the respondents who gave up part of the way through did go to jail at one point. 

Actually, this brings us to the most interesting part: collating different answers and looking for correlations and dependencies. For example:

• All of the people who never finished the game never found the sword, never fought a slime, but did go to jail. 
• All of the people who never went to jail played through the game exactly once
• Which means that 4/9 of the people who did go to jail, and did nevertheless finish the game, played the game more than once.
• 5/6 of the people who never went to jail did find the sword, but only 2 actually got to use it. 
• 3/4 of the people who found the sword but never used it also never went to jail.
• 2/3 of the people who went to jail but never found a sword did not finish the game.
• 2/8 of the people who never got to use a sword fought a slime monster; 3/8 of the people who did get to use a sword did not fight a slime monster.
• If you never went to jail, and you never fought a slime monster, (and you did respond to the survey) then you did find the sword! None of the respondents avoided *all* of the optional stuff.
• Going to jail does not seem to affect your chances of fighting a slime.

Drawing conclusions from these results can be tricky without further data. For example, while I can be pretty certain that going to jail without getting a sword is a bit of a bummer and is liable to make people quit, I can't really tell whether going to jail overall makes people more likely to try the game again, or trying the game several times makes people more likely to go to jail one of those times. 

Tune in next time for an analysis of the paragraph answers: what people liked, disliked, and were confused about!

Playable prototype

Our first truly playable prototype! We did have another prototype a while back, but it was very difficult to understand what's going on unless you already knew or you were listening to me narrate as I play.
This version shows what a short mission might look like in Micro Missions. Except that there would be a more diverse set of minigames: more travel games, more fighting games, more outcome possibilities and more paths from different outcomes, etc.
This prototype was built using the actual mini-game engine we are writing for the game, although I did have to fake some functionality that's not there yet.

After you play the game, please fill out this survey to let me know what you thought:

Architecture example

This is an example of a (pretty silly) mission, to illustrate our architecture.
First, an overview of the main parts of the architecture

• Locations in the overworld are connected by paths
• Any change to the overworld or player action happens through interactions, most of which are mini-games
• Interactions may have:
• A number of inputs
• A number of side-effects
• A primary discrete output
• Some secondary discrete outputs
• Each location specifies
• What interaction plays when you enter that location
• What interaction will play for each possible output of the first interaction
• Etc. until you leave the location through a location-changing interaction

In our example, the interaction present are:

• Select and change location
• Input: none(chosen during interaction)
• Side effect: Location Change
• Output: none
• Flip a coin
• Input:None
• Side Effect: None
• Output: Heads/Tails
• Make a Choice
• Input: 1-4 things to choose from
• Side Effect: None
• Output: The thing that was chosen
• Toggle Path
• Input: the path to toggle
• Side effect: Path is turned on/off
• Output: none

Using these three simple interactions, we can create a game like this:

To win this game, you'd have to go to the topmost location and flip coins until you get heads to turn on path 5; then go to the bottommost location, and make the choice to toggle the path. Then you can get to the Goal!

Art!

Here is some art that we've made so far for the game. We narrowed down the art style to really simple shapes and colors, made out of polygonal primitives for non-organic objects and ellipses and arcs for organic objects, with a thick outline. We're going for a clean, simple, toy-like feel for the art overall.

Here is some general art:


This is a sword-sharpening mini-game, made using the mini-game library we made for this project(the idea is to hold the up/down button until you reach the end of the sword, then switch):

"Dynamic" story?

"If we knew what we were doing, it wouldn't be called 'research', would it?"

-commonly attributed to Einstein

"[A story] is nothing more than a sequence of events that someone relates to someone else."

-Jesse Schell, The Art of Game Design

It's pretty easy to define what a "dynamic story" is not. It's not a static, pre-determined, pre-written, pre-recorded, unchangeable story.

It becomes harder to define and agree on what a "dynamic story" is, should be, or can be. I think that's why there also tends to be a lack of agreement on whether such a thing is possible and/or desirable. Does generating "dynamic stories" imply giving up authorship and leaving the story to chance, the computer, and (maybe, sometimes) the player? Can "dynamic story" mean defining a game world and rules, letting the player loose, and assuming the player will create his own stories out of the events that transpire as a result of the rules? If a computer generates a story, is it already dynamic by virtue of not being pre-written before the game shipped?


For me, in the scope of this project, "dynamic story" means the following (in dimishing order of importance):

• The story is told by the game to the player

I have encountered, several times, the point of view that all games are "storytelling machines" in that they enable players to later tell stories about events that transpired during the game ("we were almost losing but then I stole the ball and made the goal right before the game ended!"). However, I see a huge difference between storytelling (the game tells a story to the player) and story enabling (the player tells stories about the game). I'm interested in the first one. As the player, I want to be told a story.

• The player has agency in relation to the story and its progression

Specifically, the player has agency over how his character acts within the context of the story, and the character's actions are acknowledged by the story and world in a sensible manner, much like a Game Master in a pen and paper RPG must constantly adjust to the players' actions. Ideally, the player has a constrained but flexible vocabulary of actions that enable him to react to the events of the story the way he sees fit. The story, in turn, can react reasonably and believably to the player's reaction.

• The story is engaging and interesting

'Cause boring stories are boring.

• The story mimics the typical RPG story style and storytelling conventions.

The familiar story structure would give the player comfortable, easy to understand stories. This could be especially important in such an abstract, strange setting as a story-based microgames game. On the other hand, it also provides a large, easy to understand and abstract basis for a story system. Thus, to answer the question of "how will a computer tell a story, and how will it decide what story to tell?" we can first answer, "how do computer games usually tell stories, and what stories do they tell?"

These constraints are guided by introspection: what is it that I find so appealing about a dynamic story system? What do I really want to achieve or fix with such a system?

Primarily, it is the frustration of running into what I would call the "invisible walls" of on-rails storytelling: "But I don't wanna walk into the trap!" "This escape plan is stupid, I have a better one" "I want to disagree with that guy but all the dialog options agree with him!" "Can't we just kill him or take the doohickey by force?" "I'm pretty sure I could have talked my way out of this" "What am I supposed to do to make the auxilary character finish his 'research' and progress the story?" "Oh, so this guy dies whether or not I choose to kill him?" and so on. 

It would be impossible to anticipate and address all the ways a player's idea of what his character should do next will disagree with the prescribed story, let alone create viable alternatives to the story for all of them! The usual solution is to not let the player make those choices. But is there a way to instead react to those choices as they happen?

My secondary motivation is a fascination with game worlds as models. I personally feel the most engaged and connected to a game world when that world is an understandable, internally consistent model. On the other hand, I often find myself get bored of games quickly if they don't engage me with their story, even if they have good gameplay. Why not try to create a model of an engaging story?

Architecture: Overworld and Minigames

Here is an outline of the architecture we are planning to use for the overworld, mini-games, and how they are connected. This architecture is the "non-experimental" (or at least less experimental) part of the project - the basis of the game that we want to make. The more experimental part, which will be the focus of my research, is how to use this architecture to enable the game to tell dynamic, non-predetermined stories.

• The overwolrd consists of locations which are connected by paths.
• Each location can have a number of state variables and a number of mini-games associated with it
• Mini-games can take input from the world, including information about the current location and/or character state, and configuration variables such as the type of "skin" the game should use (if applicable), or an item that the player should have a chance of acquiring as part of the game.
• Each mini-game provides one main discrete output (with a small number of possible values, e.g. yes/no or left/right/center) and may also provide a number of secondary outputs, also discrete, such as whether the player got the item passed in as input.
• When a mini-game completes, the current location makes changes to the world based on that game's output.
• All changes to the overwolrd happen through mini-games (or pseudo mini-games - "Interactions" that follow the same input/output structure as mini-games). This includes changes to the character; traveling between locations connected by a path is a core type of mini-game.
• Some changes in world/story state can be communicated to the player through short cutscenes, which would essentially be non-interactive mini-games.

The story, setting, characters, etc. of a particular mission would then be defined by: placing and connecting locations; defining the mini-games available at each location; for each mini-game, defining: the conditions for its availability, its inputs, and the effects each possible output would have on the world/location/player character.

This architecture provides for a very simplified structure that is still flexible enough to allow for almost any mission, quest, story, or general setting to be encoded. The simplified nature of the architecture, especially the discrete outputs of the mini-games, should also make analysis of potential missions easier. In fact, the idea is to make it possible to traverse the tree of all possible chains of events as part of some form of automatic analysis.

What is Micro Missions (again)?

Since I first posted the definition and scope of our game, the design has gone through some changes. The major changes were de-emphasizing the social and multiplayer aspect (to the point that these are no longer in the scope of the capstone project) and focusing instead on the single-player experience and the dynamic missions/stories. Rather than edit the original definition, I though I would post the new version separately. That way, this blog will better represent the changes the game goes through as it's being developed.

A single-player casual game that combines frantic, silly mini-games into story-based missions. The stories are dynamic in nature, and are generated and modified in response to both player actions and the current world state.

In most mini-game collections, the individual games are either pass/fail or score-based. The mini-games in Micro Missions instead have a qualitative effect on the overarching world and story. For example, some mini games may allow you to travel in the world, while others are fighting mini-games to resolve conflicts with enemies, and others represent side-quests of a sort. At the same time, these games also take input from the overworld - where you are at the moment, whether you have encountered an enemy, what actions are available in the current location and story context, etc.

These mini-games are tied together into missions, and each mission outlines a story through the sequence of mini-games and short informational cut-scenes. Particular missions are based on encoded templates that define what the goal is, what the different roles are, and what must happen along the way. For example, the Hero’s Journey can be encoded into a very flexible and scalable template for missions (but not all missions have to be based on the Hero’s Journey).
Missions are presented to the player as a sequence of quests. Quests are in turn a sequence(or several possible sequences) of actions - mini-games in various locations that the player needs to play and achieve a certain result in. When the player completes a quest, the next quest is selected based on player actions, inferred or indicated player preference, and most importantly story progression.

Selecting mission templates, populating the missions with particular quests and mini-games, and reacting dynamically to player actions will be the main source of “intelligence” in the system, and the major research topic in this project.
Missions within the same mini-game world are further tied together through recurring characters and other persistent effects, for example passages that have been closed off or opened up in previous missions.

We are using Flash and ActionScript 3 to build this game for several reasons. Flash/AS3 is a great rapid development tool, allowing us to worry less about lower-level issues and concentrate on design and architecture of the game itself. ActionScript 3 is an object-oriented language with functional elements, which should work very well for the architecture(s) we have in mind. Finally, Flash makes the game really easy to distribute, and leaves the potential for social network integration if we want to return to it in the future.


A note on the definition of the term "mini-game" as it applies to this game: I imagine our mini-games to be somewhere between Warrioware-style microgames and traditional "mini-games" in terms of scale and length of play, perhaps closer to Warrioware. For the pace of the game that I have in mind, I'm imagining games that range in play length from several seconds to a minute or two at most.


The definitions of the mini-game/overworld architecture and dynamic story system are intentionally left vague in this overview; they will be the subject of future posts. The overworld architecture is fairly well-defined at this point. The story/quest architecture is what I am working on now.