Since I posted the playable prototype and survey to the blog a couple of weeks ago (and shamelessly plugged it on facebook), I've gotten over 100 views on the page, and 17 people filled out the survey, which is pretty awesome! Thanks, everyone!
Now that I have a sizable amount of data, I'd like to analyze it and see what worked well, what needs improvement, and what is just plain broken. So, this is the first post in a 3-part series analyzing all the data I got out of the surveys. Today, I'd like to analyze the most tangible, quantitative, low-hanging data: the multiple choice answers about the gameplay experience.
Completion rate
 |
| Google Docs provides handy charts for the multiple choice answers |
88% of respondents finished the game, and 24% finished it more than once! Yay! It's particularly encouraging that a significant number of people were intrigued enough by the game to try it again (and presumably try out different ways of going through it).
Swords
75% of the players found the sword, which to me is a good percentage: the sword is not strictly necessary to complete the game, but is visible (and desirable) enough that many players do get it.
The fact that only one person couldn't tell whether they ever got to use the sword is also encouraging: there is enough context provided about the sword and the situation it's used in (i.e. combat) that people can generally tell if it actually came in useful. I was worried that the game might be too confusing and chaotic for people to be able to tell what the sword is supposed to be for, especially if you never actually see it used in combat.
However, about 1/3 of the people who did get the sword never actually got to use it, which is a shame. Ideally, this mission would be longer and give people more chances to whack things with a sword.
Slimes
About half of the players ended up fighting the slimes, which makes sense, since people playing a clone of pacman would probably try to avoid the clones of the pacman ghosts. I'm pretty happy with the 50/50 breakdown; if this mission were longer, or if there were several missions where the pacman/ghost fight bit recurred, I'm guessing most players would eventually (accidentally or out of curiosity) bump into a slime and find that you can fight them.
Jail
Going to jail (perhaps "prison" might have been a more appropriate word?) after being defeated in a fight, then getting out of jail to continue your adventure, seems to be the default way of playing the game.
On the other hand, 6 people finished the game without losing a single slapfight! And yes, I do know that they all finished the game, since all of the respondents who gave up part of the way through did go to jail at one point.
Actually, this brings us to the most interesting part: collating different answers and looking for correlations and dependencies. For example:
- All of the people who never finished the game never found the sword, never fought a slime, but did go to jail.
- All of the people who never went to jail played through the game exactly once
- Which means that 4/9 of the people who did go to jail, and did nevertheless finish the game, played the game more than once.
- 5/6 of the people who never went to jail did find the sword, but only 2 actually got to use it.
- 3/4 of the people who found the sword but never used it also never went to jail.
- 2/3 of the people who went to jail but never found a sword did not finish the game.
- 2/8 of the people who never got to use a sword fought a slime monster; 3/8 of the people who did get to use a sword did not fight a slime monster.
- If you never went to jail, and you never fought a slime monster, (and you did respond to the survey) then you did find the sword! None of the respondents avoided *all* of the optional stuff.
- Going to jail does not seem to affect your chances of fighting a slime.
Drawing conclusions from these results can be tricky without further data. For example, while I can be pretty certain that going to jail without getting a sword is a bit of a bummer and is liable to make people quit, I can't really tell whether going to jail overall makes people more likely to try the game again, or trying the game several times makes people more likely to go to jail one of those times.
Tune in next time for an analysis of the paragraph answers: what people liked, disliked, and were confused about!
No comments:
Post a Comment