Update: As you are reading this please keep in mind that this was written in response to some very specific situations we have experienced during playtests. As was pointed out in the comments of the original post (thank you Josh), if this article is applied to playtesting as an entire process, it could come across as a gross oversimplification of the marathon playtesting can become.
Please give me a moment to set the table.
Finding a way to get your game design to work properly can be really complicated. People are really complicated.
When you get feedback about your game, you sometimes need to ask yourself if your game has a design problem or a player problem.
Now….before the angry mobs bring pitchforks and run me over, I am not saying the people are wrong. What I am saying is, they might be the wrong playtesters.
Update: Just to clarify, we took a game to a convention and the first group to sit down were what I would have called wargamers. Between a couple bad moods (it was early in the day) and one tester not fully understanding the combat mechanic, the feedback was less than stellar. It is possible we explained the game poorly. Especially since this was the first time we had taken a game to a convention. If we had thought this particular set of testers were right, we could have been discouraged and thought the game was broken. We ended up testing the game with 20+ more people over the course of the day and everyone really enjoyed it. Now, could they have all be wrong? Yes, but that is not what I want you to take from this. What I really want you to take away from this is to not be overwhelmed by a string of bad feedback. Just sit and think about the feedback before you react and make changes to your game.
That might sound like I am telling you to keep playtesting until you find enough people to say a broken game is good and you are done. What I am saying is that people come into games with their own biases about the games they like and dislike. This will affect your game and there are times when you need to recognize this and ask yourself if the problem is with the game or the people testing the game?
If you receive feedback saying a part of your game is broken and all the testing you did on your own says it isn’t, don’t throw out the mechanic just because 10 people in a row said it doesn’t work. Sit down with your game and think about your own conclusions. Share the feedback you have from testers with other designers and see what they say. It is possible that the mechanic needs to be changed, but there are times when your game is fine.
Update: As an example, we like to apply a 1% rule to our games. If we find a bad situation that has a 1% chance of happening in the game, we need to fix it. We want to avoid unfun situations that could take players out of the game. A 1% rule applied to playtesting means there needs to be a lot of playtesting to find these situations.
The reason the game can be ok and the playtesters are still right is because people bring their own point of view to a game. Playtesters have their own ways of looking at a particular part of a game. They have their own history and their own reasons for what makes them want to sit down to a game and play. One of the jobs a designer has is to know this and take this into account. Your game might be fine, you just need to know why it is ok and continue testing.
Update: Just wanted to say a big thank you to Josh for taking the time to leave his comments in the original post. They helped me see what was missing and add these updates. Hopefully, these updates give a little more context to the inspiration of the post and and clarify what I want designers to take away.
If you have any comments or questions, leave a comment here or email Chris at firstname.lastname@example.org
If you have made it this far, would you like to go a little farther? We have a regular Google hangout with other designers. We talk about the games we are working on and share helpful tips and ideas on how to make designing our game easier. We meetup every other Saturday. Either comment here or tweet me or email me and I will add you to the list and send you a link to the Google hangout.