Friday, July 15, 2011

On judgement

This post has been in the back of my mind for a very long time, but has recently been promoted to the front by reading reviews of The Witcher 2. I guess I should make it now, before a proximity with a competition I intend to enter may make it seem less disinterested than it, of course, is.

Here is a typical discussion you'll find on the internet about a review of a game.
A: This review sucks, because X.
B: A review is just an opinion, man! It can't be wrong or right.
C: No, a review is not just an opinion.
D, E, F, G and H: You're really stupid!
But C isn't stupid. C is right. If you write a review, you are not just giving an opinion. You are giving a judgement, and a judgement is something that aims to be reasonable and objective. There are standards to which a judgement must conform; and a judgement can fail to conform to those standards and be wrong.

It is important to understand that this reasonableness and objectivity have little to do with the final verdict expressed in the judgement. Reasonable judges can give the same game a 10 or a 3. A judgement is reasonable and objective when it is based on reason; when the judge can give an analysis of the game and explain his verdict using criteria that other people can understand and sympathise with. There may be disagreements about the exact application of those criteria, but there is in general agreement about the criteria themselves.

Why was this topic in the back of my mind for a long time? Because every IF competition has a couple of judges who flagrantly ignore the basics of what it means to be a judge. These are the people who write reviews that go: "After playing for five minutes, I really didn't like the game and I couldn't be bothered to play any further. So that is a solid 1." That is not a judgement about the quality of the game. That is an expression of personal feeling. There is nothing wrong with approaching IF with the idea that you will stop playing any game that doesn't make you enthusiastic in five minutes. Fine. But something is very wrong with having this attitude and then being a judge in a competition. When a competition is looking for judges, it is looking for judges, for people who are willing to invest enough time and thought into the games that they can form a reasonable judgement about its quality; it is not looking for people who believe that their immediate emotional reaction to something is valid and worthy of being communicated to the rest of mankind. (This may be hard to grasp in the era of Facebook and Twitter, but really, it is true!) If that is the way you approach a competition, you are just as wrong as a legal judge who doesn't weigh the evidence but bases his verdict on personal like or dislike of the accused -- the only difference being that the consequences of your actions are less dire.

So, yes, reviews can be reasonable or unreasonable; they are not just opinions; and judging a competition is serious business. Before you give a bad mark to a game because you "didn't like it", ask yourself whether you have seen enough of the game to come to a reasonable judgement. Otherwise, a little bunny cries. Really.

4 comments:

  1. I agree wholeheartedly. But then, I would, wouldn't I?

    Speaking of which, I'm still anxiously awaiting a review of Mentula Macanus: Apocolocyntosis.

    ReplyDelete
  2. Because every IF competition has a couple of judges who flagrantly ignore the basics of what it means to be a judge. These are the people who write reviews that go: "After playing for five minutes, I really didn't like the game and I couldn't be bothered to play any further. So that is a solid 1." That is not a judgement about the quality of the game. That is an expression of personal feeling.

    If you're talking about Introcomp reviews, then I have to quibble a bit. The judging rules specifically, explicitly say this:

    However, they are asked to judge games with one thought in mind, and one alone: "How much do I want to play more of this entry?"

    Which is, inevitably, a subjective question as well as an objective one.

    (Separately, I would argue that it is not possible to be completely objective in judging any piece of art. But that's an argument for another day.)

    ReplyDelete
  3. I have to disagree... well, maybe. I think that the *experience* you relate ("I played it for five minutes; it sucked; a 1") is a valid judging experience for some games. I don't think it would take me longer than 5 minutes to accurately assess the quality of, say, the original Stiffy Makane, or something like The Fat Lardo and the Rubber Ducky. However, I will agree that the review itself has to say something other than "I just didn't like it" and tell why the author didn't feel like playing it any longer.

    ReplyDelete
  4. Adam, I hope that I may be able to do a SPAG specifics about it -- but I'm not promising it for the next issue. :)

    Emily, this is probably just bad timing on my part. I was not thinking about the Introcomp at all; and I agree with your comment about it. But of course, the Introcomp must be special: there are no finished works of arts to judge.

    I doubt we disagree much about subjectivity and objectivity. Judging a work of art is not "as objective" as finding the right answer to an arithmetic problem, of course; and not just because there is no one right answer, but also because a work of art asks us to participate in it with our whole personality. (Or something like that.) But there are nevertheless more and less objective ways to judge art.

    Cendare: sure, it is sometimes possible to be justified in judging a game after playing it for five minutes. (Perhaps my point would be clearer if I pointed to some of the reviews that piqued me, but I'm not writing this post in order to put specific persons in the pillory.)

    ReplyDelete