A couple of months ago I described my experience as a juror for the ID Magazine Annual Review. For the last couple of days I had the terrific opportunity to serve on the jury of the IDSA/Businessweek IDEAs (International Design Excellence Awards).
While I am not able to discuss particulars of the entrants or awardees, I would like to share my observations on the experience. In particular, I'd like to discuss my realization that the social aspects of judging designs is analogous to the social aspects of the design process itself.
What do I mean by social aspects? Consider the various interpersonal interactions that occur during the design process - collaboration during concept generation, peer critique during refinement, and eventually, client feedback. The value and outcomes of each of these situations greatly depend on the personalities and interpersonal dynamics of those involved. This is a natural part of the design process, as it should be of most successful human endeavors.
Collaboration and iterative review were very much a part of this year's IDEA judging. (For more information on the judging process, listen to jury chair Alistair Hamilton's podcast on the 2008 IDEAs.) Much like the start of concept generation, the review process began with each of the 20 jurors working independently. During the first phase of judging, each juror independently reviewed all of the submitted documentation (forms, images, videos, etc) entries within a a set of assigned subcategories.
But this quickly became a collaborative effort as subcategories were assigned to two jurors. Following their independent reviews, pairs would discuss their recommendations with each other. When there was disagreement about particular entries, a juror had to convince his or her partner about whether or not to advance an entry to the finalist round. Although jurors were not the creators of the designs (jurors were recused from entries that they had involvement with) this back and forth reminded me of the discussion that goes on when designers meet to discuss the relative merits of their concepts or sketches.
In the next round, each judging pair reviewed their selected finalists in person. This year's IDEA awards judging was improved over previous year's, because for the first time (!), the jurors had access to most of the actual products for the design finalists. Consequently, the discussions were more focused on issues such as details, comfort and finish - much like they would be in a design review of refined prototypes. Again, each juror had to "make the case" to his or her partner for the designs that deserved merit. As in the design process, some designs which sounded good on paper, fell short in person, while others exceeded expectations.
The iterative review process continued with a larger review cycle. This time three or four pairs of jurors discussed the designs that they had selected. Up until this point, jurors had only examined their assigned subcategories, so it was comparable to presenting a refined design to a larger group that had not been directly involved, such as a broader design or management team, or even a client. During this cycle modifications were made based on constructive input and discussion within the working team. At the conclusion of this phase, each of the award winners had been selected and agreed upon by the majority of each working group.
Finally, all of the designs (both award winners and finalists) were open for discussion to the entire group of 20 jurors. In this last-check round, any juror could suggest raising or lowering the award recognition of any item, whether they had reviewed it previously or not. This provided a valuable diversity of inputs and perspectives, but the decision was ultimately left up to the relevant working team because of their deeper knowledge of the particular subcategories.
In retrospect, it was clear that jurying reflected the design process (arguably, as it should) in many essential ways: small-team collaboration, iterative review, and making the case in front of progressively larger stakeholder groups. Ultimately, like design, it was a mix of creative, intellectual fun and time-constrained challenges.
Last, but not least, a nod of recognition to all of the entrants, my peer jurors, and especially the IDSA staff for their excellent organization and support that made this process possible and succesful.
