Thirst for Truth in Card Sorting

I know the phrase “card sorting” either baffles, bores or does something else beginning with ‘b’ to almost everyone that hears it. Perhaps the most vocal source of information and critique of card sorting techniques recently has been the force that is Maadmob’s Donna Maurer. I recently caught her attention on this subject via comments on the blog of another Australian IA, Leisa Reichelt.

Leisa had been blogging about her negative experience of card sorting in the context of “validating” an information architecture. I’d been thinking about this and the wider issue of whether related techniques might be better or worse, and under which circumstances.

I suppose my own attitude to simple open card sorting is that it’s a useful, but limited, way of getting inside the heads of users if you don’t know jack about the information you are about to design for them. I’m inclined to think that if you do know something about it then it’s a much less effective tool. But how do you know how less effective it is for you as a designer, given that you’re likely to have your own ideas about taxonomy and navigation, as Leisa had?

Last week, Eric Scheid said on SIG-IA that he’d like to see the results of a card based classification evaluation compared against the results of a closed card sort, and both interpreted in light of the results of a category agreement analysis. That’s a very good idea, and one that Donna says she might be able to persue, so watch this space, sorters.