As I’ve been studying evaluation over the last month or so I’ve been struck again and again by the importance of the human aspect of the evaluation process, how it can make a world of difference to recognize the human needs and commitments of stakeholders and clients or acknowledge the human biases and shortcomings of the evaluator. I haven’t often focused on how valuable the human element is in the processes, though, and it’s a point that deserves to be emphasized: our humanity is not just a liability when it comes to evaluation or other kinds of systematic inquiry.
I was considering this while playing one of my favorite computer games last night, Sid Meier’s Civilization V. In the game you manage your own civilization: establishing and developing cities, gathering resources, and interacting with other civilizations around you. In the single-player game, the other civilizations are, of course, simulated by the computer. What was interesting to me is that I was constantly evaluating things: what buildings to build, where to invest my limited resources, what technologies to research, what resource trades or diplomatic deals to make, how big to build my military, whether to go to war. The cumulative result of these decisions was a distinct configuration of game elements that could be quickly surveyed on the main screen, where you have a from-the-sky view of the game world.
Later on in this particular game I became especially concerned about the Persian Empire, which had begun to aggressively invade the neighboring civilizations in a clear attempt to take over the world. My worry was that they were going to attack me before I could peacefully win the game through my empire’s scientific achievement. Finally, I reached the point where I felt confident in my impending victory, and felt just a little curious about this empire that been such a troubling adversary. So I flew a plane out over their territory to have a good look at them. What I saw was surprising. Their empire looked nothing like mine. The land hadn’t been developed much. There weren’t many roads. Most of their cities had been conquered from other civilizations, but rather than annexing and developing them, they had been kept as hollow (but cheap) puppet states. The whole region seemed empty and soulless and stagnant, at least in comparison with the empire I’d created.
Now, the computer had models that told it why the Persian Empire should be built in this particular way, and it had obviously proven a very effective strategy against the other computer-controlled civilizations. Yet I could see, even at a glance, that my empire not only had a better aesthetic quality than the Persian Empire, but that it was actually stronger because of the decisions I’d made as a human player. Despite being able to account for and process so much more data than me, despite the troubling show of military force along my border, this game’s Persian Empire was substantially weaker than my smaller civilization. It put up an impressive front but, if it had come to war, I could have taken them.
So, this has been a somewhat lengthy and definitely geeky diversion to make a very small point about evaluation: human beings can deal with certain kinds of complexity better than machines can, and this is in spite of the tremendous processing power of computers. I don’t know how much anyone is actually trying to turn evaluation over to computers these days, but I think we often value inquiry that is designed to operate more systematically and efficiently, more mechanically, removing the human factor as much as possible. While there is definitely a space for calculated decision-making, I also think we would do well to remember the strength of taking a human perspective on complex systems. We’re better at handling complexity than we sometimes give ourselves credit for.