Something in my last post has started me thinking. The role of the referee has evolved along with everything else in our hobby. Even the name went from referee (my preferred term, in case you haven't noticed) to Dungeon Master. I think this is worth looking into closer.
In the olden days, the guy that ran the game was called the referee. That was his role, to referee the events and conflicts of the game. He was supposed to be completely impartial, neither favoring nor hindering the player characters with his behind-the-scenes knowledge. He was instructed to devise cunning traps and carefully craft certain encounters. The rest of the encounters were to be left to chance. Once committed to paper, the traps and encounters were to be administered with impartiality. Curiously enough, even though in those early days there were very few modules (as we called adventures), and no published settings as we know them today, the referee was to be an impartial judge. Almost as if he had no creative connection to the material.
Sometime prior to the release of AD&D it was suggested that the referee be called Dungeon Master (DM). I believe it was first published as a term in Dragon magazine, called The Dragon in those days. It is ironic to me that while over time published settings became the standard, the connotation of the referee shifted. Where the impartial term referee was first used, the participant in that role was forced to devise their own setting. Later, when the referee became "master" he was master over a pre-written, pre-imagined, published setting.
In the days of the referee, there was little to no thought wasted on balancing encounters. Players were given free reign to go anywhere they wanted and pick a fight with whomever they found there. If the fight turned out to be more than they could handle, they better have a Plan B. One of the best pieces of player advice in the old days was "Know when to run away. It is always better to live to fight another day than force a bad fight."
Now, D&DIV is all about a succession of well-balanced encounters. They are designed to challenge the characters. The inference there, to me, is that it shouldn't necessarily be deadly, just challenging. Kind of like watching a Conan movie. You know there may be a desperate fight and Conan will be challenged. We don't ever fear for his life though, because it wouldn't be a Conan movie without Conan. It may very well be challenging. It may be a desperate, pitched battle. One thing it won't be, though, is suspenseful. So, the latest iteration has the DM carefully balancing encounters.
I'll take a sandbox, a few random encounter tables, and a couple of purpose-designed lairs any day. Then again I am a Referee.