Wired, that bastion of celebrating geekdom, put together a post suggesting the essential geek reads.
I love the list, which includes such masterpieces as The Hithchiker's Guide to the Galaxy, Watchmen and The Lord of the Rings. Did I enjoy the post simply because it consisted mostly of books I've read -- and if not read, have at least heard of and read enough about to fake like I have -- because it made me feel like a well-read geekology professor? Probably. But that's beside the point.
If you want to be a real geek, or be able to hold a literature-based conversation with one, you need to read these books. Or at least their Wikipedia pages.