Here are a few things to consider:
Case: You will most likely want good airflow. A side mounted 80mm-120mm fan along with a rear exaust (120mm is best) fan provides excellent airflow. Noise levels should be considered also. If you want a quieter case, i recommend steel over aluminum.
Motherboard: Want to use AMD or Intel? Choose a motherboard that has a socket that fits your choice of processor(Socket A, Socket 478, Socket 754, etc). Most motherboards geared towards gamers already include features such as on board audio, LAN, and Serial ATA interfaces.
Processor: Main choices are Intel and AMD (although you could go with VIA, but not the best choice considering gaming and price). Best choices for AMD would be the Athlon XP processors with Barton cores or the new Athlon 64 processors. I have a few AMD machines but am not as experienced with them. I'd say best bang for the buck offering from Intel would be the 2.4C Pentium 4 (socket 478) (NORTHWOOD CORE SCREW PRESCOTT). The 2.4C overclocked extremely easy to 3.2ghz with temperatures at 35*C while running folding@home.
Memory: I like DDR1. Go with a motherboard and RAM that use DDR1. Intel P4's let you run dual channel, and if I'm correct some Athlon 64's let you run dual channel also. (dual channel is where you have two sticks of ram = better performance). I'm using OCZ value ram PC3200. Very reliable.
Hard drive: A nice biggin. 7200 RPM. Be sure to read online reviews before making your decision. Also, if your motherboard supports Serial ATA (SATA), get a SATA drive.
Video Card: The newest generation of graphics cards are neck and neck with each other. Not going to go deep into this, but for the newest generation of cards, go with the X800 series(ATI) or the 6800 series (nVidia). You wont be dissapointed there.
If you want to save some dough, go for the 9800Pro (ATI). Great card. My MSI 9800Pro actually turned out to be a 9800XT, a wonderful suprise. Overclocked very nicely.
Cooling: The intel stock heatsink does a great job if you have a Northwood core on a P4. AMD's coolers do a fine job too, but I would prefer a better solution for overclocking. Copper heatsinks are better, remember that. Oh, and for thermal material between heatsink and processor, a dab of Arctic Silver 5 is king. Just dont read the intstructions included with the package. Read the online directions.
Your might want to have a fan on the side of the case pushing cold air down onto the processor, and a fan in the back pushing air out of the case. I find its always a good idea to cut out the fan guards as they cause the fans to make more noise and decrease airflow.
Its important that your read alot before building your first computer. And remember, if it doesnt start up the first time your try to turn it on, dont fret. Make sure there is no case - to - motherboard contact (this was a problem i had last year, thought i had dead hardware).