#1) Get one with expandable memory. The memory that comes with the unit is useless and will be outdated in less than a year. You want to get some cheap memory at ebay (or whereever). I like to have at least 256 mb.
#2) Choose what kind of memory you want. If you have a camera that uses "Compact Flash", or "Memory Stick", or whatever, you might want to get an mp3 player that uses the same kind. (I prefer compactflash)
#3) Forget about the junky cable and software that comes with the unit. You want to get a USB adapter that takes your memory card. This is about $20 and will mount your memory card as a disk drive in windows. You can actually use it as a portable hard drive, great for swapping software and songs with your friends. There are some exceptional mp3-players what will, by themselves, mount the memory as a disk drive (but, this is the exception).
#4) Don't be afraid of weird chinese brands being sold at supercheap prices on ebay. I have two and they're great.
#5) Happy shopping.