I just read this article on wired. The guy uses a kit from this site:
to create his own flash memory based mp3 player. Sounds pretty fun, the kit without a memory card is about 88 dollars. It can run on either MMC or SD memory cards with sizes up to 1GB - I recommend newegg for good prices on these - around 52 bucks.
I may try to do this sometime in the future, but someone wants to beat me to it and post instructions in the template style, go for it.