If you just want to modify the existing OS and write apps for it the look at the kindle hacking community on the mobileread forums. The best devices are probably up to and including 5th generation kindles since I believe they are the latest to still have working soft-jailbreaks (but that could have changed since last I checked). I believe all kindle models are trivially rootable if you are willing to buy a 1.8v usb to serial adapter, open the kindle up and solder on three wires. If you want a 100% FOSS distro on your device and you're not afraid of soldering, cross-compiling and super-pre-alpha code then read on.
I've been working on a linux distro for i.MX based e-paper readers (kindle, kobo, etc.) for a while and just had two other hackers join me on the project. Currently we are furthest along with the Kindle 4th generation non-touch with a slightly modified super minimal Debian booting and basic graphics support (Xorg works but no window manager and no screen auto-update yet. e-ink is weird). I believe we've managed to strip out all binary blobs so it's really all open source now. This system is still using an ancient kernel (a slight variation on the one used by the stock OS). We just got the latest stable kernel booting a few days ago but only barely (not even mmc support yet). There's just three of us for now, and we hang out in #fread.ink on freenode and our code is up on https://github.com/fread-ink
The Kindle hacking community believed they had a permanent solution prior to the release of the latest firmware version; unfortunately, Amazon seems to have finally found a way around it.
I don't think anyone with experience hacking kindles believed it was a permanent solution. Unfortunately, most of the technical expertise in that area is fleeting.
I'd recommend an older Kindle if you want to get root easier. 5th generation Paperwhite and older models should be cheap and have various exploits.
Probably would recommend a Kobo or something else though. Kindles are awesome pieces of hardware but they do try to keep them secure. Not the best choice for an open platform.
I've been working on a linux distro for i.MX based e-paper readers (kindle, kobo, etc.) for a while and just had two other hackers join me on the project. Currently we are furthest along with the Kindle 4th generation non-touch with a slightly modified super minimal Debian booting and basic graphics support (Xorg works but no window manager and no screen auto-update yet. e-ink is weird). I believe we've managed to strip out all binary blobs so it's really all open source now. This system is still using an ancient kernel (a slight variation on the one used by the stock OS). We just got the latest stable kernel booting a few days ago but only barely (not even mmc support yet). There's just three of us for now, and we hang out in #fread.ink on freenode and our code is up on https://github.com/fread-ink
You should look at the repo https://github.com/fread-ink/fread-vagrant to get started.