If you're talking about going to Linux on an existing PC, you could always just partition your drive and keep Windows on the other partition. It's worked well for me (Red Hat on the main partition, Windows on the other "just in case") -- I don't think I've booted into Windows in a year or two.
Quickbooks won't work on anything very well outside of windows (the Mac version is terrible, too). If you have programs like that you'll probably want to keep Windows around at least for that. I can't really comment on the "solutions" for Quickbooks, the only person I know who has tried one was not really pleased (I think lingering trouble with the Database Server Manager), and switched to GnuCash I believe. I have also heard positive things about Quasar, though I do not use either of them. My experience with OpenOffice has been great, though. Never had any trouble with that.
If you're looking at switching an existing computer over, you may also want to ensure all your hardware is compatible before you take the plunge -- most of it probably is by now, but better to find out sooner rather than later.
All in all I would say I'm very happy with the change. My computer runs faster, and I can't remember the last time it crashed. Most of what I do with the home PC (Linux box) is word processing and surfing BB&B, which Linux does flawlessly. I leave the financial stuff on the work (Windows) laptop.
Let us know how it goes...