HP gets thin
I was rooting around the HP site for information on Linux-equipped HP desktops, and I'm not exactly sure they'll ship you one with Linux pre-installed just yet.
But HP is in the thin-client business, and they offer the relatively inexpensive, diskless workstations in many configurations, including with Linux installed. They've got one with a lot of power for between $450 and $700 (that's a lot to pay for a thin client) -- but it does have adequate memory, a 1 GHz CPU and Debian.
For $199, the systems run 400 MHz Via processors with 128 MB RAM (16 MB reserved for video, unfortunately), and between 34 and 64 MB of flash memory. Underpowered, yes, but that depends on the job they're doing. There are also good deals out there from Devon IT and Cappuccino PC, among others, but HP is surprisingly competitive in this space.
Photo: HP Compaq t5125 Thin Client