What piece of retro tech is your white whale? What do you aspire to own but probably won't ever have the chance to?
I grew up around and was mentored by many DEC veterans. I even work in one of their old manufacturing facilities. I'd love to own a DEC mainframe from their peak in the early 80s. Even a minicomputer is unreasonable for most of us hobbiests. One can dream though
Amiga 4000 with one of those accelerators that comes with a compact flash hard drive adaptor, 128MB of RAM, and an 060 at 65Mhz, plus maybe a retargetable graphics card, as well as as a proper original Commodore CRT monitor, mouse, and keyboard.
Depends on what era of games you want to play, but you can't beat a pentium 2 or 3 windows 98 machine. This will get you support for all but the earliest DOS games plus the heyday of 2D and the start of 3D windows gaming.
Any old beige box of the era will work. The larger the better, since small units of the era had more custom hardware like audio chips that may be hard to find drivers for.
Those are getting harder to get cheap, so if you want an even easier option ask you relatives and neighbors if they have any old computers. There are a ton of Pentium 4 and Core series pcs with Windows XP out there still that will play a ton of retro games still. Anything that had Vista or 7 should be downgradable to XP so you have a lot of options.
It's important to not start throwing upgrades at a new-to-you system. Only upgrade what isn't meeting your needs. There's a huge market for retro PC gaming now, and you can spend thousands on upgrading an otherwise unremarkable system with hardware you don't need.
For monitors, anything with VGA will work for these recommended options. VGA flat screens are worthless nowadays so ask around for a free or super cheap one. The XP option will better support modern USB devices while that can be a struggle with 98, so keyboards, mice, and flash drives are definitely easier on the more modern systems.
Use Archive.org for games and software. Don't bother paying for original media unless you're building a collection.
It would be wrong of me to own one since I probably wouldn't use it and would just be leaving it to rot while somecritter else can't get one, buuuut... I want a TI-99/4A, just because they're weird inside :'D I like weird things.