ABIT sure is missed, their second life as Universal abit was not the same company and not nearly as good. ABIT made some truly unique products and weren't afraid to take chances and come up with solutions that no-one else were even close to at the time.
AOpen also had a few interesting boards, they weren't always the best, but they did again try to be different, with features like tube amps, although this was way too early, as the onboard audio was pretty crap back in those days.
Aureal, the first real competition to Creative Labs, which sadly died quicker than they should've. Unfortunately they partnered with everyone or anyone and there were at lot of poor quality cards as well.
Palo Alto Products, made some of the best PC cases ever, especially internally. The motherboards were mounted with a single screw. Only review I can find these days is the PA-600, which wasn't their best case
https://www.anandtech.com/show/372 although there are a couple of pictures of the PA-810 here
https://arstechnica.com/civis/viewtopic.php?f=7&t=1015981 a case I was lucky enough to own for a few years. Sadly they never made any new models after these.
USRobotics, but I guess no-one really uses modems any more, although it's a company that truly failed to keep up with times.
I'm sure there are others, but it's so long ago...
Brands I don't miss, PC Chips, WinChip, FIC (although they're kind of sort of still around), Conner (some of the most unreliable hard drives ever), ExcelStor (also really terrible hard drives), Pine (the original company behind XFX), Sparkle, both with dreadful graphics cards and equally dreadful logos. I'm sure there are others here too...