Are you sure? Went looking to refresh my memory and...
News and information from Gentoo Linux
www.gentoo.org
Yep, that's just boot media. It only gets you to a live command line to start the install on your own. The process is basically extracting a stage tarball yourself into a disk that you partitioned yourself, and then editing a plethora of config files by hand just to get the thing to boot to a package manager-enabled command line.
They have a GUI download too, but it doesn't have an installer either. Just enables you to read help docs while you work.
Here, have a looksie at the handbook:
wiki.gentoo.org
The payoff is also usually pretty minimal. I live in Linux and I'd be the first to tell you that it's not worth it.
If you mean from compile flags, I'd agree.
You get more from picking and choosing what goes into it and what features are on/off, what versions are used, etc. Which is precisely why I don't have time for it, that's time consuming shit that you can get lost in.
But at any rate, at the end of the day I've reached the point where I just have to USE my computer, lol. So yeah, what good is optimizing the heck out of it for an even (theoretical, not likely) 10% increase if you never get to use it?
So yeah, I agree completely from the point of view of compile flags, and mostly alltogether agree unless you have excess time and nothing better to do.