I was using the "most other people" in the relative sense (as in, more than half). It also depends on one's definition of "a lot". My expectation is that less than 1% of people overclock their cards because less than 1% of people actually know how. It's not nearly as straightforward a task as overclocking a CPU and I wouldn't expect that many people would be comfortable messing with their video card's BIOS, especially knowing that it would void the warranty if the vendor ever found out. I undervolted my RX 5700 XT for mining with it but I wasn't about to touch the BIOS in my RX 6800 XT and just used it stock with the fans set to a constant 100%.
I personally agree with you that there are quite a number of people who routinely do this but I'm sure that you'd agree that they're a tiny minority.
