Voltage kills CPUs, not temperature (CPU will shut the system down long before temperature becomes an issue that could cause damage).
One relevant thing to bear in mind if you're chasing an OC is that lower temperatures reduce voltage leakage, so the cooler you run your chip, the less voltage you need to feed it to achieve the same frequency and stability (although this is a relatively minor difference).
The main temperature sensitive parts of any system are the power delivery (PSU and VRM) and RAM/VRAM/SSD storage. These should be kept cool if possible. Ideal RAM temperature is below 35 °C, PSU/VRM the cooler the better up to a point, around 20-30 °C is "ideal" but hard to achieve. SSD storage should be kept below 75C if possible, NAND flash actually performs best above a certain temperature, but higher temps can degrade. AFAIK RAM stability is tied to temperature, B die in particular gets unstable above 40 °C.