When you read up on Meltdown and Spectre, when it comes to mitigation measures Spectre is far worse. Meltdown fix is simple, kernels get KAISER-based patches, there is a performance hit but it will essentially be done. Spectre needs a much more complex approach - firmware/microcode patches plus potentially vulnerable software to take mitigation measures. Compilers have been improved to do some of this automatically but that would still mean recompiling the software. These measures partially overlap with Meltdown things but not only that. If you want an example, see what all major browser vendors did with patches. Things like making timers less accurate
GPUs are not affected. Driver does stuff on CPU that might be vulnerable to Spectre variants so mitigation measures are taken.
Mitigation measures introduce additional delays, that means performance hot for CPU-limited situations. So far, small hits, but still.
From the previous page of the topic:
The gpu driver runs with priveledge, and by recoding key indirect branches, it closes a side band data leak.