Things are more interconnected than they appear. Example, many U.S companies outsource their IP to be made elsewhere for cheaper labour, and as the cost to the U.S increases, so it will also affect other countries, it's a free-for-all at the moment, interesting times, lot's of money to be made for those that have capital and are not fearful.
To expand a little more, remember there are more in silicon to make a chip, well a GPU or CPU, there are other materials that are sourced from elsewhere etc. and so forth, which are also subject to tariffs from not only the U.S, but elsewhere.
When you buy an RTX card, for example a Zotac - what's happening is that Nvidia in the US are paying TSMC for production, and then selling (as a US entity) to PC Partner (Zotac) in Taiwan.
Currently, silicon is exempt from tariffs, but even if there were tarrifs on US silicon purchased in Taiwan, PC Partner would only pay the reciprocal Taiwanese import tariff on the purchase of the bare GPU die itself from Nvidia, and presumably since Nvidia is selling out of a Taiwanese branch near TSMC, even that might have a workaround to avoid import tariffs since the sale is from TSMC/Nvidia in Taiwan to PC Partner in Taiwan. The bare dies aren't being shipped back to the US just to return to customers in Taiwan again!
Regardless, that import tariff of a GPU die sold from Nvidia to PC Partner is going to be a pretty small fraction of the overall graphics card BOM which is mostly Taiwanese and Chinese parts. Samsung RAM from South Korea, Chinese PCB, Surface mount components, cooling etc. Also packaging, distribution, and admin - none of that is subject to tariffs, and none of the goods from Taiwan sold to anywhere in the world
except the US will suffer from US tariffs which are a blight mostly afflicting just the US population for now.