Meh, there's not really much point without water cooling, but it would be damn good in a small case if it was on water, but then so would a mini nVidia card. This is way cooler than a nVidia card though.
Pretty much every case can hold a 120mm radiator, so watercooling COULD be the solution, although that radiator would have to be so small for compatibility reasons that air cooling might work just as well. Lower clockspeeds or lower CU count are the solution. Personally, I think 40-48CU and high-ish clockspeed are the way to go and then air cooling could work just fine. Vega IS an efficient and powerful arch, just not if it has to compete with the likes of 1080's and 1080 ti's, so AMD really only has to figure out a way to make high-end cards wayyyyy more efficiënt, while still keeping them profitable. Larger dies could help for example, because then you could keep clockspeeds down and increase efficiency, but that would make vega wayyyy too expensive and more 64 CU's seems to be the limit for vega and faster RAM Also would be too expensive probably, but would help a bit.
Maybe latency is a major part of the problem as well seeing as more CU's is worse sometimes. Even "12nm" would help with that, probably about as much as with zen, if not more because so many different clusters have to work together. Faster RAM, lower latency in the gpu itself and mildly increased efficiency (which probably just translates to higher clockspeeds, which also means lower latency) will make a fairly significant difference, 10% or so with gpu limited workloads.
7nm (gaming) vega/navi should come before the end of Turing/Ampere/Volta, so AMD CAN still fix things. Their lack of innovation in years past reaply messed them up and nvidia could be slowing down and in 2020 Intel will join the clustercluck, so they have about 2 years to get creative!