Nope.
In a typical modern datacenter, which mixes CPUs and GPUs (or coprocessors like Xeon Phi), the two types of processors complement each other. CPUs are better at some types of calculations, GPUs are better at others.
If you need more GPU potential, you just add a GPU. There's really no point in adding an APU, which is inherently slower than a PCI-card variant on the same architecture.
Moreover, from a practical standpoint, it's fairly important to have identical GPUs in the system.
And BTW: CPUs also cover the "maintenance" tasks - like moving data around (disks <-> RAM <-> GPU), so replacing them with APUs could actually bottleneck the whole system, not make it faster.
Nevertheless you need cpu cores and gpu cores in every box running gpu accelerated apps. Apu attempts to provide both in the same package: if you do not need external PCIe busses then you save power.
As for 'just add a gpu': I've worked in a datacenter managing large scale compute cluster systems. You do not just 'add a Gpu' to these systems. Downtime to do so costs money, and risk of hardware failures due to power cycles is very real, and the calculations running on the cluster might run for months.
Instead what they do with clusters is spec a cluster to meet expected tasks for a period of several years going forward. Then after several years they look at whether a major overhaul ie 'just add a gpu' to all or some nodes is needed. Most of the time they will be replacing cpu and gpu in such a major overhaul, so why not replace an apu?
As for Apus specifically: they are SOCs nowadays. That means less external circuitry which means cheaper to make each node and at least somewhat cheaper to operate for the same FLOPS bang. For a medium size cluster if you can knock 10% off your electricity bill or cooling bill you will save 100's of thousands of dollars. If AMD does produce an HPC APU you can be sure it will get buy in, you will see it used in some production clusters.
And why else might anyone be interested in APUs over CPU+PCie+GPU? One good reason is shared registers between cpu and gpu. Bandwidth between cpu & gpu for data sitting in shared registers is absolutely astronomical: far far higher than when the PCIe bus is in the way.
