- Joined
- Apr 24, 2020
- Messages
- 2,721 (1.60/day)
ok, can someone finally explain what the hell "AI capabilities" or "AI hardware" even is?
For what I know its just calculations, its just stuff google has been doing for years already and otherwise simulation programms.
What the heck is different about the hardware...what does it even mean? is it like an ASIC situation?
Every "AI Capability" I've seen is a 16-bit floating point matrix multiplication acceleration of some kind, but its obscure how far any particular company goes.
NVidia has 4x4 sparse matrix (if more than half the locations are 0, NVidia executes them a bit faster).
AMD Called dot-product an AI routine a few years ago.
Intel calls their int8 matrix multiplication AVX512 instructions an "AI" instruction.
Various ASICs of 8x8 to 256x256 matrix sizes are also "AI capabilities", which are more efficient in theory. But in practice, NVidia / AMD / Intel just builds on a more advanced node (ex: 3nm), which more than wipes out the advantages of larger matrix instructions. Some companies call bfloat16 AI, but others are half-float (another 16-bit format). Other companies support both 16-bit formats. Some companies (ex: Intel) like to call 8-bit integer matrix multiplications AI.
So there's no standardization at all here.