• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Intel AI Platforms Accelerate Microsoft Phi-3 GenAI Models

btarunr

Editor & Senior Moderator
Staff member
Joined
Oct 9, 2007
Messages
47,291 (7.53/day)
Location
Hyderabad, India
System Name RBMK-1000
Processor AMD Ryzen 7 5700G
Motherboard ASUS ROG Strix B450-E Gaming
Cooling DeepCool Gammax L240 V2
Memory 2x 8GB G.Skill Sniper X
Video Card(s) Palit GeForce RTX 2080 SUPER GameRock
Storage Western Digital Black NVMe 512GB
Display(s) BenQ 1440p 60 Hz 27-inch
Case Corsair Carbide 100R
Audio Device(s) ASUS SupremeFX S1220A
Power Supply Cooler Master MWE Gold 650W
Mouse ASUS ROG Strix Impact
Keyboard Gamdias Hermes E2
Software Windows 11 Pro
Intel has validated and optimized its AI product portfolio across client, edge and data center for several of Microsoft's Phi-3 family of open models. The Phi-3 family of small, open models can run on lower-compute hardware, be more easily fine-tuned to meet specific requirements and enable developers to build applications that run locally. Intel's supported products include Intel Gaudi AI accelerators and Intel Xeon processors for data center applications and Intel Core Ultra processors and Intel Arc graphics for client.

"We provide customers and developers with powerful AI solutions that utilize the industry's latest AI models and software. Our active collaboration with fellow leaders in the AI software ecosystem, like Microsoft, is key to bringing AI everywhere. We're proud to work closely with Microsoft to ensure Intel hardware - spanning data center, edge and client - actively supports several new Phi-3 models," said Pallavi Mahajan, Intel corporate vice president and general manager, Data Center and AI Software.



As part of its mission to bring AI everywhere, Intel continuously invests in the AI software ecosystem by collaborating with AI leaders and innovators.

Intel worked with Microsoft to enable Phi-3 model support for its central processing units (CPUs), graphics processing units (GPUs) and Intel Gaudi accelerators on launch day. Intel also co-designed the accelerator abstraction in DeepSpeed, which is an easy-to-use deep learning optimization software suite, and extended the automatic tensor parallelism support for Phi-3 and other models on Hugging Face.

The size of Phi-3 models is well-suited to be used for on-device inference and makes lightweight model development like fine-tuning or customization on AI PCs and edge devices possible. Intel client hardware is accelerated through comprehensive software frameworks and tools, including PyTorch and Intel Extension for PyTorch used for local research and development and OpenVINO Toolkit for model deployment and inference.

What's Next: Intel is committed to meet the generative AI needs of its enterprise customers and will continue to support and optimize software for Phi-3 and other leading state-of-the-art language models.

For performance and technical details, visit the Intel Developer Blog.

View at TechPowerUp Main Site
 
Joined
May 22, 2024
Messages
413 (1.96/day)
System Name Kuro
Processor AMD Ryzen 7 7800X3D@65W
Motherboard MSI MAG B650 Tomahawk WiFi
Cooling Thermalright Phantom Spirit 120 EVO
Memory Corsair DDR5 6000C30 2x48GB (Hynix M)@6000 30-36-36-76 1.36V
Video Card(s) PNY XLR8 RTX 4070 Ti SUPER 16G@200W
Storage Crucial T500 2TB + WD Blue 8TB
Case Lian Li LANCOOL 216
Power Supply MSI MPG A850G
Software Ubuntu 24.04 LTS + Windows 10 Home Build 19045
Benchmark Scores 17761 C23 Multi@65W
FWIW the newly-released 4k-context medium version model is actually pretty nice*; Probably one of the best open-weight LLM you can run right now with 16GB of RAM or VRAM, quantized.

You don't need anything as specific as the "Intel AI platform" for it to run well, though. A GPU with 16GB of VRAM would do just fine, and do it much faster - It should still be acceptably fast on mostly any modern CPU though, should anyone wish to try.

*Reasonable-sounding as long as you don't ask it for anything specific, specialized, or niche, as always for these sort of thing, but especially for smaller ones runnable on a consumer video card, or at usable speed in RAM.
 
Top