Friday, March 8th 2024

AMD Publishes User Guide for LM Studio - a Local AI Chatbot

by
T0@st
 Discuss (17 Comments)
AMD has caught up with NVIDIA and Intel in the race to get a locally run AI chatbot up and running on its respective hardware. Team Red's community hub welcomed a new blog entry on Wednesday—AI staffers published a handy "How to run a Large Language Model (LLM) on your AMD Ryzen AI PC or Radeon Graphics Card" step-by-step guide. They recommend that interested parties are best served by downloading the correct version of LM Studio. Their CPU-bound Windows variant—designed for higher-end Phoenix and Hawk Point chips—compatible Ryzen AI PCs can deploy instances of a GPT based LLM-powered AI chatbot. The LM Studio ROCm technical preview functions similarly, but is reliant on Radeon RX 7000 graphics card ownership. Supported GPU targets include: gfx1100, gfx1101 and gfx1102.

AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas." Their blog also puts a spotlight on LM Studio's offline functionality: "Not only does the local AI chatbot on your machine not require an internet connection—but your conversations stay on your local machine." The six-step guide invites curious members to experiment with a handful of large language models—most notably Mistral 7b and LLAMA v2 7b. They thoroughly recommend that you select options with "Q4 K M" (AKA 4-bit quantization). You can learn about spooling up "your very own AI chatbot" here.
Sources: AMD Community, Tom's Hardware

Related News

Add your own comment

17 Comments on AMD Publishes User Guide for LM Studio - a Local AI Chatbot

#1
Minus Infinity
AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.
Posted on Reply
#2
Firedrops
This is really such an insane f**king joke. They've "caught up" by just publicly admitting how incompatible all their products are with LLMs?
Posted on Reply
#3
R-T-B
Minus InfinityAMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.
It can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.
Posted on Reply
#4
Minus Infinity
R-T-BIt can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.
In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
Posted on Reply
#5
mb194dc
Minus InfinityIn the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
Got to keep hyping it whilst some killer non niche use case is found for LLMs.
Posted on Reply
#6
TumbleGeorge
Minus InfinityIn the context of a home PC it largely is useless
For today mostly agree.
Posted on Reply
#7
trsttte
Minus InfinityIn the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
It can still be usefull to bounce simple ideas from, it's useless in the context companies like Microsoft want to push it, integrating it in all corners of the OS. But having a chat bot quickly accessible and offline is pretty cool not to mention because the free party we've had with chatgpt and bard/gemini will certainly end in the not so distant future as it's simply not a sustainable service, openAI is literally bleeding money as is google, though certainly much much less because they're less used and they were much better prepared with accelerators built in house ready to go.
Posted on Reply
#8
R-T-B
Minus InfinityIn the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
That's a fair assesment.
Posted on Reply
#9
Minus Infinity
TumbleGeorgeFor today mostly agree.
Only thing I see it useful for at the moment is photo/video processing. But that doesn't need a friggin chatbot or copilot.
Posted on Reply
#10
Dr. Dro
AMD, always chasing after NVIDIA with subpar implementations of the cool things that NVIDIA does, without regard for the obvious elephant in the room which they absolutely refuse to even acknowledge, let alone do anything about it.

For an answer to "Chat with RTX", that even someone with zero knowledge of computers could use, this is sorely lacking.
Posted on Reply
#11
tpa-pr
Funnily enough, their guide didn't work for me. I couldn't get any models to run on my 7900 XTX. But it did point me in the right direction which was KoboldCPP: github.com/YellowRoseCx/koboldcpp-rocm

This was VERY easy to set up and get running and it has ROCM support in Windows too.
Posted on Reply
#12
rhqq
hilarious. at the same time one can ollama on linux on multi (mixed) GPU automagically starting with rocm 6.0 release since december 2023. i'm running it on 780m igpu + 7600m xt egpu (via usb4).
Posted on Reply
#13
ShaneTeks
Yeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.
Posted on Reply
#14
rhqq
ShaneTeksYeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.
wrong. it works great under linux. both images (sd.next) and LLMs (ollama)
Posted on Reply
#15
Dr. Dro
rhqqwrong. it works great under linux. both images (sd.next) and LLMs (ollama)
Well, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."
Posted on Reply
#16
rhqq
Dr. DroWell, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."
it just shows how little you know about the thing. the whole market for AI, both NV and AMD is almost purely Linux. things just work there, unlike windows.

so again: it works great, if you follow the industry standard.
Posted on Reply
#17
ShaneTeks
rhqqwrong. it works great under linux. both images (sd.next) and LLMs (ollama)
Have you perhaps got ComfyUI to work with the above gfx1100, 1101, 1102? and have you tested and compared the performance running SD with DirectML compared to using ROCm?
Posted on Reply
Add your own comment
May 21st, 2024 14:08 EDT change timezone

Latest GPU Drivers

New Forum Posts

Popular Reviews

Controversial News Posts