AMD Publishes User Guide for LM Studio - a Local AI Chatbot

[T0@st]

AMD has caught up with NVIDIA and Intel in the race to get a locally run AI chatbot up and running on its respective hardware. Team Red's community hub welcomed a new blog entry on Wednesday—AI staffers published a handy "How to run a Large Language Model (LLM) on your AMD Ryzen AI PC or Radeon Graphics Card" step-by-step guide. They recommend that interested parties are best served by downloading the correct version of LM Studio. Their CPU-bound Windows variant—designed for higher-end Phoenix and Hawk Point chips—compatible Ryzen AI PCs can deploy instances of a GPT based LLM-powered AI chatbot. The LM Studio ROCm technical preview functions similarly, but is reliant on Radeon RX 7000 graphics card ownership. Supported GPU targets include: gfx1100, gfx1101 and gfx1102.

AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas." Their blog also puts a spotlight on LM Studio's offline functionality: "Not only does the local AI chatbot on your machine not require an internet connection—but your conversations stay on your local machine." The six-step guide invites curious members to experiment with a handful of large language models—most notably Mistral 7b and LLAMA v2 7b. They thoroughly recommend that you select options with "Q4 K M" (AKA 4-bit quantization). You can learn about spooling up "your very own AI chatbot" here.



View at TechPowerUp Main Site | Source

 

[Minus Infinity]

AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.

 

[Firedrops]

This is really such an insane f**king joke. They've "caught up" by just publicly admitting how incompatible all their products are with LLMs?

 

[R-T-B]

AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.

It can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.

 

[Minus Infinity]

It can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.

In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.

 

[mb194dc]

In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.

Got to keep hyping it whilst some killer non niche use case is found for LLMs.

 

[TumbleGeorge]

In the context of a home PC it largely is useless

For today mostly agree.

 

[trsttte]

In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.

It can still be usefull to bounce simple ideas from, it's useless in the context companies like Microsoft want to push it, integrating it in all corners of the OS. But having a chat bot quickly accessible and offline is pretty cool not to mention because the free party we've had with chatgpt and bard/gemini will certainly end in the not so distant future as it's simply not a sustainable service, openAI is literally bleeding money as is google, though certainly much much less because they're less used and they were much better prepared with accelerators built in house ready to go.

 

[R-T-B]

In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.

That's a fair assesment.

 

[Minus Infinity]

For today mostly agree.

Only thing I see it useful for at the moment is photo/video processing. But that doesn't need a friggin chatbot or copilot.

 

[Dr. Dro]

AMD, always chasing after NVIDIA with subpar implementations of the cool things that NVIDIA does, without regard for the obvious elephant in the room which they absolutely refuse to even acknowledge, let alone do anything about it.

For an answer to "Chat with RTX", that even someone with zero knowledge of computers could use, this is sorely lacking.

 

[tpa-pr]

Funnily enough, their guide didn't work for me. I couldn't get any models to run on my 7900 XTX. But it did point me in the right direction which was KoboldCPP: https://github.com/YellowRoseCx/koboldcpp-rocm

This was VERY easy to set up and get running and it has ROCM support in Windows too.

 

[rhqq]

hilarious. at the same time one can ollama on linux on multi (mixed) GPU automagically starting with rocm 6.0 release since december 2023. i'm running it on 780m igpu + 7600m xt egpu (via usb4).

 

[ShaneTeks]

Yeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.

 

[rhqq]

Yeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.

wrong. it works great under linux. both images (sd.next) and LLMs (ollama)

 

[Dr. Dro]

wrong. it works great under linux. both images (sd.next) and LLMs (ollama)

Well, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."

 

[rhqq]

Well, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."

it just shows how little you know about the thing. the whole market for AI, both NV and AMD is almost purely Linux. things just work there, unlike windows.

so again: it works great, if you follow the industry standard.

 

[ShaneTeks]

wrong. it works great under linux. both images (sd.next) and LLMs (ollama)

Have you perhaps got ComfyUI to work with the above gfx1100, 1101, 1102? and have you tested and compared the performance running SD with DirectML compared to using ROCm?