(sorry if anyone got this post twice. I posted while Lemmy.World was down for maintenance, and it was acting weird, so I deleted and reposted)
(sorry if anyone got this post twice. I posted while Lemmy.World was down for maintenance, and it was acting weird, so I deleted and reposted)
Yeah just use llamacpp which uses cpu instead of gpu. Any model you see on huggingface.co that has “GGUF” in the name is compatible with llamacpp as long as you’re compiling llamacpp from source using the github repository.
There is also gpt4all which is runs on llamacpp and is ui based but I’ve had trouble getting it to work.
The best general purpose uncensored model is wizard vicuna uncensored