Once I got it downloaded I tried to use the oogabooga webui, but ran into issues, so I wanted to convert it to gguf format and use with GPT4ALL. I found some good instructions:https://www.secondstate.io/articles/convert-pytorch-to-gguf/ I converted the PyTorch model to GGUF in FP16 weights.Then when I got around to trying to quantize it (with out … Read more

In the previous post I covered setting up GPT4ALL, and in this post I’ll show results of tests of coding and explanation. To keep up with what LLMs are doing the best here are a couple resources: Open LLM Leaderboardhttps://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard Open LLM Leaderboard Reporthttps://github.com/dsdanielpark/open-llm-leaderboard-report A list of coding tasks suggested to me, and adapted, that … Read more

LLM capt bots are one of the coolest and most useful technologies to come out – an encyclopedia and AI assistant in conversational form at your fingertips. There’s the standard ChatGPT, but what if you want to run things on your own computer? Opensource and free to use with little restriction?Good news, you have options! … Read more

A LinkedIn post by Soumen Sarkar highlighted how tech can become depressing though negative feedback in particular.Where code reviews become tense and some engineers “lacked tact and nuance”. I suggested a fix where In this post I will explore a proof of concept using a local LLM, namely mistralai/Mistral-7B-v0.1 q8_0 Say you have this code … Read more