Category: CoolTools

Posted on

Emerging Architectures for LLM Applications | Andreessen Horowitz

Large language models are a powerful new primitive for building software. But since they are so new—and behave so differently from normal computing resources—it’s not always obvious how to use them.

In this post, we’re sharing a reference architecture for the emerging LLM app stack. It shows the most common systems, tools, and design patterns we’ve seen used by AI startups and sophisticated tech companies. This stack is still very early and may change substantially as the underlying technology advances, but we hope it will be a useful reference for developers working with LLMs now.

https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/

Posted on

Reddit :: Tutorial – train your own llama.cpp mini-ggml-model from scratch!


Tutorial – train your own llama.cpp mini-ggml-model from scratch!
by u/Evening_Ad6637 in LocalLLaMA

Here I show how to train with llama.cpp your mini ggml model from scratch! these are currently very small models (20 mb when quantized) and I think this is more fore educational reasons (it helped me a lot to understand much more, when “create” an own model from.. nothing before. And it helps to understand the parameters and their effects much better)

Otherwise, these mini models could be good enough to be experts on very specific fields, like: only gives text in the style of someone. Like one model could speak like cartman from southpark, another could be a poem and you could implement these ‘person’ in your general chat or role play coversations as supporting roles or minor roles.. to make “group” chats, brainstormings, etc.

And: the discussions on github seems to be very promissing that we will soon be able to fine tune pre-trained big models like llama or vicuna and so on. espcially creating (q)lora adapters should be possible soon : )

this will be the next game changer i think (imagine your model could be finetuned in real time incrementally on top of its lora adapter and with your current conversation as the dataset – what awesome implications would this mean?)

EDIT:

You maybe need the training-script

— Tutorial – train your own llama.cpp mini-ggml-model from scratch!

Posted on

From Medium :: Run Very Large Language Models on Your Computer | by Benjamin Marie | Towards AI

New large language models are publicly released almost every month. They are getting better and larger.

You may assume that these models can only be run on big clusters or in the cloud.

Fortunately, this is not the case. Recent versions of PyTorch propose several mechanisms that make the use of large language models relatively easy on a standard computer and without much engineering, thanks to the Hugging Face Accelerate package.

Source: Run Very Large Language Models on Your Computer | by Benjamin Marie | Towards AI

Posted on

Adding Symphora, my WordPress blog, to the #fediverse via ActivityPub (Take 2)

Second try. Probably need to be subscribed first, then post.

Why just join a Mastodon instance if I can turn my entire blog in to a node in the federated social network space? This is the first post that attempts this feat. If it works I’ll write up what I did to make it work. In the meantime you can follow these antics and more by following elmer@www.symphora.com and @emasters.

See you on the other side!

Posted on

Social annotation tools help students read together

Now, a new study offers evidence supporting what […] has long observed: online social annotation helps students understand and construct knowledge around scholarly content, while at the same time building community.

https://www.insidehighered.com/news/2022/10/12/social-annotation-technology-helps-students-read-together

There is room for the use of social annotation tools in legal education especially as more teaching resources move online. Tools like Hypothesis would provide law students with ways to highlight and annotate online materials and share those as annotations with study groups, peers, and teachers.

Hypothesis is available in CALI Lawbooks, the online publishing platform that for CALI members