Training neural networks with larger batches in PyTorch: gradient accumulation, gradient checkpointing, multi-GPUs and distributed setups…
New large language models are publicly released almost every month. They are getting better and larger.
You may assume that these models can only be run on big clusters or in the cloud.
Fortunately, this is not the case. Recent versions of PyTorch propose several mechanisms that make the use of large language models relatively easy on a standard computer and without much engineering, thanks to the Hugging Face Accelerate package.
Unlock Your Second Brain with Streamlit and Hugging Face’s Free LLM Summarization: build a Python Webapp running on your PC.
This uses a smaller language model tailored to text summarization. Maybe a good path for assessing student short answers and essays.