Joaoalvarenga

Models by this creator

🤔

bloom-8bit

joaoalvarenga

Total Score

75

The bloom-8bit model is a quantized version of the bigscience/bloom language model, a 176 billion parameter model. Inspired by Hivemind's GPT-J-6B with 8-bit weights, this model applies Low Rank Adaptation (LoRA) to reduce the memory footprint to around 180GB, making it deployable on traditional Kubernetes clusters. The main goal is to provide a compressed version of the powerful BLOOM model that can be run and fine-tuned with less memory. Model inputs and outputs Inputs Text prompts for language generation tasks Outputs Continued text sequences based on the provided prompt Capabilities The bloom-8bit model retains the impressive multilingual and text generation capabilities of the original BLOOM model, while offering a more memory-efficient version that is easier to deploy and fine-tune. It can be used to generate coherent text in 46 languages and 13 programming languages. What can I use it for? The bloom-8bit model is well-suited for a variety of natural language processing tasks, such as text generation, question answering, and language modeling. Given its smaller memory footprint, it can be particularly useful for deploying language models in resource-constrained environments or for fine-tuning on specific tasks. Things to try One interesting thing to try with the bloom-8bit model is fine-tuning it on domain-specific datasets to adapt it for specialized use cases. The provided fine-tuning example notebook demonstrates how to fine-tune the model on a custom dataset using a 3x NVIDIA A100 instance.

Read more

Updated 5/17/2024