Falcon 7b Sharded

vilsonrodrigues

falcon-7b-sharded

Resharded Resharded version of https://huggingface.co/tiiuae/falcon-7b for low RAM enviroments (e.g. Colab, Kaggle) in safetensors πŸš€ Falcon-7B Falcon-7B is a 7B parameters causal decoder-only model built by TII and trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. It is made available under the Apache 2.0 license. Paper coming soon 😊. πŸ€— To get started with Falcon (inference, finetuning, quantization, etc.), we recommend reading this great blogpost fron HF! Why use Falcon-7B? It outperforms comparable open-source models (e.g., MPT-7B, StableLM, RedPajama etc.), thanks to being trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. See the OpenLLM Leaderboard. It features an architecture optimized for inference, with FlashAttention (Dao et al., 2022) and multiquery (Shazeer et al., 2019). It is made available under a permissive Apache 2.0 license allowing for commercial use, without any royalties or restrictions. ⚠️ Falcon is now available as a core model in the transformers library! To use the in-library version, please install the latest version of transformers with pip install git+https://github.com/ huggingface/transformers.git, then simply remove the trust_remote_code=True argument from from_pretrained(). ⚠️ This is a raw, pretrained model, which should be further finetuned for most usecases. If you are looking for a version better suited to taking generic instructions in a chat format, we recommend taking a look at Falcon-7B-Instruct. πŸ”₯ Looking for an even more powerful model? Falcon-40B is Falcon-7B's big brother! πŸ’₯ Falcon LLMs require PyTorch 2.0 for use with transformers! For fast inference with Falcon, check-out Text Generation Inference! Read more in this blogpost. You will need at least 16GB of memory to swiftly run inference with Falcon-7B. Model Card for Falcon-7B Model Details Model Description Developed by: https://www.tii.ae; Model type: Causal decoder-only; Language(s) (NLP): English and French; License: Apache 2.0. Model Source Paper: coming soon. Uses Direct Use Research on large language models; as a foundation for further specialization and finetuning for specific usecases (e.g., summarization, text generation, chatbot, etc.) Out-of-Scope Use Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful. Bias, Risks, and Limitations Falcon-7B is trained on English and French data only, and will not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it will carry the stereotypes and biases commonly encountered online. Recommendations We recommend users of Falcon-7B to consider finetuning it for the specific set of tasks of interest, and for guardrails and appropriate precautions to be taken for any production use. How to Get Started with the Model Training Details Training Data Falcon-7B was trained on 1,500B tokens of RefinedWeb, a high-quality filtered and deduplicated web dataset which we enhanced with curated corpora. Significant components from our curated copora were inspired by The Pile (Gao et al., 2020). The data was tokenized with the Falcon-7B/40B tokenizer. Training Procedure Falcon-7B was trained on 384 A100 40GB GPUs, using a 2D parallelism strategy (PP=2, DP=192) combined with ZeRO. Training happened in early March 2023 and took about two weeks. Evaluation Paper coming soon. See the OpenLLM Leaderboard for early results. Technical Specifications Model Architecture and Objective Falcon-7B is a causal decoder-only model trained on a causal language modeling task (i.e., predict the next token). The architecture is broadly adapted from the GPT-3 paper (Brown et al., 2020), with the following differences: Positionnal embeddings: rotary (Su et al., 2021); Attention: multiquery (Shazeer et al., 2019) and FlashAttention (Dao et al., 2022); Decoder-block: parallel attention/MLP with a single layer norm. Compute Infrastructure Falcon-7B was trained on AWS SageMaker, on 384 A100 40GB GPUs in P4d instances. Falcon-7B was trained a custom distributed training codebase, Gigatron. It uses a 3D parallelism approach combined with ZeRO and high-performance Triton kernels (FlashAttention, etc.) Citation Paper coming soon 😊. In the meanwhile, you can use the following information to cite: To learn more about the pretraining dataset, see the πŸ““ RefinedWeb paper. License Falcon-7B is made available under the Apache 2.0 license. Contact falconllm@tii.ae
text-generation

Pricing

Cost per run
$-
USD
Avg run time
-
Seconds
Hardware
-
Prediction

Creator Models

ModelCostRuns
Falcon 7b Instruct Sharded$?18,177

Similar Models

Try it!

You can use this area to play around with demo applications that incorporate the Falcon 7b Sharded model. These demos are maintained and hosted externally by third-party creators. If you see an error, message me on Twitter.

Currently, there are no demos available for this model.

Overview

Summary of this model and related resources.

PropertyValue
Creatorvilsonrodrigues
Model NameFalcon 7b Sharded
Description

Resharded Resharded version of https://huggingface.co/tiiuae/falcon-7b for low RAM enviroments (e.g. Colab, Kaggle) in safetensors πŸš€ Falcon-7B Falcon-7B is a 7B parameters causal decoder-only model built by TII and trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. It is made available under the Apache 2.0 license. Paper coming soon 😊. πŸ€— To get started with Falcon (inference, finetuning, quantization, etc.), we recommend reading this great blogpost fron HF! Why use Falcon-7B? It outperforms comparable open-source models (e.g., MPT-7B, StableLM, RedPajama etc.), thanks to being trained on 1,500B tokens of RefinedWeb enhanced with curated corpora. See the OpenLLM Leaderboard. It features an architecture optimized for inference, with FlashAttention (Dao et al., 2022) and multiquery (Shazeer et al., 2019). It is made available under a permissive Apache 2.0 license allowing for commercial use, without any royalties or restrictions. ⚠️ Falcon is now available as a core model in the transformers library! To use the in-library version, please install the latest version of transformers with pip install git+https://github.com/ huggingface/transformers.git, then simply remove the trust_remote_code=True argument from from_pretrained(). ⚠️ This is a raw, pretrained model, which should be further finetuned for most usecases. If you are looking for a version better suited to taking generic instructions in a chat format, we recommend taking a look at Falcon-7B-Instruct. πŸ”₯ Looking for an even more powerful model? Falcon-40B is Falcon-7B's big brother! πŸ’₯ Falcon LLMs require PyTorch 2.0 for use with transformers! For fast inference with Falcon, check-out Text Generation Inference! Read more in this blogpost. You will need at least 16GB of memory to swiftly run inference with Falcon-7B. Model Card for Falcon-7B Model Details Model Description Developed by: https://www.tii.ae; Model type: Causal decoder-only; Language(s) (NLP): English and French; License: Apache 2.0. Model Source Paper: coming soon. Uses Direct Use Research on large language models; as a foundation for further specialization and finetuning for specific usecases (e.g., summarization, text generation, chatbot, etc.) Out-of-Scope Use Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful. Bias, Risks, and Limitations Falcon-7B is trained on English and French data only, and will not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it will carry the stereotypes and biases commonly encountered online. Recommendations We recommend users of Falcon-7B to consider finetuning it for the specific set of tasks of interest, and for guardrails and appropriate precautions to be taken for any production use. How to Get Started with the Model Training Details Training Data Falcon-7B was trained on 1,500B tokens of RefinedWeb, a high-quality filtered and deduplicated web dataset which we enhanced with curated corpora. Significant components from our curated copora were inspired by The Pile (Gao et al., 2020). The data was tokenized with the Falcon-7B/40B tokenizer. Training Procedure Falcon-7B was trained on 384 A100 40GB GPUs, using a 2D parallelism strategy (PP=2, DP=192) combined with ZeRO. Training happened in early March 2023 and took about two weeks. Evaluation Paper coming soon. See the OpenLLM Leaderboard for early results. Technical Specifications Model Architecture and Objective Falcon-7B is a causal decoder-only model trained on a causal language modeling task (i.e., predict the next token). The architecture is broadly adapted from the GPT-3 paper (Brown et al., 2020), with the following differences: Positionnal embeddings: rotary (Su et al., 2021); Attention: multiquery (Shazeer et al., 2019) and FlashAttention (Dao et al., 2022); Decoder-block: parallel attention/MLP with a single layer norm. Compute Infrastructure Falcon-7B was trained on AWS SageMaker, on 384 A100 40GB GPUs in P4d instances. Falcon-7B was trained a custom distributed training codebase, Gigatron. It uses a 3D parallelism approach combined with ZeRO and high-performance Triton kernels (FlashAttention, etc.) Citation Paper coming soon 😊. In the meanwhile, you can use the following information to cite: To learn more about the pretraining dataset, see the πŸ““ RefinedWeb paper. License Falcon-7B is made available under the Apache 2.0 license. Contact falconllm@tii.ae

Read more Β»
Tagstext-generation
Model LinkView on HuggingFace
API SpecView on HuggingFace
Github LinkNo Github link provided
Paper LinkNo paper link provided

Popularity

How popular is this model, by number of runs? How popular is the creator, by the sum of all their runs?

PropertyValue
Runs341
Model Rank
Creator Rank

Cost

How much does it cost to run this model? How long, on average, does it take to complete a run?

PropertyValue
Cost per Run$-
Prediction Hardware-
Average Completion Time-