Fnlp
Rank:Average Model Cost: $0.0000
Number of Runs: 37,046
Models by this creator
moss-moon-003-sft
moss-moon-003-sft
MOSS is an open-source conversational language model that has been fine-tuned on multi-turn dialogues and can follow instructions while refusing inappropriate requests. The moss-moon-003-sft model has been trained on approximately 1.1 million conversational data and can be used for various tasks such as search, calculation, and equation solving. The model has undergone supervised fine-tuning and is available in both 4-bit and 8-bit versions for reduced memory and computation cost. The model has limitations in generating incorrect or biased information and users are advised to carefully verify the outputs. Installation and usage instructions are provided, along with resources for fine-tuning the model. Future plans include improving reasoning abilities, truthfulness, safety, and incorporating multi-modal and personalized capabilities into the model. The code is licensed under Apache 2.0, the data is licensed under CC BY-NC 4.0, and the model weights are licensed under GNU AGPL 3.0. Commercial use of the models requires authorization.
$-/run
18.7K
Huggingface
bart-large-chinese
$-/run
4.6K
Huggingface
bart-base-chinese
bart-base-chinese
bart-base-chinese is a Chinese language variant of BART (Bidirectional and Auto-Regressive Transformers), a sequence-to-sequence model for text generation tasks. BART is pre-trained on a large corpus of text and can be fine-tuned for various downstream tasks such as summarization, translation, and text generation. It can generate high-quality and coherent Chinese text based on given input prompts.
$-/run
4.4K
Huggingface
moss-moon-003-sft-int4
moss-moon-003-sft-int4
MOSS Table of Contents Open-source list Models Data Engineering Solutions Introduction Chat with MOSS GPU Requirements Installation Try MOSS Fine-tuning MOSS Requirements Start Training Related Links Future Plans License :spiral_notepad: Open-source List Models moss-moon-003-base: The base language model of MOSS-003, which was initialized with CodeGen and further pre-trained on 100B Chinese tokens and 20B English tokens. The model has seen 700B tokens during pre-training and consumed ~6.67x1022 FLOPs in total. moss-moon-003-sft: We performed supervised fine-tuning on ~1.1M multi-turn conversational data. The fine-tuned model can follow instructions in multi-turn dialogues and refuse inappropriate requests. moss-moon-003-sft-plugin: We performed supervised fine-tuning on ~1.1M multi-turn conversational data and additional ~300K plugin-augmented data. The fine-tuned model is capable of using several tools including search engine, text-to-image, calculator, and equation solver. moss-moon-003-sft-int4: 4-bit version of moss-moon-003-sft, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-int8: 8-bit version of moss-moon-003-sft, which requires 24GB GPU memory to perform inference. moss-moon-003-sft-plugin-int4: 4-bit version of moss-moon-003-sft-plugin, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-plugin-int8: 8-bit version of moss-moon-003-sft-plugin, which requires 24GB GPU memory to perform inference. moss-moon-003-pm: The preference model (PM) trained on preference data collected using the responses of moss-moon-003-sft. Will be open-sourced in the near future. moss-moon-003: The final MOSS-003 model trained using moss-moon-003-pm, which demonstrated better factuality, safety, and more stable response quality. Will be open-sourced in the near future. moss-moon-003-plugin: The final MOSS-003-plugin model trained using moss-moon-003-pm, which poccessed stronger abilities in understanding user intents and using plugins. Will be open-sourced in the near future. Data moss-002-sft-data: The multi-turn conversational data used to train MOSS-002, covering helpfulness, honesty, and harmlessness. The data is consisting of 570K English and 590K Chinese conversations generated by text-davinci-003. moss-003-sft-data: The multi-turn conversational data used to train moss-moon-003-sft. The data is generated by gpt-3.5-turbo from a seed set of user prompts collected through our early deployed MOSS-002 API. In contrast to moss-002-sft-data, moss-003-sft-data is well-aligned with the real-world distribution of user intents, covering finer-grained categories and more diverse harmlessness-related data. The data consists of ~1.1M conversational data. Currently we open-sourced a small portion of it and will make public the full data in the near future. moss-003-sft-plugin-data: The plugin-augmented multi-turn conversational data, which is consisting of ~300K conversations in which the AI assistant uses four plugins (search engine, text-to-image, calculator, and equation solver) to generate responses. Currently we open-sourced a small portion of data and will make public the full data in the near future. moss-003-pm-data: The preference data used to train moss-moon-003-pm, including ~180K additional dialogue contexts and their corresponding responses generated by moss-moon-003-sft. Will be publicly available in the near future. Engineering Solutions MOSS Vortex - Solutions for MOSS model inference and deployment. MOSS WebSearchTool - Solutions for the web search plugin used by MOSS-003. MOSS Frontend - A flutter-based frontend used by MOSS-003. MOSS Backend - A Go-based backend used by MOSS-003. :fountain_pen: Introduction MOSS is an open-sourced plugin-augmented conversational language model. moss-moon models have 16B parameters, allowing users to perform inference on a single A100 GPU or 2 NVIDIA 3090 GPUs with FP16 precision, and on a single NVIDIA 3090 GPU with INT-4/8 precision. The base language model of MOSS was pre-trained on ~700B English, Chinese, and code tokens, including the PILE, BigQuery, BigPython, and our private Chinese corpus. The base model was then fine-tuned on multi-turn plugin-augmented conversational data. Finally, we performed preference-aware training to further improve the model. Limitations: Due to the (relatively) small number of parameters and the autoregressive nature, MOSS is still possible to generate outputs that contain incorrect, misleading, or biased information. Please carefully check the contents generated by MOSS before you use them. MOSS Use Cases： :robot: Chat with MOSS GPU Requirements The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that currently the quantized models do not support model parallism. Installation Clone this repo to your local/remote machine. Create a new conda environment Install requirements (Optional) 4/8-bit quantization requirement Note that the version of torch and transformers should be equal or higher than recommended. Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS. Try MOSS Below is an example of performing inference of moss-moon-003-sft, which can be executed on a single A100/A800 GPU or CPU with FP16 precision: You can also perform MOSS inference using the below code snippet on >=2 NVIDIA 3090 GPUs: Note: Currently our quantized models do not support model parallism. In the case of limited GPU memory, you can use the quantized MOSS models to reduce memory and computation cost. We used GPTQ and OpenAI triton backend (only supports Linux) to implement quantized inference. You can use moss-moon-003-sft-plugin and its quantized versions to use external plugins. The data format of a single turn interaction is as follows, in which "Human" is the user input and "Results" is the contents returned by the invoked plugins, so "Human" and "Results" should be written by the program, and the rest fields are generated by the model. Therefore we need to call two times of model inference: (1) at the first time the model generates until reaching <eoc>, we extract the predicted plugins (and their parameters) and obtain corresponding results by executing these plugins. (2) at the second time we write results returned by the used plugins into "Results" and feed the concatenated text into MOSS to get responses. At this time the model should generate until reaching <eom>. We control the use of the plugins through meta instruction. By default, the status of all the plugins is disabled. If you want to enable some plugins, first set the "Inner Thoughts" as enabled, and then change the status of the plugins to enabled and provide the interface. An example is as follows, Above is an example that enables web search and calculator. Please follow the API format below: Below shows a use case of search-augmented MOSS: We successfully obtained the plugin command Search("黑暗荣耀 主演"). Then we execute the search plugin and put the returned contents into "Results". The contents returned by the plugins should follow the format below: Then we concatenate the prefix and all the results we obtained so far and feed them into MOSS: The full data of this single-turn conversation is as follows: Please refer to conversation_with_plugins for data formats of other plugins. See also our open-sourced MOSS WebSearchTool for the web search plugin. Streamlit We provide a Streamlit-based web demo. First install Streamlit by pip install streamlit and then run moss_web_demo_streamlit.py in this repo to present a web demo: Gradio Thank Pull Request for providing a gradio-based web demo. You can try MOSS with a simple CLI demo by running moss_cli_demo.py: You can chat with MOSS in the demo. Clear dialogue history by typing clear and stop the demo by typing stop. :fire: Fine-tuning MOSS We also provided the Python code finetune_moss.py for fine-tuning MOSS base model. Requirements Start Training Here we show an example of fine-tuning moss-moon-003-base on conversational data without plugins. It would be straightforward to fine-tune it on plugin-augmented data. Step 1, prepare your data following the format in conversation_without_plugins and put it in the folder sft_data. Step 2, download the accelerate configs to your machine and modify it according to your compute configuration. Learn more on accelerate documentation. Step 3, create run.sh and copy the following snippet: Now you can start training: Note: In the tokenizer of moss-moon-003-base, the eos token is <|endoftext|>, your need to specify it as <eom> when performing supervised fine-tuning. :link: Related Links VideoChat with MOSS - Watch videos with MOSS! ModelWhale - A compute platform for deploying MOSS! If you have other open-sourced projects that used or improved MOSS, please feel free to submit Pull Requests to README or reach out to us in Issues. :construction: Future Plans We constantly improved the Chinese skills, honesty, harmlessness from MOSS-001 to MOSS-003, and enabled the model to use external plugins. However, MOSS-003 is still a very early version, and our journey has just begun. In the future, we will continue developing more advanced foundation models and open-sourcing more powerful MOSS. Reasoning: We are improving the reasoning abilities of MOSS by scaling up its base model and performing math-specific training. Truthfulness & Safety: We will reduce the hallucination of MOSS and improve its safety in the following versions. Multi-modal: Enabling the language model to see and to hear is a critical step towards general AI. We are working on integrating cross-modal abilities into MOSS. Personalized: Our expected MOSS should be personalized, it updates its knowledge during the interaction with users, and finally becomes an unique AI for each user. :page_with_curl: License The code in this repo is licensed by Apache 2.0, the data on huggingface and this repo are licensed by CC BY-NC 4.0, the model weights on huggingface are licensed by GNU AGPL 3.0. If you wish to use our models for commercial purpose or public serving, please sign this form and send it to robot@fudan.edu.cn to get authorized. We only track the commercial use but charge nothing. The service provider shall be responsible for misleading or injurious statements and adverse effects caused by the use of the models contained in this repo and their modified versions. :heart: Acknowledgement CodeGen: Our base language model is initialized with CodeGen-16B. Mosec: Model deployment and streaming responses. Shanghai AI Lab: GPU support. GPTQ/GPTQ-for-LLaMa: Quantization and inference backend.
$-/run
4.3K
Huggingface
moss-moon-003-sft-plugin-int4
$-/run
1.1K
Huggingface
moss-moon-003-base
moss-moon-003-base
MOSS Table of Contents Open-source list Models Data Engineering Solutions Introduction Chat with MOSS GPU Requirements Installation Try MOSS Fine-tuning MOSS Requirements Start Training Related Links Future Plans License :spiral_notepad: Open-source List Models moss-moon-003-base: The base language model of MOSS-003, which was initialized with CodeGen and further pre-trained on 100B Chinese tokens and 20B English tokens. The model has seen 700B tokens during pre-training and consumed ~6.67x1022 FLOPs in total. moss-moon-003-sft: We performed supervised fine-tuning on ~1.1M multi-turn conversational data. The fine-tuned model can follow instructions in multi-turn dialogues and refuse inappropriate requests. moss-moon-003-sft-plugin: We performed supervised fine-tuning on ~1.1M multi-turn conversational data and additional ~300K plugin-augmented data. The fine-tuned model is capable of using several tools including search engine, text-to-image, calculator, and equation solver. moss-moon-003-sft-int4: 4-bit version of moss-moon-003-sft, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-int8: 8-bit version of moss-moon-003-sft, which requires 24GB GPU memory to perform inference. moss-moon-003-sft-plugin-int4: 4-bit version of moss-moon-003-sft-plugin, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-plugin-int8: 8-bit version of moss-moon-003-sft-plugin, which requires 24GB GPU memory to perform inference. moss-moon-003-pm: The preference model (PM) trained on preference data collected using the responses of moss-moon-003-sft. Will be open-sourced in the near future. moss-moon-003: The final MOSS-003 model trained using moss-moon-003-pm, which demonstrated better factuality, safety, and more stable response quality. Will be open-sourced in the near future. moss-moon-003-plugin: The final MOSS-003-plugin model trained using moss-moon-003-pm, which poccessed stronger abilities in understanding user intents and using plugins. Will be open-sourced in the near future. Data moss-002-sft-data: The multi-turn conversational data used to train MOSS-002, covering helpfulness, honesty, and harmlessness. The data is consisting of 570K English and 590K Chinese conversations generated by text-davinci-003. moss-003-sft-data: The multi-turn conversational data used to train moss-moon-003-sft. The data is generated by gpt-3.5-turbo from a seed set of user prompts collected through our early deployed MOSS-002 API. In contrast to moss-002-sft-data, moss-003-sft-data is well-aligned with the real-world distribution of user intents, covering finer-grained categories and more diverse harmlessness-related data. The data consists of ~1.1M conversational data. Currently we open-sourced a small portion of it and will make public the full data in the near future. moss-003-sft-plugin-data: The plugin-augmented multi-turn conversational data, which is consisting of ~300K conversations in which the AI assistant uses four plugins (search engine, text-to-image, calculator, and equation solver) to generate responses. Currently we open-sourced a small portion of data and will make public the full data in the near future. moss-003-pm-data: The preference data used to train moss-moon-003-pm, including ~180K additional dialogue contexts and their corresponding responses generated by moss-moon-003-sft. Will be publicly available in the near future. Engineering Solutions MOSS Vortex - Solutions for MOSS model inference and deployment. MOSS WebSearchTool - Solutions for the web search plugin used by MOSS-003. MOSS Frontend - A flutter-based frontend used by MOSS-003. MOSS Backend - A Go-based backend used by MOSS-003. :fountain_pen: Introduction MOSS is an open-sourced plugin-augmented conversational language model. moss-moon models have 16B parameters, allowing users to perform inference on a single A100 GPU or 2 NVIDIA 3090 GPUs with FP16 precision, and on a single NVIDIA 3090 GPU with INT-4/8 precision. The base language model of MOSS was pre-trained on ~700B English, Chinese, and code tokens, including the PILE, BigQuery, BigPython, and our private Chinese corpus. The base model was then fine-tuned on multi-turn plugin-augmented conversational data. Finally, we performed preference-aware training to further improve the model. Limitations: Due to the (relatively) small number of parameters and the autoregressive nature, MOSS is still possible to generate outputs that contain incorrect, misleading, or biased information. Please carefully check the contents generated by MOSS before you use them. MOSS Use Cases： :robot: Chat with MOSS GPU Requirements The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that currently the quantized models do not support model parallism. Installation Clone this repo to your local/remote machine. Create a new conda environment Install requirements (Optional) 4/8-bit quantization requirement Note that the version of torch and transformers should be equal or higher than recommended. Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS. Try MOSS Below is an example of performing inference of moss-moon-003-sft, which can be executed on a single A100/A800 GPU or CPU with FP16 precision: You can also perform MOSS inference using the below code snippet on >=2 NVIDIA 3090 GPUs: Note: Currently our quantized models do not support model parallism. In the case of limited GPU memory, you can use the quantized MOSS models to reduce memory and computation cost. We used GPTQ and OpenAI triton backend (only supports Linux) to implement quantized inference. You can use moss-moon-003-sft-plugin and its quantized versions to use external plugins. The data format of a single turn interaction is as follows, in which "Human" is the user input and "Results" is the contents returned by the invoked plugins, so "Human" and "Results" should be written by the program, and the rest fields are generated by the model. Therefore we need to call two times of model inference: (1) at the first time the model generates until reaching <eoc>, we extract the predicted plugins (and their parameters) and obtain corresponding results by executing these plugins. (2) at the second time we write results returned by the used plugins into "Results" and feed the concatenated text into MOSS to get responses. At this time the model should generate until reaching <eom>. We control the use of the plugins through meta instruction. By default, the status of all the plugins is disabled. If you want to enable some plugins, first set the "Inner Thoughts" as enabled, and then change the status of the plugins to enabled and provide the interface. An example is as follows, Above is an example that enables web search and calculator. Please follow the API format below: Below shows a use case of search-augmented MOSS: We successfully obtained the plugin command Search("黑暗荣耀 主演"). Then we execute the search plugin and put the returned contents into "Results". The contents returned by the plugins should follow the format below: Then we concatenate the prefix and all the results we obtained so far and feed them into MOSS: The full data of this single-turn conversation is as follows: Please refer to conversation_with_plugins for data formats of other plugins. See also our open-sourced MOSS WebSearchTool for the web search plugin. Streamlit We provide a Streamlit-based web demo. First install Streamlit by pip install streamlit and then run moss_web_demo_streamlit.py in this repo to present a web demo: Gradio Thank Pull Request for providing a gradio-based web demo. You can try MOSS with a simple CLI demo by running moss_cli_demo.py: You can chat with MOSS in the demo. Clear dialogue history by typing clear and stop the demo by typing stop. :fire: Fine-tuning MOSS We also provided the Python code finetune_moss.py for fine-tuning MOSS base model. Requirements Start Training Here we show an example of fine-tuning moss-moon-003-base on conversational data without plugins. It would be straightforward to fine-tune it on plugin-augmented data. Step 1, prepare your data following the format in conversation_without_plugins and put it in the folder sft_data. Step 2, download the accelerate configs to your machine and modify it according to your compute configuration. Learn more on accelerate documentation. Step 3, create run.sh and copy the following snippet: Now you can start training: Note: In the tokenizer of moss-moon-003-base, the eos token is <|endoftext|>, your need to specify it as <eom> when performing supervised fine-tuning. :link: Related Links VideoChat with MOSS - Watch videos with MOSS! ModelWhale - A compute platform for deploying MOSS! If you have other open-sourced projects that used or improved MOSS, please feel free to submit Pull Requests to README or reach out to us in Issues. :construction: Future Plans We constantly improved the Chinese skills, honesty, harmlessness from MOSS-001 to MOSS-003, and enabled the model to use external plugins. However, MOSS-003 is still a very early version, and our journey has just begun. In the future, we will continue developing more advanced foundation models and open-sourcing more powerful MOSS. Reasoning: We are improving the reasoning abilities of MOSS by scaling up its base model and performing math-specific training. Truthfulness & Safety: We will reduce the hallucination of MOSS and improve its safety in the following versions. Multi-modal: Enabling the language model to see and to hear is a critical step towards general AI. We are working on integrating cross-modal abilities into MOSS. Personalized: Our expected MOSS should be personalized, it updates its knowledge during the interaction with users, and finally becomes an unique AI for each user. :page_with_curl: License The code in this repo is licensed by Apache 2.0, the data on huggingface and this repo are licensed by CC BY-NC 4.0, the model weights on huggingface are licensed by GNU AGPL 3.0. If you wish to use our models for commercial purpose or public serving, please sign this form and send it to robot@fudan.edu.cn to get authorized. We only track the commercial use but charge nothing. The service provider shall be responsible for misleading or injurious statements and adverse effects caused by the use of the models contained in this repo and their modified versions. :heart: Acknowledgement CodeGen: Our base language model is initialized with CodeGen-16B. Mosec: Model deployment and streaming responses. Shanghai AI Lab: GPU support. GPTQ/GPTQ-for-LLaMa: Quantization and inference backend.
$-/run
1.1K
Huggingface
moss-moon-003-sft-plugin
moss-moon-003-sft-plugin
MOSS Table of Contents Open-source list Models Data Engineering Solutions Introduction Chat with MOSS GPU Requirements Installation Try MOSS Fine-tuning MOSS Requirements Start Training Related Links Future Plans License :spiral_notepad: Open-source List Models moss-moon-003-base: The base language model of MOSS-003, which was initialized with CodeGen and further pre-trained on 100B Chinese tokens and 20B English tokens. The model has seen 700B tokens during pre-training and consumed ~6.67x1022 FLOPs in total. moss-moon-003-sft: We performed supervised fine-tuning on ~1.1M multi-turn conversational data. The fine-tuned model can follow instructions in multi-turn dialogues and refuse inappropriate requests. moss-moon-003-sft-plugin: We performed supervised fine-tuning on ~1.1M multi-turn conversational data and additional ~300K plugin-augmented data. The fine-tuned model is capable of using several tools including search engine, text-to-image, calculator, and equation solver. moss-moon-003-sft-int4: 4-bit version of moss-moon-003-sft, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-int8: 8-bit version of moss-moon-003-sft, which requires 24GB GPU memory to perform inference. moss-moon-003-sft-plugin-int4: 4-bit version of moss-moon-003-sft-plugin, which requires 12GB GPU memory to perform inference. moss-moon-003-sft-plugin-int8: 8-bit version of moss-moon-003-sft-plugin, which requires 24GB GPU memory to perform inference. moss-moon-003-pm: The preference model (PM) trained on preference data collected using the responses of moss-moon-003-sft. Will be open-sourced in the near future. moss-moon-003: The final MOSS-003 model trained using moss-moon-003-pm, which demonstrated better factuality, safety, and more stable response quality. Will be open-sourced in the near future. moss-moon-003-plugin: The final MOSS-003-plugin model trained using moss-moon-003-pm, which poccessed stronger abilities in understanding user intents and using plugins. Will be open-sourced in the near future. Data moss-002-sft-data: The multi-turn conversational data used to train MOSS-002, covering helpfulness, honesty, and harmlessness. The data is consisting of 570K English and 590K Chinese conversations generated by text-davinci-003. moss-003-sft-data: The multi-turn conversational data used to train moss-moon-003-sft. The data is generated by gpt-3.5-turbo from a seed set of user prompts collected through our early deployed MOSS-002 API. In contrast to moss-002-sft-data, moss-003-sft-data is well-aligned with the real-world distribution of user intents, covering finer-grained categories and more diverse harmlessness-related data. The data consists of ~1.1M conversational data. Currently we open-sourced a small portion of it and will make public the full data in the near future. moss-003-sft-plugin-data: The plugin-augmented multi-turn conversational data, which is consisting of ~300K conversations in which the AI assistant uses four plugins (search engine, text-to-image, calculator, and equation solver) to generate responses. Currently we open-sourced a small portion of data and will make public the full data in the near future. moss-003-pm-data: The preference data used to train moss-moon-003-pm, including ~180K additional dialogue contexts and their corresponding responses generated by moss-moon-003-sft. Will be publicly available in the near future. Engineering Solutions MOSS Vortex - Solutions for MOSS model inference and deployment. MOSS WebSearchTool - Solutions for the web search plugin used by MOSS-003. MOSS Frontend - A flutter-based frontend used by MOSS-003. MOSS Backend - A Go-based backend used by MOSS-003. :fountain_pen: Introduction MOSS is an open-sourced plugin-augmented conversational language model. moss-moon models have 16B parameters, allowing users to perform inference on a single A100 GPU or 2 NVIDIA 3090 GPUs with FP16 precision, and on a single NVIDIA 3090 GPU with INT-4/8 precision. The base language model of MOSS was pre-trained on ~700B English, Chinese, and code tokens, including the PILE, BigQuery, BigPython, and our private Chinese corpus. The base model was then fine-tuned on multi-turn plugin-augmented conversational data. Finally, we performed preference-aware training to further improve the model. Limitations: Due to the (relatively) small number of parameters and the autoregressive nature, MOSS is still possible to generate outputs that contain incorrect, misleading, or biased information. Please carefully check the contents generated by MOSS before you use them. MOSS Use Cases： :robot: Chat with MOSS GPU Requirements The table below shows the minimal GPU memory required by performing MOSS inference when batch size is 1. Please note that currently the quantized models do not support model parallism. Installation Clone this repo to your local/remote machine. Create a new conda environment Install requirements (Optional) 4/8-bit quantization requirement Note that the version of torch and transformers should be equal or higher than recommended. Currently triton only supports Linux and WSL. Please wait for later updates if you are using Windows/MacOS. Try MOSS Below is an example of performing inference of moss-moon-003-sft, which can be executed on a single A100/A800 GPU or CPU with FP16 precision: You can also perform MOSS inference using the below code snippet on >=2 NVIDIA 3090 GPUs: Note: Currently our quantized models do not support model parallism. In the case of limited GPU memory, you can use the quantized MOSS models to reduce memory and computation cost. We used GPTQ and OpenAI triton backend (only supports Linux) to implement quantized inference. You can use moss-moon-003-sft-plugin and its quantized versions to use external plugins. The data format of a single turn interaction is as follows, in which "Human" is the user input and "Results" is the contents returned by the invoked plugins, so "Human" and "Results" should be written by the program, and the rest fields are generated by the model. Therefore we need to call two times of model inference: (1) at the first time the model generates until reaching <eoc>, we extract the predicted plugins (and their parameters) and obtain corresponding results by executing these plugins. (2) at the second time we write results returned by the used plugins into "Results" and feed the concatenated text into MOSS to get responses. At this time the model should generate until reaching <eom>. We control the use of the plugins through meta instruction. By default, the status of all the plugins is disabled. If you want to enable some plugins, first set the "Inner Thoughts" as enabled, and then change the status of the plugins to enabled and provide the interface. An example is as follows, Above is an example that enables web search and calculator. Please follow the API format below: Below shows a use case of search-augmented MOSS: We successfully obtained the plugin command Search("黑暗荣耀 主演"). Then we execute the search plugin and put the returned contents into "Results". The contents returned by the plugins should follow the format below: Then we concatenate the prefix and all the results we obtained so far and feed them into MOSS: The full data of this single-turn conversation is as follows: Please refer to conversation_with_plugins for data formats of other plugins. See also our open-sourced MOSS WebSearchTool for the web search plugin. Streamlit We provide a Streamlit-based web demo. First install Streamlit by pip install streamlit and then run moss_web_demo_streamlit.py in this repo to present a web demo: Gradio Thank Pull Request for providing a gradio-based web demo. You can try MOSS with a simple CLI demo by running moss_cli_demo.py: You can chat with MOSS in the demo. Clear dialogue history by typing clear and stop the demo by typing stop. :fire: Fine-tuning MOSS We also provided the Python code finetune_moss.py for fine-tuning MOSS base model. Requirements Start Training Here we show an example of fine-tuning moss-moon-003-base on conversational data without plugins. It would be straightforward to fine-tune it on plugin-augmented data. Step 1, prepare your data following the format in conversation_without_plugins and put it in the folder sft_data. Step 2, download the accelerate configs to your machine and modify it according to your compute configuration. Learn more on accelerate documentation. Step 3, create run.sh and copy the following snippet: Now you can start training: Note: In the tokenizer of moss-moon-003-base, the eos token is <|endoftext|>, your need to specify it as <eom> when performing supervised fine-tuning. :link: Related Links VideoChat with MOSS - Watch videos with MOSS! ModelWhale - A compute platform for deploying MOSS! If you have other open-sourced projects that used or improved MOSS, please feel free to submit Pull Requests to README or reach out to us in Issues. :construction: Future Plans We constantly improved the Chinese skills, honesty, harmlessness from MOSS-001 to MOSS-003, and enabled the model to use external plugins. However, MOSS-003 is still a very early version, and our journey has just begun. In the future, we will continue developing more advanced foundation models and open-sourcing more powerful MOSS. Reasoning: We are improving the reasoning abilities of MOSS by scaling up its base model and performing math-specific training. Truthfulness & Safety: We will reduce the hallucination of MOSS and improve its safety in the following versions. Multi-modal: Enabling the language model to see and to hear is a critical step towards general AI. We are working on integrating cross-modal abilities into MOSS. Personalized: Our expected MOSS should be personalized, it updates its knowledge during the interaction with users, and finally becomes an unique AI for each user. :page_with_curl: License The code in this repo is licensed by Apache 2.0, the data on huggingface and this repo are licensed by CC BY-NC 4.0, the model weights on huggingface are licensed by GNU AGPL 3.0. If you wish to use our models for commercial purpose or public serving, please sign this form and send it to robot@fudan.edu.cn to get authorized. We only track the commercial use but charge nothing. The service provider shall be responsible for misleading or injurious statements and adverse effects caused by the use of the models contained in this repo and their modified versions. :heart: Acknowledgement CodeGen: Our base language model is initialized with CodeGen-16B. Mosec: Model deployment and streaming responses. Shanghai AI Lab: GPU support. GPTQ/GPTQ-for-LLaMa: Quantization and inference backend.
$-/run
862
Huggingface
moss-moon-003-sft-int8
$-/run
687
Huggingface