Moreh

Models by this creator

🛸

MoMo-72B-LoRA-V1.4

moreh

Total Score

87

MoMo-72B-LoRA-V1.4 is a large language model trained using Supervised Fine-Tuning (SFT) with LoRA on the QWEN-72B base model. It was developed by Moreh using their MoAI platform and AMD's MI250 GPU. The model was trained on the Open-Orca/SlimOrca dataset, with no other datasets or benchmark test sets used. Similar models include MoMo-72B-lora-1.8.7-DPO, which was further fine-tuned using Direct Preference Optimization (DPO), and ALMA-13B-R, which leverages Contrastive Preference Optimization (CPO) for improved performance. Model inputs and outputs Inputs Text prompts for language generation tasks Outputs Generated text continuations based on the input prompts Capabilities MoMo-72B-LoRA-V1.4 is a large language model capable of a variety of text-generation tasks, such as summarization, question answering, and open-ended dialogue. It has shown strong performance on benchmarks like ARC, MMLU, and TruthfulQA, with results above 70% in some cases. What can I use it for? MoMo-72B-LoRA-V1.4 can be used for a wide range of natural language processing applications, from chatbots and virtual assistants to content generation and text summarization. Its large scale and strong performance make it a versatile tool for businesses and researchers looking to leverage the capabilities of advanced language models. Things to try Consider prompting the model with instructions that require step-by-step reasoning or logical deduction, as it may excel at tasks that benefit from its large scale and training on diverse datasets. You could also experiment with different prompt strategies, such as using the suggestion suffixes mentioned in the description of the SuperCOT-LoRA model, to see how they impact the model's outputs.

Read more

Updated 5/23/2024

🎲

MoMo-72B-lora-1.8.7-DPO

moreh

Total Score

67

The MoMo-72B-lora-1.8.7-DPO model is a large language model trained by Moreh using Direct Preference Optimization (DPO) from the MoMo-72B-LoRA-V1.4 base model. It was trained on the slimorca, truthy, and orca_dpo_pairs datasets, using Moreh's MoAI platform and AMD's MI250 GPU. The model is an optimization over previous versions, with improvements in hyperparameters and performance on benchmarks like TruthfulQA and GSM8K. It was trained to be a capable text-to-text model, able to assist users with a variety of tasks. Model inputs and outputs Inputs Natural language text prompts Outputs Coherent, relevant text responses to input prompts Capabilities The MoMo-72B-lora-1.8.7-DPO model can be used for a wide range of text-to-text tasks, such as question answering, language generation, and text summarization. It has shown strong performance on benchmarks like TruthfulQA and GSM8K, indicating its ability to provide truthful and informative responses. What can I use it for? The MoMo-72B-lora-1.8.7-DPO model can be a valuable tool for various applications that require natural language processing, such as customer service chatbots, content generation, and research assistance. Its large size and strong performance make it a suitable choice for companies or individuals looking to incorporate powerful language models into their products or workflows. Things to try One interesting aspect of the MoMo-72B-lora-1.8.7-DPO model is its use of Direct Preference Optimization (DPO) during training. This technique aims to directly optimize the model's outputs to be more truthful and informative, which is evident in its strong performance on benchmarks like TruthfulQA. Users could experiment with prompts that require the model to provide truthful and insightful responses, to further explore the benefits of the DPO training process.

Read more

Updated 5/28/2024