AI model creator details for sultan

BioM-ELECTRA-Large-SQuAD2

The BioM-ELECTRA-Large-SQuAD2 model is a large-scale language model trained on biomedical text. It is specifically trained for question answering tasks and is based on the ELECTRA architecture. The model is fine-tuned on the SQuAD 2.0 dataset, which contains questions related to various topics in the biomedical domain. By inputting a question and a passage of text, the model can generate an answer that is relevant to the question based on its understanding of the passage.

$-/run

60.7K

Huggingface

ArabicT5-Large

sultan

ArabicT5: Efficient Adaptation of T5 on Arabic Language Model Description This model adapts T5 on the Arabic Language by pre-training T5 on : Arabic Wikipedia. Marefa encyclopedia. Hindawi Books. a collection of Arabic News. Total Corpora size is 17GB. We restrict our corpora to News and Encyclopedias to enhance the performance of the model on informative tasks such as Factoid Question Answering and Generative task that uses classic Arabic ( الفصحى ). This also gives our models an advantage if you don't want the generative text to contain inappropriate language. This model uses an efficient implementation of T5 which reduces the fine-tuning and memory used Link . Pre-training Settings and Results on TyDi QA Development Dataset ( Model in this card is highlighted in bold ) Results on TyDi QA, HARD, Sentiment Analysis, Sarcasm Detection ( Best Score is highlighted in bold ) Evaluation Metrics: TyDi QA (EM/F1), HARD (Accuracy), Sentiment Analysis (Accuracy / F1-PN positive-negative), Sarcasm Detection (F1-sarcastic), XL-SUM (Rouge-L with Stemmer). You can download the full details of our grid search for all models in all tasks above from this link: https://github.com/salrowili/ArabicT5/raw/main/ArabicT5_Grid_Search.zip For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function. In our XL-Sum results, although we show that AraT5-Base exceeded our ArabicT5-Large, in most runs, our ArabicT5-Large shows better results, as you can see from our grid search file. Speedup Results Below are our speedup results on the TyDi QA dataset, where all models have fine-tuned 13 epochs with a learning rate of 2e-4 and batch size of 3 on each device on the TPU (TPU3v-8 batch=3x8->24). Please note these results when we fixed our hyperparameters for all models. Refer to the table above to get the best results after doing a grid search. Please note that we can further speed up our ArabicT5-Base by increasing the batch size since it could handle larger batch size than other base-scale models due to its hidden layer size (512). Paper Generative Approach for Gender-Rewriting Task with ArabicT5 FineTuning our ArabicT5 model on generative and abstractive tasks with FLAX GitHub Page https://github.com/salrowili/ArabicT5 Acknowledgment We would like to acknowledge the support we have from The TPU Research Cloud (TRC) team to grant us access to TPUv3 units. Citation

$-/run

908

Huggingface

ArabicT5-Base

sultan

Platform did not provide a description for this model.

$-/run

906

Huggingface

ArabicT5-xLarge

sultan

ArabicT5: Efficient Adaptation of T5 on Arabic Language Model Description This model adapts T5 on the Arabic Language by pre-training T5 on : Arabic Wikipedia. Marefa encyclopedia. Hindawi Books. a collection of Arabic News. Total Corpora size is 17GB. We restrict our corpora to News and Encyclopedias to enhance the performance of the model on informative tasks such as Factoid Question Answering and Generative task that uses classic Arabic ( الفصحى ). This also gives our models an advantage if you don't want the generative text to contain inappropriate language. This model uses an efficient implementation of T5 which reduces the fine-tuning and memory used Link . Pre-training Settings and Results on TyDi QA Development Dataset ( Model in this card is highlighted in bold ) Results on TyDi QA, HARD, Sentiment Analysis, Sarcasm Detection ( Best Score is highlighted in bold ) Evaluation Metrics: TyDi QA (EM/F1), HARD (Accuracy), Sentiment Analysis (Accuracy / F1-PN positive-negative), Sarcasm Detection (F1-sarcastic), XL-SUM (Rouge-L with Stemmer). You can download the full details of our grid search for all models in all tasks above from this link: https://github.com/salrowili/ArabicT5/raw/main/ArabicT5_Grid_Search.zip For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function. In our XL-Sum results, although we show that AraT5-Base exceeded our ArabicT5-Large, in most runs, our ArabicT5-Large shows better results, as you can see from our grid search file. Speedup Results Below are our speedup results on the TyDi QA dataset, where all models have fine-tuned 13 epochs with a learning rate of 2e-4 and batch size of 3 on each device on the TPU (TPU3v-8 batch=3x8->24). Please note these results when we fixed our hyperparameters for all models. Refer to the table above to get the best results after doing a grid search. Please note that we can further speed up our ArabicT5-Base by increasing the batch size since it could handle larger batch size than other base-scale models due to its hidden layer size (512). Paper Generative Approach for Gender-Rewriting Task with ArabicT5 FineTuning our ArabicT5 model on generative and abstractive tasks with FLAX GitHub Page https://github.com/salrowili/ArabicT5 Acknowledgment We would like to acknowledge the support we have from The TPU Research Cloud (TRC) team to grant us access to TPUv3 units. Citation

$-/run

152

Huggingface

BioM-ELECTRA-Base-Discriminator

sultan

Platform did not provide a description for this model.

$-/run

121

Huggingface

BioM-ALBERT-xxlarge-SQuAD2