Heegyu

Rank:

Average Model Cost: $0.0000

Number of Runs: 8,692

Models by this creator

gpt2-emotion

gpt2-emotion

heegyu

No description available.

Read more

$-/run

5.0K

Huggingface

ajoublue-gpt2-medium

ajoublue-gpt2-medium

๋ชจ๋ธ ๊ตฌ์„ฑ GPT2(Flax, Pytorch) 24 Layers, 1024 hidden dim, 4096 intermediate, 16 heads, 51200 vocab size 1024 max_seq_len ํŒŒ๋ผ๋ฏธํ„ฐ ์ˆ˜: 355M ์„ฑ๋Šฅ ๋ฒค์น˜๋งˆํฌ ํ•™์Šต ํ™˜๊ฒฝ ๋ฐ ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ TPU V2-8 Learning Rate: 3e-4, Batch Size: 512(=64 accum x 8 devices), Scheduler: Linear, WarmUp: 1000 step Optimizer: AdamW(adam_beta1=0.9 adam_beta2=0.98, weight_decay=0.01) bfloat16 Training Steps: 43247 (3 epoch) ํ•™์Šต ํ† ํฐ ์ˆ˜: 21.11B (43247 * 512 * 1024seq / 1024^3) ํ•™์Šต ๊ธฐ๊ฐ„: 2023/1/30 ~ 2023/2/5(6์ผ 11์‹œ๊ฐ„ ์†Œ์š”) ํ•™์Šต ์ฝ”๋“œ: https://github.com/HeegyuKim/language-model ํ•™์Šต์— ์‚ฌ์šฉํ•œ ๋ฐ์ดํ„ฐ AIHub SNS ๋Œ€ํ™”(730MB) AIHub ๊ตฌ์–ด์ฒด(422MB) AIHub ๋„์„œ(1.6MB) AIHub ๋Œ€๊ทœ๋ชจ ์›น๋ฐ์ดํ„ฐ ๊ธฐ๋ฐ˜ ํ•œ๊ตญ์–ด ๋ง๋ญ‰์น˜(12GB) ํ•œ๊ตญ์–ด ์œ„ํ‚ค(867MB) ๋‚˜๋ฌด์œ„ํ‚ค(6.4GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๋ฉ”์‹ ์ € ๋Œ€ํ™”(21MB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ์ผ์ƒ๋Œ€ํ™” ๋ง๋ญ‰์น˜(23MB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๋ฌธ์–ด ๋ง๋ญ‰์น˜(3.2GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๊ตฌ์–ด ๋ง๋ญ‰์น˜(1.1GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ์‹ ๋ฌธ ๋ง๋ญ‰์น˜(~2022, 17GB) ์ฒญ์™€๋Œ€ ๊ตญ๋ฏผ์ฒญ์›(525MB) ๋ฐ์ดํ„ฐ์…‹ ํฌ๊ธฐ๋Š” ์ „์ฒ˜๋ฆฌํ•œ jsonlํŒŒ์ผ์„ ๊ธฐ์ค€์œผ๋กœ ํ•จ. ์ด ํ† ํฐ ์ˆ˜๋Š” ์•ฝ 7B์ž„ ์‚ฌ์šฉ ์˜ˆ์‹œ ๊ฒฐ๊ณผ ์ฃผ์˜์‚ฌํ•ญ ์ด ๋ชจ๋ธ์˜ ํ•™์Šต ๋ฐ์ดํ„ฐ๋Š” ๊ฐ์ข… ์ฐจ๋ณ„/ํ˜์˜ค ๋ฐ์ดํ„ฐ๊ฐ€ ํฌํ•จ๋์„ ์ˆ˜ ์žˆ์œผ๋ฉฐ, ๋ณ„๋„์˜ ์ œ๊ฑฐ์ž‘์—…์„ ์ง„ํ–‰ํ•˜์ง€ ์•Š์•˜์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ๋ชจ๋ธ์ด ์ƒ์„ฑํ•˜๋Š” ๋ฌธ์žฅ์— ํŠน์ • ์ธ๋ฌผ์ด๋‚˜ ์ธ์ข…, ์„ฑ๋ณ„, ์žฅ์• ์— ๋”ฐ๋ฅธ ์ฐจ๋ณ„/ํ˜์˜ค๋ฐœ์–ธ์„ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Read more

$-/run

156

Huggingface

ajoublue-gpt2-base

ajoublue-gpt2-base

๋ชจ๋ธ ๊ตฌ์„ฑ GPT2(Flax, Pytorch) 12 Layers, 768 hidden dim, 3072 intermediate, 12 heads, 51200 vocab size 1024 max_seq_len ํŒŒ๋ผ๋ฏธํ„ฐ ์ˆ˜: 125M ์„ฑ๋Šฅ ๋ฒค์น˜๋งˆํฌ ํ•™์Šต ํ™˜๊ฒฝ ๋ฐ ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ TPU V2-8 Learning Rate: 6e-4, Batch Size: 512(=64 accum x 8 devices), Scheduler: Linear, WarmUp: 1000 step Optimizer: AdamW(adam_beta1=0.9 adam_beta2=0.98, weight_decay=0.01) Training Steps: 43247 (3 epoch) ํ•™์Šต ํ† ํฐ ์ˆ˜: 21.11B (43247 * 512 * 1024seq / 1024^3) ํ•™์Šต ๊ธฐ๊ฐ„: 2023/1/17 ~ 2023/1/19 (2์ผ 6์‹œ๊ฐ„) ํ•™์Šต ์ฝ”๋“œ: https://github.com/HeegyuKim/language-model ํ•™์Šต์— ์‚ฌ์šฉํ•œ ๋ฐ์ดํ„ฐ AIHub SNS ๋Œ€ํ™”(730MB) AIHub ๊ตฌ์–ด์ฒด(422MB) AIHub ๋„์„œ(1.6MB) AIHub ๋Œ€๊ทœ๋ชจ ์›น๋ฐ์ดํ„ฐ ๊ธฐ๋ฐ˜ ํ•œ๊ตญ์–ด ๋ง๋ญ‰์น˜(12GB) ํ•œ๊ตญ์–ด ์œ„ํ‚ค(867MB) ๋‚˜๋ฌด์œ„ํ‚ค(6.4GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๋ฉ”์‹ ์ € ๋Œ€ํ™”(21MB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ์ผ์ƒ๋Œ€ํ™” ๋ง๋ญ‰์น˜(23MB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๋ฌธ์–ด ๋ง๋ญ‰์น˜(3.2GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ๊ตฌ์–ด ๋ง๋ญ‰์น˜(1.1GB) ๊ตญ๋ฆฝ๊ตญ์–ด์› ์‹ ๋ฌธ ๋ง๋ญ‰์น˜(~2022, 17GB) ์ฒญ์™€๋Œ€ ๊ตญ๋ฏผ์ฒญ์›(525MB) ๋ฐ์ดํ„ฐ์…‹ ํฌ๊ธฐ๋Š” ์ „์ฒ˜๋ฆฌํ•œ jsonlํŒŒ์ผ์„ ๊ธฐ์ค€์œผ๋กœ ํ•จ. ์ด ํ† ํฐ ์ˆ˜๋Š” ์•ฝ 7B์ž„ ์‚ฌ์šฉ ์˜ˆ์‹œ ๊ฒฐ๊ณผ ์ฃผ์˜์‚ฌํ•ญ ์ด ๋ชจ๋ธ์˜ ํ•™์Šต ๋ฐ์ดํ„ฐ๋Š” ๊ฐ์ข… ์ฐจ๋ณ„/ํ˜์˜ค ๋ฐ์ดํ„ฐ๊ฐ€ ํฌํ•จ๋์„ ์ˆ˜ ์žˆ์œผ๋ฉฐ, ๋ณ„๋„์˜ ์ œ๊ฑฐ์ž‘์—…์„ ์ง„ํ–‰ํ•˜์ง€ ์•Š์•˜์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ๋ชจ๋ธ์ด ์ƒ์„ฑํ•˜๋Š” ๋ฌธ์žฅ์— ํŠน์ • ์ธ๋ฌผ์ด๋‚˜ ์ธ์ข…, ์„ฑ๋ณ„, ์žฅ์• ์— ๋”ฐ๋ฅธ ์ฐจ๋ณ„/ํ˜์˜ค๋ฐœ์–ธ์„ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Read more

$-/run

41

Huggingface

Similar creators