User avatar
charlotte @halva@mk.absturztau.be
1y
thinking of finetuning a small language model on my posts and then putting it out on fedi specifically to annoy a certain type of guy
2
0
3
0
User avatar
charlotte @halva@mk.absturztau.be
1y
it's probably going to be pretty stupid though because i don't really have the hardware to finetune even a 1b model
1
0
1
0
User avatar
charlotte @halva@mk.absturztau.be
1y
actually some people say they've managed to squeeze in 4-7b parameter models into 12 Gb of VRAM for finetuning with QLoRA
1
0
1
0
User avatar
charlotte @halva@mk.absturztau.be
1y
maybe i can convince my friend to let me remotely play around with his 7900xtx for this lol,,,
1
0
1
0
User avatar
Two Hollywood Phonies @autumn@cafe.autumn.town
1y
@halva ive done finetuning on BERT which is like 440 Million and that was 40 minutes on 10k lines of data on a 6700XT with 12gb of VRAM. its doable but slow. anything beyond that is gonna be rough yeah. i think its even funnier to have a 0.5 bil parameter shit model try to write out a sentence like me tho 😭
🧡1
1
0
0
1
User avatar
charlotte @halva@mk.absturztau.be
1y
@autumn modern embedding models are quite decent at just Talking tbh
1
0
2
0
User avatar
Two Hollywood Phonies @autumn@cafe.autumn.town
1y
@halva yeah im sure it'd be a lot better than the markov now lol
🧡1
0
0
1
1