User avatar
jade @karebu@social.karebu.gay
2mo
last time i tried to do home assistant tool calling it didn't work even with large cloud llms now its working with this 4gb gemma model
2
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu i set this up the other day too and im using e2b it works crazy good
1
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu cafe.autumn.town/notes/apiojnsednktpzs0
1
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn this setup is super fucked and slow
speech recognition running on a raspberry pi
gemma E4B running on a desktop gpu
charlie kirk tts running on my laptop home assistant won't stream it in real time for some reason
1
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn
1
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu LMAOOOOO IM DYINNGGG
1
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu honestly the way ive got it setup, whisper/piper on my main server (on CPU) and ollama on my desktop for the model, its like almost instant for everything. i wanna try do the voice thing so fucking bad tho 😭😭😭 what's that ui you're using btw?
2
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn github.com/jxlarrea/voice-satellite-card-integration


i switched just now from whisper to qwen3-asr running on llama.cpp on the same system running gemma and now it's instantaneous. still haven't figured out the tts issue though.
1
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn i could use gemma for speech recognition too but i assume it'll be way slower
1
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn it's also less accurate
1
0
1
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn llama.cpp currently doesn't strip the language detection tag so i have to do osmethgn about that... but its insanely fast
1
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu yeahhh llama.cpp is great for performance but i just like that ollama is out of the box no dicking about on stuff like H.A.
2
0
0
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu also i get like 180 t/s on it or something extremely stupid so im not fussed lmao, if you havent already then start using the gemma QAT models on hf they're great
0
0
0
0
User avatar
jade @karebu@social.karebu.gay
2mo
@autumn can't use ASR models with ollama doe
1
0
1
0
User avatar
plautumnpus @autumn@cafe.autumn.town
2mo
@karebu ohhh igy yeahh
0
0
0
0