GitHub - huggingface/candle: Minimalist ML framework for Rust
github.comMinimalist ML framework for Rust. Contribute to huggingface/candle development by creating an account on GitHub.
Out of the box models: Llama, Whisper, Falcon, StarCoder, etc, WASM support (run in browser), model training, and much much more, woo-hoo!
You must log in or register to comment.
- 0 users online
- 2 users / day
- 2 users / week
- 2 users / month
- 74 users / 6 months
- 1 subscriber
- 258 Posts
- 707 Comments
- Modlog
i was able to run the llama demo on my phone, how cool is that!
Very cool! In my company I had to serve BERT in rust using the tensorflow C API. If I knew of this framework I would have given it a shot, the examples are there and they look easy to understand.
I can’t believe I’ll get excited about phone specs again 🙌🏻✨. Do you think it could be possible to parallelize computation among various phones to run inference on transformer models? I assume is not worth it since you would need to transfer a ton of data among devices to run attention per layer, but the llama people have pulled so many tricks at this point…