> this is a “holy shit” moment for Rust in AI applications
Yeah because I realized the 2MB is just a wrapper that reads stdin and offloads everything to wasi-nn API.
> The core Rust source code is very simple. It is only 40 lines of code. The Rust program manages the user input, tracks the conversation history, transforms the text into the llama2’s chat template, and runs the inference operations using the WASI NN API.
You can do the same using Python with fewer lines of code and maybe smaller executable size.
Yeah because I realized the 2MB is just a wrapper that reads stdin and offloads everything to wasi-nn API.
> The core Rust source code is very simple. It is only 40 lines of code. The Rust program manages the user input, tracks the conversation history, transforms the text into the llama2’s chat template, and runs the inference operations using the WASI NN API.
You can do the same using Python with fewer lines of code and maybe smaller executable size.