Here is another inference implementation in Python (only dependency is PyTorch)....

		xml on Jan 6, 2024 \| parent \| context \| favorite \| on: TinyLlama: An Open-Source Small Language Model Here is another inference implementation in Python (only dependency is PyTorch). https://github.com/99991/SimpleTinyLlama The new checkpoints did not seem much better and they changed the chat format for some reason, so I did not port the new checkpoints yet. Perhaps I'll get to it this weekend.