LLamaSharp

333 MB

Tree: 9bc129e252

Author	SHA1	Message	Date
Martin Evans	b0acecf080	Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). Conversations can be "forked", to create a copy of a conversation at a given point. This allows e.g. prompting a conversation with a system prefix just once and then forking it again and again for each individual conversation. Conversations can also be "rewound" to an earlier state. Added two new examples, demonstrating forking and rewinding.	2 years ago
Martin Evans	2eb52b1630	made casts to/from int explicit, fixed places affected	2 years ago
Martin Evans	42be9b136d	Switched form using raw integers, to a `LLamaToken` struct	2 years ago
Martin Evans	835958398c	- Removed the object wrappers and configurable pipeline, they can be better written in code. - Added BaseSamplingPipeline which provides a base impl of `ISamplingPipeline` - Added `DefaultSamplingPipeline` which mimics normal llama.cpp sampling	2 years ago