5 Commits (a31391edd7ddd9d7e3e85d8848b037f2dcc9d674)

Author SHA1 Message Date
  Martin Evans 91a7967869
`ReadOnlySpan<float>` in ISamplingPipeline (#538) 2 years ago
  Martin Evans b0acecf080 Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). 2 years ago
  Martin Evans 2eb52b1630 made casts to/from int explicit, fixed places affected 2 years ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 2 years ago
  Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 2 years ago