2 Commits (91ca9d2732a178d4207a7a663fcc17b789ddc4ae)

Author SHA1 Message Date
  Scott W Harden 06ffe3ac95
LLama.Examples: improve model path prompt (#526) 2 years ago
  Martin Evans b0acecf080 Created a new `BatchedExecutor` which processes multiple "Conversations" in one single inference batch. This is faster, even when the conversations are unrelated, and is much faster if the conversations share some overlap (e.g. a common system prompt prefix). 2 years ago