Martin Evans
|
a45d9089e1
|
Fixed demos
|
2 years ago |
Martin Evans
|
a9e6f21ab8
|
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
|
2 years ago |
Martin Evans
|
ae8ef17a4a
|
- Added various convenience overloads to `LLamaContext.Eval`
- Converted `SafeLLamaContextHandle` to take a `ReadOnlySpan` for Eval, narrower type better represents what's really needed
|
2 years ago |
Martin Evans
|
02a46fc363
|
Updated demos to use the new loading/multi context system
|
2 years ago |
Martin Evans
|
e2fe08a9a2
|
Added a higher level `LLamaWeights` wrapper around `SafeLlamaModelHandle`
|
2 years ago |
Martin Evans
|
f31bdf6b93
|
Using the right context for Bob
|
2 years ago |
Martin Evans
|
f3511e390f
|
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
The biggest single change is renaming `LLamaModel` to `LLamaContext`
|
2 years ago |
Yaohui Liu
|
1062fe1a7e
|
feat: upgrade the native libraries.
|
2 years ago |
Yaohui Liu
|
eed96248b5
|
docs: add the documentations with mkdocs.
|
2 years ago |
Yaohui Liu
|
2eb2d6df83
|
test: add 9 examples of the new version.
|
2 years ago |
Yaohui Liu
|
bdbd6aa824
|
feat: add transforms for chat session.
|
2 years ago |
Yaohui Liu
|
3bf74ec9b9
|
feat: add chat session for refactored code.
|
3 years ago |
Yaohui Liu
|
908b79e855
|
feat: add stateless executor.
|
3 years ago |
Yaohui Liu
|
e603a09137
|
fix: state loading and saving not working.
|
3 years ago |