Rinne
c5e8b3eba2
Merge pull request #56 from martindevans/memory_mapped_save_loading_and_saving
Memory Mapped LoadState/SaveState
3 years ago
Rinne
d17fa991cc
Merge pull request #53 from martindevans/xml_docs_fixes
XML docs fixes
3 years ago
Rinne
1b0523f630
Merge branch 'master' into master
3 years ago
Martin Evans
4d72420a04
Replaced `SaveState` and `LoadState` implementations. These new implementations map the file into memory and then pass the pointer directly into the native API. This improves things in two ways:
- A C# array cannot exceed 2,147,483,591 bytes. In my own use of LlamaSharp I encountered this limit.
- This saves an extra copy of the entire state data into a C# `byte[]`, so it should be faster.
This does _not_ fix some other places where `GetStateData` is used. I'll look at those in a separate PR.
3 years ago
Martin Evans
2e76b79af6
Various minor XML docs fixes
3 years ago
SignalRT
56a37a0d7d
Update to lates llama.cpp
Adapt the interface change in llama_backend_init
3 years ago
unknown
dba866ffcf
Update API method name
3 years ago
Yaohui Liu
1062fe1a7e
feat: upgrade the native libraries.
3 years ago
Yaohui Liu
9850417a12
feat: update quantize native params.
3 years ago
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
3 years ago
Yaohui Liu
264fb9a706
refactor: LLamaModel and LLamaExecutor.
3 years ago
Yaohui Liu
3a62f087fe
fix: encoding error when using other languages.
3 years ago
Yaohui Liu
18c2ff2395
refactor: instruct mode and examples.
3 years ago
Yaohui Liu
55d5a8ae51
fix: quantization error with fp16.
3 years ago
Yaohui Liu
19979f664a
feat: support loading and saving state.
3 years ago
Yaohui Liu
00d91cf99e
refactor: some parts of code of LLamaModel.
3 years ago
Yaohui Liu
1fca06dc7f
fix: n_gpu_layers miss in llama context.
3 years ago
Yaohui Liu
4314f64b9c
feat: add check for backend package.
3 years ago
Yaohui Liu
6ffcb5306b
refactor: use official api of quantization instead.
3 years ago
Yaohui Liu
0958bbac2c
feat: add get-embedding api to LLamaModel.
3 years ago
Yaohui Liu
33067f990f
feat: run quantization in csharp.
3 years ago
Yaohui Liu
118d410d52
build: revise build informations.
3 years ago
Yaohui Liu
856d6549de
build: add linux support.
3 years ago
Yaohui Liu
02524ae4eb
build: add package informations.
3 years ago
Yaohui Liu
d6a7997e46
feat: add gpt model.
3 years ago
Yaohui Liu
5a79edeb51
feat: add the framework and basic usages.
3 years ago