Steven Kennedy
988f2fa302
Reverted Net8.0
2 years ago
Steven Kennedy
cf2e9e35f8
Updating the GpuLayerCount to mirror the Python Port of Llama.cpp
2 years ago
Martin Evans
9573e2c9ba
Merge pull request #417 from martindevans/safe_handle_initialisation
Safer Handle Initialisation
2 years ago
Martin Evans
1e69e265b6
Moved some native methods to do with creating/destroying resources into their respective handles. There is **no** safe way to call most of these methods, everything must be done through through handles.
2 years ago
Martin Evans
8416cb866a
Improved exceptions in IModelParams for unknown KV override types.
2 years ago
Martin Evans
48ef3bb080
Added runtime checks that UseMemoryLock and UseMemorymap are actually supported.
2 years ago
Martin Evans
4e5e994dda
- directly returning a SafeLlamaModelHandle, instead of an IntPtr which is wrapped in a handle.
- made `llama_backend_init` private. This is automatically called, there is no way it can correctly be used externally.
- made `llama_token_to_piece` safe (Span instead of pointer)
2 years ago
Martin Evans
bac3e43498
Fixed handling of empty spans
2 years ago
Martin Evans
3925545147
Fixed LLamaExecutorBase.cs
2 years ago
Martin Evans
c002642268
- Removed some `unsafe` where it wasn't necessary
- Wrapped some native functions which take (pointer, length) in function which take a `span` instead.
2 years ago
Martin Evans
f860f88c36
Code cleanup driven by R# suggestions:
- Made `NativeApi` into a `static class` (it's not intended to be instantiated)
- Moved `LLamaTokenType` enum out into a separate file
- Made `LLamaSeqId` and `LLamaPos` into `record struct`, convenient to have equality etc
2 years ago
Martin Evans
2cded1b296
Fixed alignment of value fields in `LLamaModelMetadataOverride`
2 years ago
Martin Evans
3fc0f34cbe
Fixed some issues which were causing metadata overrides not to work (mostly importantly, converting the key was failing so all keys were null bytes and thus ignored).
2 years ago
Martin Evans
6be3f62321
Fixed loading of very large metadata values (over 1kb)
2 years ago
Martin Evans
f0d7468b22
Merge pull request #356 from xbotter/deps/sk-rc3
bump sk to 1.0.1 & km to 0.18
2 years ago
xbotter
211ce12bf5
LLamaEmbedder exposes the Context
2 years ago
Martin Evans
fb606c2488
Fixed incorrect values
2 years ago
Martin Evans
47e4fcef2a
Fixed GetString on netstandard2
2 years ago
Martin Evans
2a1e1b6183
Removed unused imports
2 years ago
Martin Evans
a2bae178fa
Added a `Metadata` property to `LLamaWeights`
2 years ago
Martin Evans
7139281e4d
Merge pull request #373 from martindevans/enhanced_avx512
Improved support for AVX512
2 years ago
Martin Evans
1b13f7c717
Improved support for AVX512:
- Enabled more features in build process (VBMI and VNNI)
- Added runtime checking for this features
- Improved runtime checking to no longer require dotnet8.0
2 years ago
Martin Evans
82d84afaea
Resetting the custom sampling pipeline in the stateless executor
2 years ago
Martin Evans
c298ab828a
Merge pull request #368 from martindevans/context_set_seed
Context Set Seed
2 years ago
Martin Evans
dc8e5d88f7
Update LLama/LLamaContext.cs
2 years ago
Martin Evans
a3177ab140
Merge pull request #369 from martindevans/rename_llama_sample_temperature
Renamed `llama_sample_temperature` to `llama_sample_temp`
2 years ago
Martin Evans
db7ecf5a43
Added a method to create a clone of a grammar instance
2 years ago
Martin Evans
ea523d2e2a
Renamed `llama_sample_temperature` to `llama_sample_temp`, Mirroring the same change made in llama.cpp
2 years ago
Martin Evans
2df3e7617e
Added a method to set the RNG seed on the context
2 years ago
Martin Evans
cedef5e45a
Added the `pure` field to `LLamaModelQuantizeParams` (it's been added to llama.cpp)
2 years ago
Martin Evans
199b4418c3
Fixed JSON serialization for ModelParams
2 years ago
Martin Evans
2f0deeadcd
Implemented serialization for `MetadataOverride`. Deserialization is broken (converter is never called)
2 years ago
Martin Evans
b868b056f7
Added metadata overrides to `IModelParams`
2 years ago
xbotter
ce20b30e06
Merge branch 'SciSharp:master' into deps/sk-rc3
2 years ago
Martin Evans
b22d8b7495
- Added `GroupDisposable` to dispose a collection of items all together
- Renamed `LLamaModelKvOverride` to `LLamaModelMetadataOverride`
2 years ago
Martin Evans
5ad2cd1d3c
Added a comment on the type itself
2 years ago
Martin Evans
b0270b5788
Added comments on GGMLType
2 years ago
Martin Evans
b3e576608b
fixed safe handle
2 years ago
Martin Evans
bab6b65b61
Added a safe handle for LLamaKvCacheView
2 years ago
Martin Evans
439d14a061
Updated binaries:
- build run: https://github.com/SciSharp/LLamaSharp/actions/runs/7196891440
- commit: 9fb13f9584
2 years ago
xbotter
340bbbcf48
Move JSON converter for TensorSplitsCollection
2 years ago
Martin Evans
01c7f1b4da
Update LLama/Common/ModelParams.cs
2 years ago
Martin Evans
e6148c952e
Fixed encoding of `Encoding`
2 years ago
Martin Evans
d87d654a34
Merge pull request #348 from martindevans/new_object_based_sampling_pipeline
Custom Sampling Pipelines
2 years ago
xbotter
13a312b4ec
update sk to 1.0.0-rc3 & km to 0.18
2 years ago
Martin Evans
835958398c
- Removed the object wrappers and configurable pipeline, they can be better written in code.
- Added BaseSamplingPipeline which provides a base impl of `ISamplingPipeline`
- Added `DefaultSamplingPipeline` which mimics normal llama.cpp sampling
2 years ago
Martin Evans
3afc007499
- Added "protected" logits, instead of the awkward save/load mechanism
- Added an example usage to one of the tests
2 years ago
Martin Evans
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
2 years ago
Martin Evans
4fc743c9ba
Merge branch 'master' into master
2 years ago
Philipp Bauer
67e6d633fd
Rebuild ChatSession class
- Saves with serialized ChatHistory of session
- Only allows use of ChatHistory.Message (instead of raw text)
for easy post-processing with IHistoryTransform implementation
- Provides History Management methods
- Allows user to regenerate last assistant message
2 years ago