Martin Evans
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
MinP Sampler
2 years ago
Rinne
da6718c387
docs: adjust some descriptions.
2 years ago
Yaohui Liu
d7675f7936
Merge branch 'master' of github.com:AsakusaRinne/LLamaSharp into cuda_detection
2 years ago
Martin Evans
d743516070
- Added support for the MinP sampler
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2 years ago
Yaohui Liu
cb5fb210b1
feat: optimize apis for cuda feature detection.
2 years ago
SignalRT
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2 years ago
Yaohui Liu
bbbfbd20b5
fix: cannot load library under some conditions.
2 years ago
Martin Evans
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2 years ago
SignalRT
7691f83516
Test build and nuget packages
2 years ago
Yaohui Liu
d03e1dbe30
feat: support cuda feature detection.
2 years ago
SignalRT
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
This reverts commit 091b8d58b3502a99b3bfbec9db457c92cc736beb, reversing
changes made to 9b2ca9cf8e .
2 years ago
SignalRT
200011e186
Revert "Merge feat: add detection template for cuda and avx. #268"
This reverts commit b4b3ea9d99 .
2 years ago
SignalRT
b4b3ea9d99
Merge feat: add detection template for cuda and avx. #268
Just merge cuda and avx detection and change layout.
2 years ago
Yaohui Liu
b893c6f609
feat: add detection template for cuda and avx.
2 years ago
Martin Evans
db1bc741b0
Modified `ContextSize` in parameters to be nullable. A null value means autodetect from the model.
2 years ago
Martin Evans
04ee64a6be
Exposed YaRN scaling parameters in IContextParams
2 years ago
SignalRT
46fb472d42
Align with llama.cpp b1488
2 years ago
Martin Evans
a03fdc4818
Using a reference to an array instead of pointer arithmetic. This means it will benefit from bounds checking on the array.
2 years ago
Martin Evans
08c29d52c5
Slightly refactored `SafeLLamaGrammarHandle.Create` to solve CodeQL warning about pointer arithmetic.
2 years ago
Martin Evans
b6d242193e
Debugging slowdown by removing some things:
- Removed all `record struct` uses in native code
- Removed usage of `readonly` in native structs
Minor fix:
- Added sequential layout to `LLamaModelQuantizeParams`
2 years ago
Martin Evans
51c292ebd8
Added a safe method for `llama_get_logits_ith`
2 years ago
Martin Evans
7e3cde4c13
Moved helper methods into `LLamaBatchSafeHandle`
2 years ago
Martin Evans
c7fdb9712c
Added binaries, built from ` 6961c4bd0b`
2 years ago
Martin Evans
e81b3023d5
Rewritten sampling API to be accessed through the `LLamaTokenDataArray` object
2 years ago
Martin Evans
3c5547b2b7
Reduced some uses of `NativeApi` in `BatchedDecoding` by adding some helper methods
2 years ago
Martin Evans
a024d2242e
It works!
had to update binary to `b1426`
2 years ago
Martin Evans
8cd81251b4
initial setup
2 years ago
Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
a03fe003de
Fixed decoding of text "accumulating" over time (never properly clearing buffer)
2 years ago
Martin Evans
51d4411a58
Added two new classes for detokenization tasks:
- `AntipromptProcessor` accepts chunks of text and returns a value indicating if any antiprompt has been detected.
- `StreamingTokenDecoder` decodes tokens into text, maintaining some internal state to handle single characters which are encoded as multiple tokens.
Added tests for these classes and updated StatelessExecutor to use them.
Removed most DeTokenize methods, marked the rest as obsolete (should always use a `StreamingTokenDecoder`).
2 years ago
Martin Evans
efdf3d630c
- Removed all `TokenToString` methods (it's never correct to use them, because sometimes one single character may be represented by multiple tokens).
- Built a new (hacky) `Detokenize` method which handles this
2 years ago
Martin Evans
1d0620e634
Created a test that "roundtrips" strings through tokenization. This reveals some flaws with certain characters
2 years ago
Martin Evans
04acbf8c42
Improved doc comment on `tensor_split`
2 years ago
Martin Evans
15db194c17
Added multi GPU support
2 years ago
Martin Evans
e89ca5cc17
Fixed a few minor warnings
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
1f8c94e386
Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538 )
2 years ago
Martin Evans
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
- Replaced all binaries
2 years ago
Martin Evans
9a0a0ae9fe
Removed cloning support
2 years ago
Martin Evans
0d40338692
Fixed out-of-context handling in stateless executor
2 years ago
Martin Evans
b306ac23dd
Added `Decode` method to `SafeLLamaContextHandle`
2 years ago
Martin Evans
9e958e896b
safe handle for batch
2 years ago
Martin Evans
ce1fc51163
Added some more native methods
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
Haiping
10678a83d6
Merge pull request #65 from martindevans/alternative_dependency_loading
CPU Feature Detection
2 years ago
Martin Evans
daf09eae64
Skipping tokenization of empty strings (saves allocating an empty array every time)
2 years ago
Martin Evans
bba801f4b7
Added a property to get the KV cache size from a context
2 years ago
sa_ddam213
09d8f434f2
Extract LLamaLogLevel, Remove Logger class
2 years ago
Martin Evans
d3b8ee988c
Beam Search ( #155 )
* Added the low level bindings to beam search.
2 years ago
Martin Evans
614ba40948
- Added a `TokensEndsWithAnyString` extension to `IReadOnlyList<int>` which efficiently checks if a set of tokens ends with one of a set of strings.
- Minimal amount of characters converted
- Allocation free
- Added `TokensToSpan` to `SafeLlamaModelHandle` which converts as many tokens as possible into a character span
- Allocation free
2 years ago