Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
f6a472ae86
Setting the default seed to `0xFFFFFFFF` (no seed, randomised)
2 years ago
Martin Evans
36c71abcfb
Fixed `LLama.StreamingTokenDecoderLLamaLLama.StreamingTokenDecoderLLamaLLama.StreamingTokenDecoderLLama` spam in all executors except Stateless.
2 years ago
Martin Evans
5b6408b072
Merge pull request #205 from martindevans/roundtrip_tokenization_investigation
RoundTrip Tokenization Errors
2 years ago
Martin Evans
a03fe003de
Fixed decoding of text "accumulating" over time (never properly clearing buffer)
2 years ago
Martin Evans
51d4411a58
Added two new classes for detokenization tasks:
- `AntipromptProcessor` accepts chunks of text and returns a value indicating if any antiprompt has been detected.
- `StreamingTokenDecoder` decodes tokens into text, maintaining some internal state to handle single characters which are encoded as multiple tokens.
Added tests for these classes and updated StatelessExecutor to use them.
Removed most DeTokenize methods, marked the rest as obsolete (should always use a `StreamingTokenDecoder`).
2 years ago
Martin Evans
efdf3d630c
- Removed all `TokenToString` methods (it's never correct to use them, because sometimes one single character may be represented by multiple tokens).
- Built a new (hacky) `Detokenize` method which handles this
2 years ago
Rinne
231efe06f2
Update LLama/runtimes/build/LLamaSharp.Backend.Cpu.nuspec
Co-authored-by: Martin Evans <martindevans@gmail.com>
2 years ago
Rinne
ecf852c4e2
Update LLama/runtimes/build/LLamaSharp.Backend.MacMetal.nuspec
Co-authored-by: Martin Evans <martindevans@gmail.com>
2 years ago
Rinne
95669c2ea3
Update LLama/runtimes/build/LLamaSharp.Backend.Cuda12.nuspec
Co-authored-by: Martin Evans <martindevans@gmail.com>
2 years ago
Rinne
5eaebd68ba
Update LLama/runtimes/build/LLamaSharp.Backend.Cuda11.nuspec
Co-authored-by: Martin Evans <martindevans@gmail.com>
2 years ago
Rinne
6724b39713
Update LLama/runtimes/build/LLamaSharp.Backend.Cpu.nuspec
Co-authored-by: Martin Evans <martindevans@gmail.com>
2 years ago
Martin Evans
1d0620e634
Created a test that "roundtrips" strings through tokenization. This reveals some flaws with certain characters
2 years ago
Yaohui Liu
b7a7dc00b6
ci: fix typos.
2 years ago
Yaohui Liu
252992ec6e
ci: fix icon and typos.
2 years ago
Yaohui Liu
53eedf1428
ci: fix error.
2 years ago
Yaohui Liu
f9a98c6e23
ci: add auto release workflow.
2 years ago
Martin Evans
f621ec67e8
Fixed serialization
2 years ago
Martin Evans
768747c652
spelling
2 years ago
Martin Evans
b4e7f64e76
Added System.Text.Json serialization for `TensorSplitsCollectionConverter`
2 years ago
Martin Evans
281e58f059
Fixed default value
2 years ago
Martin Evans
04acbf8c42
Improved doc comment on `tensor_split`
2 years ago
Martin Evans
6a4cd506bd
Added a safe `TensorSplitsCollection` to the params which prevents incorrectly setting the `tensor_splits` collection
2 years ago
Martin Evans
15db194c17
Added multi GPU support
2 years ago
Martin Evans
328022b13d
Fixed merge conflicts
2 years ago
Martin Evans
7ec318aab5
Added logging to embedder too
2 years ago
Martin Evans
f1e5a8f995
- Passing the `ILogger` through to every call of `CreateContext`
- Passing `ILogger` into executors
2 years ago
sa_ddam213
4ec9aed47a
Revert LLamasSharp project changes
2 years ago
sa_ddam213
b4b4000342
Merge branch 'master' into upstream_master
# Conflicts:
# LLama.Web/Common/ModelOptions.cs
# LLama.Web/Services/ConnectionSessionService.cs
# LLama/LLamaStatelessExecutor.cs
# LLama/LLamaWeights.cs
2 years ago
Martin Evans
e89ca5cc17
Fixed a few minor warnings
2 years ago
Martin Evans
9daf586ba8
Assorted cleanup leftover after the huge change in the last PR (comments, syntax style, etc)
2 years ago
Martin Evans
d8434ea9d6
Merge pull request #185 from martindevans/wip_major_api_change
Major llama.cpp API Change
2 years ago
Martin Evans
1f8c94e386
Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538 )
2 years ago
Martin Evans
efb0664df0
- Added new binaries
- Fixed stateless executor out-of-context handling
- Fixed token tests
2 years ago
Martin Evans
b8f0eff080
- Added `GetCharCountImpl` tests, fixed handling of empty strings
- Added ifdef to remove `Deconstruct` extension on everything except `NETSTANDARD2_0`
2 years ago
Martin Evans
45118520fa
- Improved coverage of `GBNFGrammarParser` up to 96%
- Covered text transforms
- Removed unnecessary non-async transforms
2 years ago
Martin Evans
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
- Replaced all binaries
2 years ago
Martin Evans
4e9b1f8cdc
- Split extension methods into separate files
2 years ago
sa_ddam213
9b8de007dc
Propagate ILogger
2 years ago
Martin Evans
669ae47ef7
- Split parameters into two interfaces
- params contains a list of loras, instead of just one
2 years ago
Martin Evans
9a0a0ae9fe
Removed cloning support
2 years ago
Martin Evans
0d40338692
Fixed out-of-context handling in stateless executor
2 years ago
Martin Evans
b306ac23dd
Added `Decode` method to `SafeLLamaContextHandle`
2 years ago
Martin Evans
9e958e896b
safe handle for batch
2 years ago
Martin Evans
ce1fc51163
Added some more native methods
2 years ago
Martin Evans
bca55eace0
Initial changes to match the llama.cpp changes
2 years ago
Martin Evans
d58fcbbd13
Fixed antiprompt checking
2 years ago
Martin Evans
08f1615e60
- Converted LLamaStatelessExecutor to run `Exec` calls inside an awaited task. This unblocks async callers while the model is being evaluated.
- Added a "spinner" to the `StatelessModeExecute` demo, which spins while waiting for the next token (demonstrating that it's not blocked).
2 years ago
Martin Evans
fe54f6764f
- Added unit tests for extension methods
- Removed unused `AddRangeSpan` extension
2 years ago
Haiping
79fa74d59c
Merge pull request #177 from redthing1/fix/context-getstate
fix opaque GetState (fixes #176 )
2 years ago