Martin Evans
dff308e31b
Added a test for the LLamaEmbedder
2 years ago
Martin Evans
d92e2664ac
Merge pull request #116 from zombieguy98/master
Changed type conversion
2 years ago
zombieguy
45b01d5a78
Improved type conversion
Type conversion is now done in the property rather than the utils class and uses the System.Convert class to ensure consistency.
2 years ago
Martin Evans
42fc5e4983
Merge pull request #114 from martindevans/minor_code_cleanup
Minor Code Cleanup
2 years ago
Martin Evans
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
- Deleted `NativeInfo` (internal class, not used anywhere)
2 years ago
Martin Evans
854532c08e
Merge pull request #112 from martindevans/classifier_free_guidance
Added native symbol for CFG
2 years ago
Martin Evans
a6cd0cf848
Merge pull request #111 from erinloy/master
Re-renaming some arguments to allow for easy deserialization from appsettings.json.
2 years ago
Martin Evans
4b7d718551
Added native symbol for CFG
2 years ago
Erin Loy
8f0b52eb09
Re-renaming some arguments to allow for easy deserialization from appsettings.json.
2 years ago
Martin Evans
976ca6a740
Merge pull request #102 from martindevans/grammar_basics
Grammar basics
2 years ago
Martin Evans
9fc17f3136
Fixed unit tests
2 years ago
Martin Evans
759ae26f36
Merge branch 'master' into grammar_basics
2 years ago
Martin Evans
70835a0010
Merge pull request #107 from martindevans/stateless_improvements
Stateless Executor Fix
2 years ago
Martin Evans
a45d9089e1
Fixed demos
2 years ago
Martin Evans
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2 years ago
Martin Evans
48bc0a6f8a
Doe the same for the second test, hopefully fixing CI
2 years ago
Martin Evans
6f2ab8e039
Not asserting the answer, just that it didn't change
2 years ago
Martin Evans
e7b217f462
Fixed out of context logic
2 years ago
Martin Evans
4738c26299
- Reduced context size of test, to speed it up
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2 years ago
Martin Evans
ae8ef17a4a
- Added various convenience overloads to `LLamaContext.Eval`
- Converted `SafeLLamaContextHandle` to take a `ReadOnlySpan` for Eval, narrower type better represents what's really needed
2 years ago
Martin Evans
4d0c044b9f
Added tests for the StatelessExecutor, one is currently failing
2 years ago
Martin Evans
8d5eccd28b
Merge pull request #108 from erinloy/master
renamed some arguments in ModelParams constructor so that class can be serialized easily
2 years ago
Erin Loy
592a80840b
renamed some arguments in ModelParams constructor so that classcan be serialized easily
2 years ago
Martin Evans
5e31c6db82
Merge pull request #105 from martindevans/update_demos
Updated Demos
2 years ago
Martin Evans
02a46fc363
Updated demos to use the new loading/multi context system
2 years ago
Martin Evans
1db7292b05
Fixed conflicts caused by merging of multi context PR
2 years ago
Martin Evans
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
- Integrated grammar into sampling
- Added a test for the grammar sampling
2 years ago
Martin Evans
0294bb1303
Some of the basics of the grammar API
2 years ago
Rinne
62331852bc
Merge pull request #90 from martindevans/proposal_multi_context
Multi Context
2 years ago
Martin Evans
eab073bd2b
Merge pull request #104 from zerosoup/docs/sample-code-update
Add missing semi-colon to README sample code
2 years ago
Shaun Cook
81a26a27ec
Add missing semi-colon to README sample code
2 years ago
zombieguy
10f88ebd0e
Potential fix for .Net Framework issues ( #103 )
* Added a bool to sbyte Utils convertor
As an attempt to avoid using any MarshalAs attribute for .Net Framework support this Utils method will take in a bool value and return a 1 for true or 0 for false sbyte.
* Changed all bool "MarshalAs" types to sbytes
Changed all previous BOOL types with "MarshalAs" attributes to SBYTEs and changed all the setters of them to use the Utils.BoolToSignedByte() convertor method.
* Fixed Utils bool convertor & added sbyte to bool
Improved the Utils bool convertor just casting an sbyte value to get rid of the unneeded sbyte array and added an sbyte to bool convertor to convert back the way to a C# bool assuming any positive value above 0 is a bool and no bools are packed in the single byte integer.
* bool to & from sbyte conversions via properties
All 1byte bools are now handled where they "sit", via public properties which perform the conversions to keep all external data able to communicate as it did before.
2 years ago
Martin Evans
7ebff89f68
Merge pull request #101 from martindevans/llama_sample_classifier_free_guidance
llama_sample_classifier_free_guidance
2 years ago
Martin Evans
6c84accce8
Added `llama_sample_classifier_free_guidance` method from native API
2 years ago
Martin Evans
8e8cf1404d
Merge pull request #100 from martindevans/Logger_comments
Logger Comments
2 years ago
Martin Evans
afe559ef1c
Added comments to `Logger` and fixed some nullability warnings
2 years ago
Martin Evans
76d991f376
Removed embedding test, moved to another PR
2 years ago
Martin Evans
aeb7943710
Removed Console.WriteLine
2 years ago
Martin Evans
6473f8d5e5
Temporarily added a `Console.WriteLine` into the test, to print the embedding vector for "cat" in CI
2 years ago
Martin Evans
1b35be2e0c
Added some additional basic tests
2 years ago
Martin Evans
f5a260926f
Renamed `EmbeddingCount` to `EmbeddingSize` in higher level class
2 years ago
Martin Evans
479ff57853
Renamed `EmbeddingCount` to `EmbeddingSize`
2 years ago
Martin Evans
d0a7a8fcd6
- Cleaned up disposal in LLamaContext
- sealed some classes not intended to be extended
2 years ago
Martin Evans
4d741d24f2
Marked old `LLamaContext` constructor obsolete
2 years ago
Martin Evans
20bdc2ec6f
- Apply LoRA in `LLamaWeights.LoadFromFile`
- Sanity checking that weights are not disposed when creating a context from them
- Further simplified `Utils.InitLLamaContextFromModelParams`
2 years ago
Martin Evans
e2fe08a9a2
Added a higher level `LLamaWeights` wrapper around `SafeLlamaModelHandle`
2 years ago
Martin Evans
fda7e1c038
Fixed mirostat/mirostate
2 years ago
Martin Evans
f31bdf6b93
Using the right context for Bob
2 years ago
Martin Evans
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2 years ago
Martin Evans
8fbb7e8eae
Merge pull request #99 from martindevans/native_api_improvements
Improved `NativeApi` file a bit
2 years ago