Changed type conversion

Type conversion is now done in the property rather than the utils class and uses the System.Convert class to ensure consistency.

Minor Code Cleanup

- Deleted `NativeInfo` (internal class, not used anywhere)

Added native symbol for CFG

Re-renaming some arguments to allow for easy deserialization from appsettings.json.

Grammar basics

Stateless Executor Fix

- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around

- Removed some unnecessary `ToArray` calls
 - Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from

- Converted `SafeLLamaContextHandle` to take a `ReadOnlySpan` for Eval, narrower type better represents what's really needed

renamed some arguments in ModelParams constructor so that class can be serialized easily

Updated Demos

- Integrated grammar into sampling
 - Added a test for the grammar sampling

Multi Context

Add missing semi-colon to README sample code

* Added a bool to sbyte Utils convertor

As an attempt to avoid using any MarshalAs attribute for .Net Framework support this Utils method will take in a bool value and return a 1 for true or 0 for false sbyte.

* Changed all bool "MarshalAs" types to sbytes

Changed all previous BOOL types with "MarshalAs" attributes to SBYTEs and changed all the setters of them to use the Utils.BoolToSignedByte() convertor method.

* Fixed Utils bool convertor & added sbyte to bool

Improved the Utils bool convertor just casting an sbyte value to get rid of the unneeded sbyte array and added an sbyte to bool convertor to convert back the way to a C# bool assuming any positive value above 0 is a bool and no bools are packed in the single byte integer.

* bool to & from sbyte conversions via properties

All 1byte bools are now handled where they "sit", via public properties which perform the conversions to keep all external data able to communicate as it did before.

llama_sample_classifier_free_guidance

Logger Comments