Rinne
112e33eee8
build: add the dependabot.
2 years ago
Rinne
16d5d5514f
docs: update README.md.
2 years ago
Martin Evans
5a9e13c689
Merge pull request #223 from martindevans/batch_decoding
New Binaries, Improved Sampling API, Batch Decoding Prototype
2 years ago
Martin Evans
db8f3980ea
New binaries from this commit: 207b51900e
Should fix the extreme speed loss.
2 years ago
Martin Evans
f8b2c5de3d
Merge pull request #227 from martindevans/fixed_build_process
Fixed GitHub Build Process
2 years ago
Martin Evans
0255aa2c6e
Disabled march=native in github builds!
2 years ago
Martin Evans
b6d242193e
Debugging slowdown by removing some things:
- Removed all `record struct` uses in native code
- Removed usage of `readonly` in native structs
Minor fix:
- Added sequential layout to `LLamaModelQuantizeParams`
2 years ago
Martin Evans
529b06b35b
- Fixed rope frequency/base to use the values in the model by default, instead of always overriding them by default!
2 years ago
Martin Evans
dcc82e582e
Fixed `Eval` on platforms < dotnet 5
2 years ago
Martin Evans
09bc688b3c
Skipped slow test again
2 years ago
Martin Evans
51c292ebd8
Added a safe method for `llama_get_logits_ith`
2 years ago
Martin Evans
cdf20d3c7a
Added timing to stateless test
2 years ago
Martin Evans
7e3cde4c13
Moved helper methods into `LLamaBatchSafeHandle`
2 years ago
Martin Evans
ccb8afae46
Cleaned up stateless executor as preparation for changing it to use the new batched decoding system.
2 years ago
Martin Evans
c786fb0ec8
Using `IReadOnlyList` instead of `IEnumerable` in `IInferenceParams`
2 years ago
Martin Evans
aae63a5b92
Minor cleanup on the BatchedDecoding example
2 years ago
Martin Evans
c7fdb9712c
Added binaries, built from ` 6961c4bd0b`
2 years ago
Martin Evans
f41d26fe32
Removed hardcoded model path
2 years ago
Martin Evans
e81b3023d5
Rewritten sampling API to be accessed through the `LLamaTokenDataArray` object
2 years ago
Martin Evans
3c5547b2b7
Reduced some uses of `NativeApi` in `BatchedDecoding` by adding some helper methods
2 years ago
Martin Evans
b38e3f6fe2
binaries (avx512)
2 years ago
Martin Evans
a024d2242e
It works!
had to update binary to `b1426`
2 years ago
Martin Evans
8cd81251b4
initial setup
2 years ago
Martin Evans
321d0b58c4
Merge pull request #202 from martindevans/multi_gpu
Multi GPU
2 years ago
Martin Evans
c1ce547ebe
Merge pull request #222 from martindevans/ci_dotnet7.0_only
CI dotnet 7.0
2 years ago
Martin Evans
4436ea4715
Modified CI to use dotnet 7.0 only, hopefully more stable!
2 years ago
Martin Evans
f6a472ae86
Setting the default seed to `0xFFFFFFFF` (no seed, randomised)
2 years ago
Rinne
bb5dedf5eb
docs: create CONTRIBUTING.md
2 years ago
Martin Evans
b72b253549
removed OOM killer step (not allowed on GitHub actions)
2 years ago
Martin Evans
a769ab3a40
Logging out events from the OOM killer
2 years ago
Martin Evans
d5874a279c
Updated test runner to latest version
2 years ago
Martin Evans
8b582a4cc8
Added `if: always()` to force logs to be written even if earlier steps fail
2 years ago
Martin Evans
b4e57f7966
Added extra diagnostics to test runner
2 years ago
Martin Evans
666455d144
Merge pull request #219 from martindevans/fix/broken_text_decoding
Fixed Broken Text Decoding
2 years ago
Martin Evans
36c71abcfb
Fixed `LLama.StreamingTokenDecoderLLamaLLama.StreamingTokenDecoderLLamaLLama.StreamingTokenDecoderLLama` spam in all executors except Stateless.
2 years ago
Martin Evans
5b6408b072
Merge pull request #205 from martindevans/roundtrip_tokenization_investigation
RoundTrip Tokenization Errors
2 years ago
Martin Evans
4a6319742d
Update README.md
2 years ago
Martin Evans
4e315423a1
Update README.md
2 years ago
Rinne
f0552dc813
Merge pull request #217 from AsakusaRinne/master
docs: update the readme.
2 years ago
Yaohui Liu
4c80b40870
docs: update the readme.
2 years ago
Rinne
1299de3e5b
Merge pull request #216 from AsakusaRinne/auto_release_ci
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
4b86bc8d9d
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
27b21cf136
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
0f6408f60a
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
c6c478f52b
ci: try to fix error of pushing package.
2 years ago
Rinne
f62afad1e8
Merge pull request #215 from AsakusaRinne/auto_release_ci
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
a5e66b31dc
ci: fix typo.
2 years ago
Yaohui Liu
20d8cb1e84
ci: try to fix error of pushing package.
2 years ago
Rinne
3fd1063a87
Merge pull request #214 from AsakusaRinne/auto_release_ci
ci: try to fix error of pushing package.
2 years ago
Yaohui Liu
e04c2c51ab
ci: try to fix error of pushing package.
2 years ago