Browse Source

docs: update readme.

tags/v0.2.1
Yaohui Liu 3 years ago
parent
commit
7eec970dd3
No known key found for this signature in database GPG Key ID: E86D01E1809BD23E
4 changed files with 92 additions and 1 deletions
  1. BIN
      Assets/LLamaSharpLogo.png
  2. BIN
      Assets/console_demo.gif
  3. +14
    -0
      LLamaSharp.sln
  4. +78
    -1
      README.md

BIN
Assets/LLamaSharpLogo.png View File

Before After
Width: 1079  |  Height: 162  |  Size: 41 kB

BIN
Assets/console_demo.gif View File

Before After
Width: 1971  |  Height: 1078  |  Size: 688 kB

+ 14
- 0
LLamaSharp.sln View File

@@ -9,6 +9,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "LLama.Examples", "LLama.Exa
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "LLamaSharp", "LLama\LLamaSharp.csproj", "{01A12D68-DE95-425E-AEEE-2D099305036D}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "WPFDemo", "WPFDemo\WPFDemo.csproj", "{1E952A70-B720-4F76-9856-EC3B4259A80B}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@@ -55,6 +57,18 @@ Global
{01A12D68-DE95-425E-AEEE-2D099305036D}.Release|Any CPU.Build.0 = Release|Any CPU
{01A12D68-DE95-425E-AEEE-2D099305036D}.Release|x64.ActiveCfg = Release|x64
{01A12D68-DE95-425E-AEEE-2D099305036D}.Release|x64.Build.0 = Release|x64
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Debug|Any CPU.Build.0 = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Debug|x64.ActiveCfg = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Debug|x64.Build.0 = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.GPU|Any CPU.ActiveCfg = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.GPU|Any CPU.Build.0 = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.GPU|x64.ActiveCfg = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.GPU|x64.Build.0 = Debug|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Release|Any CPU.ActiveCfg = Release|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Release|Any CPU.Build.0 = Release|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Release|x64.ActiveCfg = Release|Any CPU
{1E952A70-B720-4F76-9856-EC3B4259A80B}.Release|x64.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE


+ 78
- 1
README.md View File

@@ -1 +1,78 @@
C#/.NET binding of llama.cpp, provided with Web API and Native UI.
# LLamaSharp - .NET Bindings for llama.cpp

![logo](Assets/LLamaSharpLogo.png)

The C#/.NET binding of llama.cpp. It provides APIs to inference the LLaMa Models and deploy it on native environment or Web. It works on
both Windows and Linux and does NOT require compiling the library yourself.

## Installation

Just search `LLama` in nuget package manager and install it!

```
PM> Install-Package LLama
```

## Usages

Currently, `LLamaSharp` provides two kinds of model, `LLamaModelV1` and `LLamaModel`. Both of them works but `LLamaModel` is more recommended
because it provides better alignment with the master branch of [llama.cpp](https://github.com/ggerganov/llama.cpp).

Besides, `ChatSession` makes it easier to wrap your own chat bot. The code below is a simple example. For all examples, please refer to
[Examples](./LLama.Examples).

```cs

var model = new LLamaModel(new LLamaParams(model: "<Your path>", n_ctx: 512, repeat_penalty: 1.0f));
var session = new ChatSession<LLamaModel>(model).WithPromptFile("<Your prompt file path>")
.WithAntiprompt(new string[] { "User:" );
Console.Write("\nUser:");
while (true)
{
Console.ForegroundColor = ConsoleColor.Green;
var question = Console.ReadLine();
Console.ForegroundColor = ConsoleColor.White;
var outputs = _model.Call(question);
foreach (var output in outputs)
{
Console.Write(output);
}
}
```

## Demo

![demo-console](Assets/console_demo.gif)

## Roadmap

✅ LLaMa model inference.

✅ Embeddings generation.

✅ Chat session.

🔳 Quantization

🔳 ASP.NET core Integration

🔳 WPF UI Integration

## Assets

The model weights is too large to include in the project. However some resources could be found below:

- [eachadea/ggml-vicuna-13b-1.1](https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/tree/main)
- [TheBloke/wizardLM-7B-GGML](https://huggingface.co/TheBloke/wizardLM-7B-GGML)
- Magnet: [magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA](magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA)

The weights included in the magnet is exactly the weights from [Facebook LLaMa](https://github.com/facebookresearch/llama).

The prompts could be found below:
- [llama.cpp prompts](https://github.com/ggerganov/llama.cpp/tree/master/prompts)
- [ChatGPT_DAN](https://github.com/0xk1h0/ChatGPT_DAN)
- [awesome-chatgpt-prompts-zh](https://github.com/PlexPt/awesome-chatgpt-prompts-zh)

## License

This project is licensed under the terms of the MIT license.

Loading…
Cancel
Save