Browse Source

docs: update readme file.

tags/v0.2.2
Yaohui Liu 3 years ago
parent
commit
1afba03c5d
No known key found for this signature in database GPG Key ID: E86D01E1809BD23E
1 changed files with 9 additions and 1 deletions
  1. +9
    -1
      README.md

+ 9
- 1
README.md View File

@@ -33,6 +33,8 @@ with Intel i7-12700, 3060Ti with 7B model. Note that the benchmark uses `LLamaMo

## Usages

#### Model Inference and Chat Session

Currently, `LLamaSharp` provides two kinds of model, `LLamaModelV1` and `LLamaModel`. Both of them works but `LLamaModel` is more recommended
because it provides better alignment with the master branch of [llama.cpp](https://github.com/ggerganov/llama.cpp).

@@ -58,6 +60,8 @@ while (true)
}
```

#### Quantization

The following example shows how to quantize the model. With LLamaSharp you needn't to compile c++ project and run scripts to quantize the model, instead, just run it in C#.

```cs
@@ -74,6 +78,10 @@ else
}
```

#### Web API

We provide the integration of ASP.NET core [here](./LLama.WebAPI). Since currently the API is not stable, please clone the repo and use it. In the future we'll publish it on NuGet.

## Demo

![demo-console](Assets/console_demo.gif)
@@ -88,7 +96,7 @@ else

✅ Quantization

🔳 ASP.NET core Integration
ASP.NET core Integration

🔳 WPF UI Integration



Loading…
Cancel
Save