wangtao
/
autogen
Not watched
1
0
Fork 0
Code
Releases 127 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
Tree: 6bc896f6e2
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '6bc896f6e2'
${ noResults }
autogen/dotnet/website/articles/Consume-LLM-server-from-LM-...

1.2 kB
Raw Blame History

Consume LLM server from LM Studio

You can use @AutoGen.LMStudio.LMStudioAgent from AutoGen.LMStudio package to consume openai-like API from LMStudio local server.

What's LM Studio

LM Studio is an app that allows you to deploy and inference hundreds of thousands of open-source language model on your local machine. It provides an in-app chat ui plus an openai-like API to interact with the language model programmatically.

Installation

  • Install LM studio if you haven't done so. You can find the installation guide here
  • Add AutoGen.LMStudio to your project.
<ItemGroup>
    <PackageReference Include="AutoGen.LMStudio" Version="AUTOGEN_LMSTUDIO_VERSION" />
</ItemGroup>

Usage

The following code shows how to use LMStudioAgent to write a piece of C# code to calculate 100th of fibonacci. Before running the code, make sure you have local server from LM Studio running on localhost:1234.

[!code-csharp]
[!code-csharp]