Repository files navigation

The SDK can be installed from PyPI as follows:

$pip install lmstudio

Installation from the repository URL or a local clone is alsosupported for development and pre-release testing purposes.

The base component of the LM Studio SDK is the (synchronous)Client.This should be created once and used to manage the underlyingwebsocket connections to the LM Studio instance.

However, a top level convenience API is provided for convenience ininteractive use (this API implicitly creates a defaultClient instancewhich will remain active until the Python interpreter is terminated).

Using this convenience API, requesting text completion from an alreadyloaded LLM is as straightforward as:

importlmstudioaslmsmodel=lms.llm()model.complete("Once upon a time,")

Requesting a chat response instead only requires the extra step ofsetting up aChat helper to manage the chat history and includeit in response prediction requests:

importlmstudioaslmsEXAMPLE_MESSAGES= ("My hovercraft is full of eels!","I will not buy this record, it is scratched.")model=lms.llm()chat=lms.Chat("You are a helpful shopkeeper assisting a foreign traveller")formessageinEXAMPLE_MESSAGES:chat.add_user_message(message)print(f"Customer:{message}")response=model.respond(chat)chat.add_assistant_response(response)print(f"Shopkeeper:{response}")

Additional SDK examples and usage recommendations may be found in the mainLM Studio Python SDK documentation.

The LM Studio Python SDK uses a 3-partX.Y.Z numeric version identifier:

• X: incremented when the minimum version of significant dependencies is updated(for example, dropping support for older versions of Python or LM Studio).Previously deprecated features may be dropped when this part of the version numberincreases.
• Y: incremented when new features are added, or some other notable change isintroduced (such as support for additional versions of Python). New deprecationwarnings may be introduced when this part of the version number increases.
• Z: incremented for bug fix releases which don't contain any other changes.Adding exceptions and warnings for previously undetected situations is considereda bug fix.

This versioning policy is intentionally similar tosemantic versioning,but differs in the specifics of when the different parts of the version number will be updated.

Release candidatesmay be published prior to full releases, but this will typically onlyoccur when seeking broader feedback on particular features prior to finalizing the release.

Outside the preparation of a new release, the SDK repository will include a.devN suffixon the nominal Python package version.

$git clone https://github.com/lmstudio-ai/lmstudio-python$cd lmstudio-python

To be able to runtox -e sync-sdk-schema, it is alsonecessary to ensure thelmstudio-js submodule is updated:

$git submodule update --init --recursive

In order to work on the Python SDK, you need to install:pypi:pdm, :pypi:tox, and :pypi:tox-pdm(everything else can be executed viatox environments).

Given these tools, the default development environment can be set upand other commands executed as described below.

The simplest option for handling that is to installuv, and then useitsuv tool command to set uppdm and a second environmentwithtox +tox-pdm.pipx is another reasonable option for this task.

In order touse the Python SDK, you just need some form ofPython environment manager (sincelmstudio-python publishesthe packagelmstudio to PyPI).

The set of checks recommended for local execution are accessible viathecheck marker intox:

$tox -m check

This runs the same checks as thestatic andtest markers (described below).

The project source code is autoformatted and linted using :pypi:ruff.It also uses :pypi:mypy in strict mode to statically check that Python APIsare being accessed as expected.

All of these commands can be invoked via tox:

$tox -e format

$tox -e lint

$tox -e typecheck

Linting and type checking can be executed together using thestatic marker:

$tox -m static

Avoid using# noqa comments to suppress these warnings - whereverpossible, warnings should be fixed instead.# noqa comments arereserved for rare cases where the recommended style causes severereadability problems, and there isn't a more explicit mechanism(such astyping.cast) to indicate which check is being skipped.

# fmt: off/on and# fmt: skip comments may be used as neededwhen the autoformatter makes readability worse instead of better(for example, collapsing lists to a single line when they intentionallycover multiple lines, or breaking alignment of end-of-line comments).

The project's tests are written using the :pypi:pytest test framework.:pypi:tox is used to automate the setup and execution of these testsacross multiple Python versions. One of these is nominated as thedefault test target, and is accessible via thetest marker:

$tox -mtest

You can also use other defined versions by specifying the targetenvironment directly:

$tox -e py3.11

There are additional labels defined for running the oldest test environment,the latest test environment, and all test environments:

$tox -m test_oldest$tox -m test_latest$tox -m test_all

To ensure all the required models are loaded before running the tests, run thefollowing command:

$ tox -e load-test-models

tox has been configured to forward any additional arguments it is given topytest. This enables the use of pytest'srich CLI.In particular, you can select tests using all the options that pytest provides:

$# Using file name$tox -mtest -- tests/test_basics.py$# Using markers$tox -mtest -- -m"slow"$# Using keyword text search$tox -mtest -- -k"catalog"

Additional notes on running and updating the tests can be found in thetests/README.md file.

• the content ofsrc/lmstudio/_sdk_models is automatically generated by thesync-sdk-schema.py script insdk-schema and should not be modified directly.Runtox -e sync-sdk-schema to regenerate the Python submodule from the existingexport of thelmstudio-js schema (for example, after modifying the data modeltemplate). Runtox -e sync-sdk-schema -- --regen-schema after updating thesdk-schema/lmstudio-js submodule itself to a newer iteration of thelmstudio-js JSON API.
• as support for new API namespaces is added to the SDK, each should get a dedicatedsession type (similar to those for the already supported namespaces), even if itis only used privately by the client implementation.
• as support for new API channel endppoints is added to the SDK, each should get adedicated base endpoint type (similar to those for the already supported channels).This avoids duplicating the receive message processing between the sync and async APIs.
• thejson_api.SessionData base class is useful for defining rich result objects whichoffer additional methods that call back into the SDK (for example, this is how downloadedmodel listings offer their interfaces to load a new instance of a model).