Repository files navigation

Laminar is the open-source platform for tracing and evaluating AI applications.

• Tracing
  • OpenTelemetry-based automatic tracing of common AI frameworks and SDKs (LangChain, OpenAI, Anthropic ...) with just 2 lines of code. (powered byOpenLLMetry).
  • Trace input/output, latency, cost, token count.
  • Function tracing withobserve decorator/wrapper.
  • Image tracing.
• Evals
  • Run evals in parallel with a simple SDK
• Datasets
  • Export production trace data to datasets.
  • Run evals on hosted datasets.
• Built for scale
  • Written in Rust 🦀
  • Traces are sent via gRPC, ensuring the best performance and lowest overhead.
• Modern Open-Source stack
  • RabbitMQ for message queue, Postgres for data, Clickhouse for analytics.
• Dashboards for statistics / traces / evaluations / tags.

Check out full documentation heredocs.lmnr.ai.

The fastest and easiest way to get started is with our managed platform ->lmnr.ai

For a quick start, clone the repo and start the services with docker compose:

git clone https://github.com/lmnr-ai/lmnrcd lmnrdocker compose up -d

This will spin up a lightweight version of the stack with Postgres, clickhouse, app-server, and frontend. This is good for a quickstartor for lightweight usage. You can access the UI athttp://localhost:5667 in your browser.

You will also need to properly configure the SDK, withbaseUrl and correct ports. Seehttps://docs.lmnr.ai/self-hosting/setup

For production environment, we recommend using ourmanaged platform ordocker compose -f docker-compose-full.yml up -d.

docker-compose-full.yml is heavy but it will enable all the features.

• app-server – core Rust backend
• rabbitmq – message queue for reliable trace processing
• frontend – Next.js frontend and backend
• postgres – Postgres database for all the application data
• clickhouse – columnar OLAP database for more efficient trace and tag analytics

For running and building Laminar locally, or to learn more about docker compose files,follow the guide inContributing.

First,create a project and generate a project API key. Then,

npm add @lmnr-ai/lmnr

It will install Laminar TS SDK and all instrumentation packages (OpenAI, Anthropic, LangChain ...)

To start tracing LLM calls just add

import{Laminar}from'@lmnr-ai/lmnr';Laminar.initialize({projectApiKey:process.env.LMNR_PROJECT_API_KEY});

To trace inputs / outputs of functions useobserve wrapper.

import{OpenAI}from'openai';import{observe}from'@lmnr-ai/lmnr';constclient=newOpenAI({apiKey:process.env.OPENAI_API_KEY});constpoemWriter=observe({name:'poemWriter'},async(topic)=>{constresponse=awaitclient.chat.completions.create({model:"gpt-4o-mini",messages:[{role:"user",content:`write a poem about${topic}`}],});returnresponse.choices[0].message.content;});awaitpoemWriter();

First,create a project and generate a project API key. Then,

pip install --upgrade'lmnr[all]'

It will install Laminar Python SDK and all instrumentation packages. See list of all instrumentshere

To start tracing LLM calls just add

fromlmnrimportLaminarLaminar.initialize(project_api_key="<LMNR_PROJECT_API_KEY>")

To trace inputs / outputs of functions use@observe() decorator.

importosfromopenaiimportOpenAIfromlmnrimportobserve,LaminarLaminar.initialize(project_api_key="<LMNR_PROJECT_API_KEY>")client=OpenAI(api_key=os.environ["OPENAI_API_KEY"])@observe()# annotate all functions you want to tracedefpoem_writer(topic):response=client.chat.completions.create(model="gpt-4o",messages=[            {"role":"user","content":f"write a poem about{topic}"},        ],    )poem=response.choices[0].message.contentreturnpoemif__name__=="__main__":print(poem_writer(topic="laminar flow"))

Running the code above will result in the following trace.

To learn more about instrumenting your code, check out our client libraries: