Repository files navigation

Prompt Foundry is a comprehensive tool for prompt engineering, management, and evaluation. It is designed to simplify the development and integration process for developers working on TypeScript, JavaScript, and NodeJS AI applications utilizing large language models (LLMs).

This SDK provides convenient access to the Prompt Foundry REST API from server-side TypeScript or JavaScript applications.

To use this SDK, you need a Prompt Foundry account. Sign up atpromptfoundry.ai. Follow the getting started guide in ourdocumentation to get set up.

npm install @prompt-foundry/typescript-sdk

The full Prompt Foundry documentation can be found atdocs.promptfoundry.ai.

Initiates a completion request to the configured LLM provider using specified parameters and provided variables. This endpoint abstracts the integration with different model providers, enabling seamless switching between models while maintaining a consistent data model for your application.

importPromptFoundryfrom'@prompt-foundry/typescript-sdk';// Initialize Prompt Foundry SDK with your API keyconstpromptFoundry=newPromptFoundry({apiKey:process.env['PROMPT_FOUNDRY_API_KEY'],});asyncfunctionmain(){// Retrieve model parameters for the promptconstcompletionCreateResponse=awaitclient.completion.create('637ae1aa8f4aa6fad144ccbd',{// Optionally append additional messages to the converstation thread on top of your configured prompt messagesappendMessages:[{role:'user',content:[{type:'TEXT',text:'What is the weather in Seattle, WA?',},],},],// Supports prompt template variablesvariables:{},});// completion responseconsole.log(completionCreateResponse.message);}main().catch(console.error);

Fetches the configured model parameters and messages rendered with the provided variables mapped to the set LLM provider. This endpoint abstracts the need to handle mapping between different providers, while still allowing direct calls to the providers.

Install the OpenAI SDK

npm install openai

Import the OpenAI and Prompt Foundry SDKs

importPromptFoundryfrom'@prompt-foundry/typescript-sdk';import{Configuration,OpenAIApi}from'openai';// Initialize Prompt Foundry SDK with your API keyconstpromptFoundry=newPromptFoundry({apiKey:process.env['PROMPT_FOUNDRY_API_KEY'],});// Initialize OpenAI SDK with your API keyconstconfiguration=newConfiguration({apiKey:process.env['OPENAI_API_KEY'],});constopenai=newOpenAIApi(configuration);asyncfunctionmain(){// Retrieve model parameters for the promptconstmodelParameters=awaitpromptFoundry.prompts.getParameters('1212121',{variables:{hello:'world'},appendMessages:[{role:'user',content:[{type:'TEXT',text:'What is the weather in Seattle, WA?',},],},],});// check if provider is Open AIif(modelParameters.provider==='openai'){// Use the retrieved parameters to create a chat completion requestconstmodelResponse=awaitopenai.chat.completions.create(modelParameters.parameters);// Print the response from OpenAIconsole.log(modelResponse.data);}}main().catch(console.error);

Install the Anthropic SDK

npm install @anthropic-ai/sdk

Import the Anthropic and Prompt Foundry SDKs

importPromptFoundryfrom'@prompt-foundry/typescript-sdk';importAnthropicfrom'@anthropic-ai/sdk';// Initialize Prompt Foundry SDK with your API keyconstpromptFoundry=newPromptFoundry({apiKey:process.env['PROMPT_FOUNDRY_API_KEY'],});// Initialize Anthropic SDK with your API keyconstanthropic=newAnthropic({apiKey:process.env['ANTHROPIC_API_KEY'],});asyncfunctionmain(){// Retrieve model parameters for the promptconstmodelParameters=awaitpromptFoundry.prompts.getParameters('1212121',{variables:{hello:'world'},appendMessages:[{role:'user',content:[{type:'TEXT',text:'What is the weather in Seattle, WA?',},],},],});// check if provider is Open AIif(modelParameters.provider==='anthropic'){// Use the retrieved parameters to create a chat completion requestconstmessage=awaitanthropic.messages.create(modelParameters.parameters);// Print the response from Anthropicconsole.log(message.content);}}main().catch(console.error);

This library includes TypeScript definitions for all request params and response fields. You may import and use them like so:

importPromptFoundryfrom'@prompt-foundry/typescript-sdk';constclient=newPromptFoundry({apiKey:process.env['PROMPT_FOUNDRY_API_KEY'],// This is the default and can be omitted});asyncfunctionmain(){constcompletionCreateResponse:PromptFoundry.CompletionCreateResponse=awaitclient.completion.create('1212121',);}main();

Documentation for each method, request param, and response field are available in docstrings and will appear on hover in most modern editors.

When the library is unable to connect to the API,or if the API returns a non-success status code (i.e., 4xx or 5xx response),a subclass ofAPIError will be thrown:

asyncfunctionmain(){constcompletionCreateResponse=awaitclient.completion.create('1212121').catch(async(err)=>{if(errinstanceofPromptFoundry.APIError){console.log(err.status);// 400console.log(err.name);// BadRequestErrorconsole.log(err.headers);// {server: 'nginx', ...}}else{throwerr;}});}main();

Error codes are as followed:

Certain errors will be automatically retried 2 times by default, with a short exponential backoff.Connection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict,429 Rate Limit, and >=500 Internal errors will all be retried by default.

You can use themaxRetries option to configure or disable this:

// Configure the default for all requests:constclient=newPromptFoundry({maxRetries:0,// default is 2});// Or, configure per-request:awaitclient.completion.create('1212121',{maxRetries:5,});

Requests time out after 1 minute by default. You can configure this with atimeout option:

// Configure the default for all requests:constclient=newPromptFoundry({timeout:20*1000,// 20 seconds (default is 1 minute)});// Override per-request:awaitclient.completion.create('1212121',{timeout:5*1000,});

On timeout, anAPIConnectionTimeoutError is thrown.

Note that requests which time out will beretried twice by default.

The "raw"Response returned byfetch() can be accessed through the.asResponse() method on theAPIPromise type that all methods return.

You can also use the.withResponse() method to get the rawResponse along with the parsed data.

constclient=newPromptFoundry();constresponse=awaitclient.completion.create('1212121').asResponse();console.log(response.headers.get('X-My-Header'));console.log(response.statusText);// access the underlying Response objectconst{data:completionCreateResponse,response:raw}=awaitclient.completion.create('1212121').withResponse();console.log(raw.headers.get('X-My-Header'));console.log(completionCreateResponse.message);

This library is typed for convenient access to the documented API. If you need to access undocumentedendpoints, params, or response properties, the library can still be used.

To make requests to undocumented endpoints, you can useclient.get,client.post, and other HTTP verbs.Options on the client, such as retries, will be respected when making these requests.

awaitclient.post('/some/path',{body:{some_prop:'foo'},query:{some_query_arg:'bar'},});

To make requests using undocumented parameters, you may use// @ts-expect-error on the undocumentedparameter. This library doesn't validate at runtime that the request matches the type, so any extra values yousend will be sent as-is.

client.foo.create({foo:'my_param',bar:12,//@ts-expect-error baz is not yet publicbaz:'undocumented option',});

For requests with theGET verb, any extra params will be in the query, all other requests will send theextra param in the body.

If you want to explicitly send an extra argument, you can do so with thequery,body, andheaders requestoptions.

To access undocumented response properties, you may access the response object with// @ts-expect-error onthe response object, or cast the response object to the requisite type. Like the request params, we do notvalidate or strip extra properties from the response from the API.

By default, this library usesnode-fetch in Node, and expects a globalfetch function in other environments.

If you would prefer to use a global, web-standards-compliantfetch function even in a Node environment,(for example, if you are running Node with--experimental-fetch or using NextJS which polyfills withundici),add the following import before your first importfrom "PromptFoundry":

// Tell TypeScript and the package to use the global web fetch instead of node-fetch.// Note, despite the name, this does not add any polyfills, but expects them to be provided if needed.import'@prompt-foundry/typescript-sdk/shims/web';importPromptFoundryfrom'@prompt-foundry/typescript-sdk';

To do the inverse, addimport "@prompt-foundry/typescript-sdk/shims/node" (which does import polyfills).This can also be useful if you are getting the wrong TypeScript types forResponse (more details).

You may also provide a customfetch function when instantiating the client,which can be used to inspect or alter theRequest orResponse before/after each request:

import{fetch}from'undici';// as one exampleimportPromptFoundryfrom'@prompt-foundry/typescript-sdk';constclient=newPromptFoundry({fetch:async(url:RequestInfo,init?:RequestInit):Promise<Response>=>{console.log('About to make a request',url,init);constresponse=awaitfetch(url,init);console.log('Got response',response);returnresponse;},});

Note that if given aDEBUG=true environment variable, this library will log all requests and responses automatically.This is intended for debugging purposes only and may change in the future without notice.

By default, this library uses a stable agent for all http/https requests to reuse TCP connections, eliminating many TCP & TLS handshakes and shaving around 100ms off most requests.

If you would like to disable or customize this behavior, for example to use the API behind a proxy, you can pass anhttpAgent which is used for all requests (be they http or https), for example:

importhttpfrom'http';import{HttpsProxyAgent}from'https-proxy-agent';// Configure the default for all requests:constclient=newPromptFoundry({httpAgent:newHttpsProxyAgent(process.env.PROXY_URL),});// Override per-request:awaitclient.completion.create('1212121',{httpAgent:newhttp.Agent({keepAlive:false}),});

This package generally followsSemVer conventions, though certain backwards-incompatible changes may be released as minor versions:

1. Changes that only affect static types, without breaking runtime behavior.
2. Changes to library internals which are technically public but not intended or documented for external use.(Please open a GitHub issue to let us know if you are relying on such internals).
3. Changes that we do not expect to impact the vast majority of users in practice.

We take backwards-compatibility seriously and work hard to ensure you can rely on a smooth upgrade experience.

We are keen for your feedback; please open anissue with questions, bugs, or suggestions.

TypeScript >= 4.5 is supported.

The following runtimes are supported:

• Node.js 18 LTS or later (non-EOL) versions.
• Deno v1.28.0 or higher, usingimport PromptFoundry from "npm:@prompt-foundry/typescript-sdk".
• Bun 1.0 or later.
• Cloudflare Workers.
• Vercel Edge Runtime.
• Jest 28 or greater with the"node" environment ("jsdom" is not supported at this time).
• Nitro v2.6 or greater.

Note that React Native is not supported at this time.

If you are interested in other runtime environments, please open or upvote an issue on GitHub.