PDF (A4) - 41.3Mb
Man Pages (TGZ) - 262.8Kb
Man Pages (Zip) - 368.8Kb
Info (Gzip) - 4.1Mb
Info (Zip) - 4.1Mb
This class represents a large language model. Members accessible from instances of this class are listed here:
The LLM class has the constructor shown here:
LLM class constructor
LLM( Stringname, Objectoptions)
Arguments
name(String): the name of the modeloptions(Object) (default{}): an object containing the options used by this instance
Return type
An instance of
LLM
Usage
let model = LLM("llama3.2-3b-instruct-v1", {max_tokens: 10})
LLM provides methods for creating embeddings, generating responses, and performing Retrieval Augmented Generation. The API also provides convenience methods; seeSection 27.3.10.8, “Convenience Methods”. Both versions of these methods support variants of the methods for performing single jobs and batch processing.
Unloads the model that was loaded in the constructor. This is optional, but recommended, since doing so can reduce memory usage; after unloading, any subsequent attempt to use the instance raises an error.
Signature
undefined LLM.unload()
Arguments
None
Return type
undefined
Usage
model.unload()
This method acts as a wrapper forML_GENERATE, and generates a response using the prompt and options provided for the loaded model. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.
Signature (single job)
Object LLM.generate( Stringprompt, Objectoptions)
Arguments
prompt(String): prompt to be used for text generationoptions(Object) (default{}): an object containing the options used by this instance; see the description ofML_GENERATEfor available options
Return type
Object: The structure is similar to that ofML_GENERATE.
Usage
let response = model.generate("What is MySql?", {"top_k": 2})
Signature (batch processing)
undefined LLM.generate( TableinputTable, StringinputColumn, StringoutputColumn, Objectoptions)
Arguments
inputTable(Table): Table to use for operationsinputColumn(String): Name of column frominputTableto be embeddedoutputColumn(String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified nameoptions(Object) (optional; default{}): An object containing the options used for embedding; see the description ofML_EMBED_ROWfor available options
Return type
undefined
Usage
let schema = session.getSchema("mlcorpus")let table = schema.getTable("genai_table")model.generate(table, "input", "mlcorpus.predictions.response")
This method is a wrapper forML_EMBED_ROW, and generates an embedding whose type corresponds to the MySQLVECTOR type. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.
Signature (single job)
Float32Array LLM.embed( Stringquery, Objectoptions)
Arguments
query(String): The text of the query to be embeddedoptions(Object) (optional; default{}): An object containing the options used for embedding; see the description ofML_EMBED_ROWfor available options
Return type
Float32Array(MySQLVECTOR): The embedding
Usage
let embedding = model.embed("What is MySql?")
Signature (batch processing)
undefined LLM.embed( TableinputTable, StringinputColumn, StringoutputColumn, Objectoptions)
Arguments
inputTable(Table): Table to use for operationsinputColumn(String): Name of column frominputTableto be embeddedoutputColumn(String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified nameoptions(Object) (optional; default{}): An object containing the options used for embedding; see the description ofML_EMBED_ROWfor available options
Return type
undefined
Usage
// Using fully-qualified output column namelet schema = session.getSchema("mlcorpus")let table = schema.getTable("genai_table")model.embed(table, "input", "mlcorpus.predictions.response")// Using output column name only; constructs the fully-qualfied name// "mlcorpus.genai_table.response"let schema = session.getSchema("mlcorpus")let table = schema.getTable("genai_table")model.embed(table, "input", "response")
This method performs Retrieval Augmented Generation for a given query using the loaded genAI model, acting as a wrapper forML_RAG. It supports two variants, one for a single invocation, and one for batch processing; both of these are described in the next few paragraphs.
Signature (single job)
Object LLM.rag( Stringquery, Objectoptions)
Arguments
query(String): The text to be used for content retrieval and generationoptions(Object) (default{}): The options employed for generation; these follow the same rules as the options used withLLM.generate()
Return type
Object: The structure is similar to that of the object returned byML_RAG.
Usage
let result = model.rag("What is MySql?", {schema: ["vector_store"], n_citations: 1})
Signature (batch processing)
undefined LLM.rag( TableinputTable, StringinputColumn, StringoutputColumn, Objectoptions)
Arguments
inputTable(Table): Table to use for operationsinputColumn(String): Name of column frominputTableto be embeddedoutputColumn(String): Name of column in which to store embeddings; this can be either a fully-qualified name of a column or the name of the column only; in the latter case, the input table and its schema are used to construct the fully-qualified nameoptions(Object) (optional; default{}): An object containing the options used for embedding; see the description ofML_EMBED_ROWfor available options
Return type
undefined
Usage
let schema = session.getSchema("mlcorpus")let table = schema.getTable("genai_table")model.rag(table, "input", "mlcorpus.predictions.response", {schema: ["vector_store"], n_citations: 1})
PDF (A4) - 41.3Mb
Man Pages (TGZ) - 262.8Kb
Man Pages (Zip) - 368.8Kb
Info (Gzip) - 4.1Mb
Info (Zip) - 4.1Mb