-[Text-to-Text Generation](#text-to-text-generation)

-[Fill-Mask](#fill-mask)

-[Vector Database](#vector-database)

-[LLM Fine-tuning](#llm-fine-tuning)

-[Text Classification](#llm-fine-tuning-text-classification)

In this section, we will provide a step-by-step walkthrough for fine-tuning a Language Model (LLM) for differnt tasks.

1. Ensure you have the PostgresML extension installed and configured in your PostgreSQL database. You can find installation instructions for PostgresML in the official documentation.

2. Obtain a Hugging Face API token to push the fine-tuned model to the Hugging Face Model Hub. Follow the instructions on the[Hugging Face website](https://huggingface.co/settings/tokens) to get your API token.

To begin, create a table to store your dataset. In this example, we use the 'imdb' dataset from Hugging Face. IMDB dataset contains three splits: train (25K rows), test (25K rows) and unsupervised (50K rows). In train and test splits, negative class has label 0 and positive class label 1. All rows in unsupervised split has a label of -1.

SELECTpgml.load_dataset('imdb');

We will create a view of the dataset by performing the following operations:

- Add a new text column named "class" that has positive and negative classes.

- Shuffled view of the dataset to ensure randomness in the distribution of data.

- Remove all the unsupervised splits that have label = -1.

CREATEVIEWpgml.imdb_shuffled_viewAS

label,

CASE WHEN label=0 THEN'negative'

WHEN label=1 THEN'positive'

ELSE'neutral'

ENDAS class,

FROMpgml.imdb

WHERE label!=-1

ORDER BY RANDOM();

Before splitting the data into training and test sets, it's essential to perform exploratory data analysis (EDA) to understand the distribution of labels and other characteristics of the dataset. In this section, we'll use the`pgml.imdb_shuffled_view` to explore the shuffled data.

To analyze the distribution of labels in the shuffled dataset, you can use the following SQL query:

label,

COUNT(*)AS label_count

FROMpgml.imdb_shuffled_view

GROUP BY label

ORDER BY label;

This query provides insights into the distribution of labels, helping you understand the balanceor imbalance of classesin your dataset.

To get a glimpse of the data, you can retrieve a sample of recordsfrom the shuffled dataset:

This query allows you to inspect a few records to understand the structureand content of the shuffled data.

Feel free to explore other aspects of the data, suchas the distribution oftext lengths, word frequencies,or any other features relevant to your analysis. Performing EDA is crucial for gaining insights into your datasetand making informed decisions during subsequent steps of the workflow.

Create views for trainingand test data by splitting the shuffled dataset.In this example,80% is allocated for training,and20% for testing.

Now, fine-tune the Language Model fortext classification using the created training view.In the following sections, you will see a detailed explanation of different parameters used during fine-tuning.

* project_name ('imdb_review_sentiment'): The project_name parameter specifies a unique name for your fine-tuning project. It helps identifyand organize different fine-tuning tasks within the PostgreSQL database.In this example, the project is named'imdb_review_sentiment,' reflecting the sentiment analysis taskon the IMDb dataset. You cancheck`pgml.projects` for list of projects.

* task ('text-classification'): The task parameter defines the nature of the machine learning task to be performed.In this case, it's set to'text-classification,' indicating that the fine-tuning is geared towards training a model for text classification.

* relation_name ('pgml.imdb_train_view'): The relation_name parameter identifies the training dataset to be used for fine-tuning. It specifies the view or table containing the training data. In this example,'pgml.imdb_train_view' is the view created from the shuffled IMDb dataset, and it serves as the source for model training.

* model_name ('distilbert-base-uncased'): The model_name parameter denotes the pre-trained language model architecture to be fine-tuned. In this case,'distilbert-base-uncased' is selected. DistilBERT is a distilled version of BERT, and the'uncased' variant indicates that the model does not differentiate between uppercase and lowercase letters.

* test_size (0.2): The test_size parameter determines the proportion of the dataset reserved for testing during fine-tuning. In this example, 20% of the dataset is set aside for evaluation, helping assess the model's performanceon unseen data.

* test_sampling ('last'): The test_sampling parameter defines the strategy for sampling test datafrom the dataset.In this case,'last' indicates that the most recent portion of the data, following the specified test size, is used for testing. Adjusting this parameter might be necessary basedon your specific requirementsand dataset characteristics.

The dataset_args section allows you to specify critical parameters related to your dataset for language model fine-tuning.

* text_column: The name of the column containing thetext datain your dataset.In this example, it's set to "text."

* class_column: The name of the column containing the class labels in your dataset. In this example, it'sset to"class."

Fine-tuning a language model requires careful consideration of training parametersin the training_args section. Below is a subset of training args that you can pass to fine-tuning. You can find an exhaustive list of parametersin Hugging Face documentationon [TrainingArguments](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments).

* learning_rate: The learning rate for the training. It controls the step size during the optimization process. Adjust basedon your model's convergence behavior.

* num_train_epochs: The number of training epochs. An epoch is one complete pass through the entire training dataset. Adjust based on the model's convergenceand your dataset size.

* weight_decay: L2 regularization term for weight decay. It helps prevent overfitting. Adjust basedon the complexity of your model.

* hub_token: Your Hugging Face API token to push the fine-tuned model to the Hugging Face Model Hub. Replace"YOUR_HUB_TOKEN" with the actual token.

* push_to_hub: Aboolean flag indicating whether to push the model to the Hugging Face Model Hub after fine-tuning.

During training, metricslike loss, gradient norm will be printedas infoand also loggedinpgml.logs table. Below is a snapshot of such output.

Once the training is completed, model will be evaluated against the validation dataset. You will see the belowin the client terminal. Accuracyon the evaluation dataset is0.934and F1-score is0.93.

Once the training is completed, you cancheck querypgml.logs table using the model_idor by finding the latest modelon the project.

During training, model is periodically uploaded to Hugging Face Hub. You will find the model at`https://huggingface.co/<username>/<project_name>`. An example model that was automatically pushed to Hugging Face Hub is [here](https://huggingface.co/santiadavani/imdb_review_sentiement).

Now, that we have fine-tuned modelon Hugging Face Hub, we can use [`pgml.transform`](https://postgresml.org/docs/introduction/apis/sql-extensions/pgml.transform/text-classification) to performreal-time predictionsas wellas batch predictions.

**Real-time predictions**

Here is an examplepgml.transform call forreal-time predictionson the newly minted LLM fine-tunedon IMDB review dataset.

**Batch predictions**

Sometimes, it's necessary to restart the training process from a previously trained model. This can be advantageous for various reasons, such as model fine-tuning, hyperparameter adjustments, or addressing interruptions in the training process. `pgml.tune` provides a seamless way to restart training while leveraging the progress made in the existing model. Below is a guide on how to restart training using a previous model as a starting point:

Specify the name of the existing model you want to use as a starting point. This is achieved by setting the `model_name` parameter in the `pgml.tune` function. In the example below, it is set to'santiadavani/imdb_review_sentiement'.

model_name =>'santiadavani/imdb_review_sentiement',

'imdb_review_sentiement',

task =>'text-classification',

relation_name =>'pgml.imdb_train_view',

model_name =>'santiadavani/imdb_review_sentiement',

test_sampling =>'last',

hyperparams =>'{

"training_args": {

"learning_rate": 2e-5,

"per_device_train_batch_size":16,

"per_device_eval_batch_size":16,

"num_train_epochs":1,

"weight_decay":0.01,

"hub_token":"",

"push_to_hub": true

},

"dataset_args": {"text_column":"text","class_column":"class" }

}'