After you choose a model, you must import it and its tokenizer vocabulary to your cluster. When you import the model, it must be chunked and imported one chunk at a time for storage in parts due to its size.

Eland is an Elasticsearch Python client that provides a simple script to perform the conversion of Hugging Face transformer models to their TorchScript representations, the chunking process, and upload to Elasticsearch; it is therefore the recommended import method. You can either install the Python Eland client on your machine or use a Docker image to build Eland and run the model import script.

1. Install theEland Python client with PyTorch extra dependencies.

   python -m pip install 'eland[pytorch]'

2. Run theeland_import_hub_model script to download the model from Hugging Face, convert it to TorchScript format, and upload to the Elasticsearch cluster. For example:

   eland_import_hub_model \--cloud-id <cloud-id> \-u <username> -p <password> \--hub-model-id elastic/distilbert-base-cased-finetuned-conll03-english \--task-type ner

   1. Specify the Elastic Cloud identifier. Alternatively, use--url.
   2. Provide authentication details to access your cluster. Refer toAuthentication methods to learn more.
   3. Specify the identifier for the model in the Hugging Face model hub.
   4. Specify the type of NLP task. Supported values arefill_mask,ner,question_answering,text_classification,text_embedding,text_expansion,text_similarity, andzero_shot_classification.

For more details, refer to theEland documentation.

If you want to use Eland without installing it, run the following command:

$ docker run -it --rm --network host docker.elastic.co/eland/eland

Theeland_import_hub_model script can be run directly in the docker command:

docker run -it --rm docker.elastic.co/eland/eland \    eland_import_hub_model \      --url $ELASTICSEARCH_URL \      --hub-model-id elastic/distilbert-base-uncased-finetuned-conll03-english \      --start

Replace the$ELASTICSEARCH_URL with the URL for your Elasticsearch cluster. Refer toAuthentication methods to learn more.

The following authentication options are available when using the import script:

• username/password authentication (specified with the-u and-p options):

eland_import_hub_model --url https://<hostname>:<port> -u <username> -p <password> ...

• username/password authentication (embedded in the URL):

eland_import_hub_model --url https://<user>:<password>@<hostname>:<port> ...

• API key authentication:

eland_import_hub_model --url https://<hostname>:<port> --es-api-key <api-key> ...