Repository files navigation

This repositories enable third-party libraries integrated withhuggingface_hub to createtheir own docker so that the widgets on the hub can work as thetransformers one do.

The hardware to run the API will be provided by Hugging Face for now.

Thedocker_images/common folder is intended to be a starter point for all new libs thatwant to be integrated.

1. Copy thedocker_images/common folder into your library's namedocker_images/example.

2. Edit:

   • docker_images/example/requirements.txt
   • docker_images/example/app/main.py
   • docker_images/example/app/pipelines/{task_name}.py

   to implement the desired functionality. All required code is marked withIMPLEMENT_THIS markup.

3. Remove:

   • Any pipeline files indocker_images/example/app/pipelines/ that are not used.
   • Any tests associated with deleted pipelines indocker_images/example/tests.
   • Any imports of the pipelines you deleted fromdocker_images/example/app/pipelines/__init__.py

4. Feel free to customize anything required by your lib everywhere you want. The only real requirements, are to honor the HTTP endpoints, in the same fashion as thecommon folder for all your supported tasks.

5. Editexample/tests/test_api.py to add TESTABLE_MODELS.

6. Pass the test suitepytest -sv --rootdir docker_images/example/ docker_images/example/

7. Submit your PR and enjoy !

Doing the first 7 steps is good enough to get started, however in the processyou can anticipate some problems corrections early on. Maintainers will help youalong the way if you don't feel confident to follow those steps yourself

1. Test your creation within a docker

./manage.pydockerMY_MODEL

should work and responds on port 8000.curl -X POST -d "test" http://localhost:8000 for instance ifthe pipeline deals with simple text.

If it doesn't work out of the box and/or docker is slow for some reason youcan test locally (using your local python environment) with :

./manage.py start MY_MODEL

2. Test your docker uses cache properly.

When doing subsequent docker launch with the same model_id, the docker should start up very fast and not redownload the whole model file. If you see the model/repo being downloaded over and over, it means the cache is not being used correctly.You can edit thedocker_images/{framework}/Dockerfile and add an environment variable (by default it assumesHUGGINGFACE_HUB_CACHE), or your code directly to putthe model files in the/data folder.

3. Add a docker test.

Edit thetests/test_dockers.py file to add a new test with your new frameworkin it (def test_{framework}(self): for instance). As a basic you should have 1 line per task in this test function with a real working model on the hub. Those tests are relatively slow but will check automatically that correct errors are replied by your API and that the cache works properly. To run those tests your can simply do:

RUN_DOCKER_TESTS=1 pytest -sv tests/test_dockers.py::DockerImageTests::test_{framework}

If you ever come across a bug withinapi-inference-community/ package or want to update itthe development process is slightly more involved.

• First, make sure you need to change this package, each framework is very autonomousso if your code can get away by being standalone go that way first as it's much simpler.
• If you can make the change only inapi-inference-community without depending on itthat's also a great option. Make sure to add the proper tests to your PR.
• Finally, the best way to go is to develop locally usingmanage.py command:
• Do the necessary modifications withinapi-inference-community first.
• Install it locally in your environment withpip install -e .
• Install your package dependencies locally.
• Run your webserver locally:./manage.py start --framework example --task audio-source-separation --model-id MY_MODEL
• When everything is working, you will need to split your PR in two, 1 for theapi-inference-community part.The second one will be for your package specific modifications and will only land once theapi-inference-community tag has landed.
• This workflow is still work in progress, don't hesitate to ask questions to maintainers.

Another similar command./manage.py docker --framework example --task audio-source-separation --model-id MY_MODELWill launch the server, but this time in a protected, controlled docker environment making sure the behaviorwill be exactly the one in the API.

• Automatic speech recognition: Input is a file, output is a dict of understood words being said within the file
• Text generation: Input is a text, output is a dict of generated text
• Image recognition: Input is an image, output is a dict of generated text
• Question answering: Input is a question + some context, output is a dict containing necessary information to locate the answer to thequestion within thecontext.
• Audio source separation: Input is some audio, and the output is n audio files that sum up to the original audio but contain individual sources of sound (either speakers or instruments for instant).
• Token classification: Input is some text, and the output is a list of entities mentioned in the text. Entities can be anything remarkable like locations, organisations, persons, times etc...
• Text to speech: Input is some text, and the output is an audio file saying the text...
• Sentence Similarity: Input is some sentence and a list of reference sentences, and the list of similarity scores.