We now support plugins in optillm. Plugins enhance the capabilities of models with features like:

• Memory
• Privacy
• Execute Code
• Read URLs
• ... your own plugin

You can write your own plugins in python and the proxy will load them at runtime and you can use it.

We also use a combination of plugins and techniques using the operators & and |. E.g. to combine Read URLs with Memory you can just usereadurls&memory-<model-name> as the model name. Using & will apply the plugins left to right one after the other in a pipeline. Using | will run the approaches in parallel and return a list of responses.