omit='.env/*'

source='.'

show_missing = true

.hypothesis/

DJANGO_SETTINGS_MODULE = webtas.settings

addopts = --cov=. --cov-config=.coveragerc --cov-report xml:cov.xml

Applications are built on top of the invoke framework. They should construct`invoker` and then interact through it. They should avoid interacting directly with core code in order to support a variety of configurations.

The Web UI is built on top of an HTTP API built with[FastAPI](https://fastapi.tiangolo.com/) and[Socket.IO](https://socket.io/). The frontend code is found in`/frontend` and the backend code is found in`/ldm/invoke/app/api_app.py` and`/ldm/invoke/app/api/`. The code is further organized as such:

| Component| Description|

| ---| ---|

| api_app.py| Sets up the API app, annotates the OpenAPI spec with additional data, and runs the API|

| dependencies| Creates all invoker services and the invoker, and provides them to the API|

| events| An eventing system that could in the future be adapted to support horizontal scale-out|

| sockets| The Socket.IO interface - handles listening to and emitting session events (events are defined in the events service module)|

| routers| API definitions for different areas of API functionality|

The CLI is built automatically from invocation metadata, and also supports invocation piping and auto-linking. Code is available in`/ldm/invoke/app/cli_app.py`.

The Invoke framework provides the interface to the underlying AI systems and is built with flexibility and extensibility in mind. There are four major concepts: invoker, sessions, invocations, and services.

The invoker (`/ldm/invoke/app/services/invoker.py`) is the primary interface through which applications interact with the framework. Its primary purpose is to create, manage, and invoke sessions. It also maintains two sets of services:

-**invocation services**, which are used by invocations to interact with core functionality.

-**invoker services**, which are used by the invoker to manage sessions and manage the invocation queue.

Invocations and links between them form a graph, which is maintained in a session. Sessions can be queued for invocation, which will execute their graph (either the next ready invocation, or all invocations). Sessions also maintain execution history for the graph (including storage of any outputs). An invocation may be added to a session at any time, and there is capability to add and entire graph at once, as well as to automatically link new invocations to previous invocations. Invocations can not be deleted or modified once added.

The session graph does not support looping. This is left as an application problem to prevent additional complexity in the graph.

Invocations represent individual units of execution, with inputs and outputs. All invocations are located in`/ldm/invoke/app/invocations`, and are all automatically discovered and made available in the applications. These are the primary way to expose new functionality in Invoke.AI, and the[implementation guide](INVOCATIONS.md) explains how to add new invocations.

Services provide invocations access AI Core functionality and other necessary functionality (e.g. image storage). These are available in`/ldm/invoke/app/services`. As a general rule, new services should provide an interface as an abstract base class, and may provide a lightweight local implementation by default in their module. The goal for all services should be to enable the usage of different implementations (e.g. using cloud storage for image storage), but should not load any module dependencies unless that implementation has been used (i.e. don't import anything that won't be used, especially if it's expensive to import).

The AI Core is represented by the rest of the code base (i.e. the code outside of`/ldm/invoke/app/`).

Invocations represent a single operation, its inputs, and its outputs. These operations and their outputs can be chained together to generate and modify images.

To create a new invocation, either find the appropriate module file in`/ldm/invoke/app/invocations` to add your invocation to, or create a new one in that folder. All invocations in that folder will be discovered and made available to the CLI and API automatically. Invocations make use of[typing](https://docs.python.org/3/library/typing.html) and[pydantic](https://pydantic-docs.helpmanual.io/) for validation and integration into the CLI and API.

An invocation looks like this:

classUpscaleInvocation(BaseInvocation):

type: Literal['upscale']='upscale'

image: Union[ImageField,None]= Field(description="The input image")

strength:float= Field(default=0.75,gt=0,le=1,description="The strength")

level: Literal[2,4]= Field(default=2,description="The upscale level")

definvoke(self,context: InvocationContext) -> ImageOutput:

image= context.services.images.get(self.image.image_type,self.image.image_name)

results= context.services.generate.upscale_and_reconstruct(

image_list= [[image,0]],

upscale= (self.level,self.strength),

strength=0.0,# GFPGAN strength

save_original=False,

image_callback=None,

)

image_type= ImageType.RESULT

image_name= context.services.images.create_name(context.graph_execution_state_id,self.id)

context.services.images.save(image_type, image_name, results[0][0])

return ImageOutput(

image= ImageField(image_type= image_type,image_name= image_name)

)

Each portion is important to implement correctly.

classUpscaleInvocation(BaseInvocation):

type: Literal['upscale']='upscale'

All invocations must derive from`BaseInvocation`. They should have a docstring that declares what they do in a single, short line. They should also have a`type` with a type hint that's`Literal["command_name"]`, where`command_name` is what the user will type on the CLI or use in the API to create this invocation. The`command_name` must be unique. The`type` must be assigned to the value of the literal in the type hint.

image: Union[ImageField,None]= Field(description="The input image")

strength:float= Field(default=0.75,gt=0,le=1,description="The strength")

level: Literal[2,4]= Field(default=2,description="The upscale level")

Inputs consist of three parts: a name, a type hint, and a`Field` with default, description, and validation information. For example:

| Part| Value| Description|

| ----| -----| -----------|

| Name|`strength`| This field is referred to as`strength`|

| Type Hint|`float`| This field must be of type`float`|

| Field|`Field(default=0.75, gt=0, le=1, description="The strength")`| The default value is`0.75`, the value must be in the range (0,1], and help text will show "The strength" for this field.|

Notice that`image` has type`Union[ImageField,None]`. The`Union` allows this field to be parsed with`None` as a value, which enables linking to previous invocations. All fields should either provide a default value or allow`None` as a value, so that they can be overwritten with a linked output from another invocation.

The special type`ImageField` is also used here. All images are passed as`ImageField`, which protects them from pydantic validation errors (since images only ever come from links).

Finally, note that for all linking, the`type` of the linked fields must match. If the`name` also matches, then the field can be**automatically linked** to a previous invocation by name and matching.

definvoke(self,context: InvocationContext) -> ImageOutput:

image= context.services.images.get(self.image.image_type,self.image.image_name)

results= context.services.generate.upscale_and_reconstruct(

image_list= [[image,0]],

upscale= (self.level,self.strength),

strength=0.0,# GFPGAN strength

save_original=False,

image_callback=None,

)

image_type= ImageType.RESULT

image_name= context.services.images.create_name(context.graph_execution_state_id,self.id)

context.services.images.save(image_type, image_name, results[0][0])

return ImageOutput(

image= ImageField(image_type= image_type,image_name= image_name)

)

The`invoke` function is the last portion of an invocation. It is provided an`InvocationContext` which contains services to perform work as well as a`session_id` for use as needed. It should return a class with output values that derives from`BaseInvocationOutput`.

Before being called, the invocation will have all of its fields set from defaults, inputs, and finally links (overriding in that order).

Assume that this invocation may be running simultaneously with other invocations, may be running on another machine, or in other interesting scenarios. If you need functionality, please provide it as a service in the`InvocationServices` class, and make sure it can be overridden.

classImageOutput(BaseInvocationOutput):

type: Literal['image']='image'

image: ImageField= Field(default=None,description="The output image")

Output classes look like an invocation class without the invoke method. Prefer to use an existing output class if available, and prefer to name inputs the same as outputs when possible, to promote automatic invocation linking.

image_callback=None,

prefix=None,

):

results= []

forrinimage_list:

image,seed=r

try:

else:

r[0]=image

results.append([image,seed])

defapply_textmask(

self,image_path:str,prompt:str,callback,threshold:float=0.5

):

from ..services.processorimportDefaultInvocationProcessor

from ..services.graphimportGraphExecutionState

from ..services.sqliteimportSqliteItemStorage

from ...globalsimportGlobals

from ..services.image_storageimportDiskImageStorage

from ..services.invocation_queueimportMemoryInvocationQueue

from ..services.invocation_servicesimportInvocationServices

from ..services.invokerimportInvoker,InvokerServices

from ..services.generate_initializerimportget_generate

from .eventsimportFastAPIEventService

defcheck_internet()->bool:

importurllib.request

try:

urllib.request.urlopen(host,timeout=1)

except:

classApiDependencies:

invoker:Invoker=None

definitialize(

args,

config,

event_handler_id:int

):

Globals.try_patchmatch=args.patchmatch

Globals.always_use_cpu=args.always_use_cpu

Globals.internet_available=args.internet_availableandcheck_internet()

Globals.disable_xformers=notargs.xformers

Globals.ckpt_convert=args.ckpt_convert

print(f'>> Internet connectivity is{Globals.internet_available}')

generate=get_generate(args,config)

events=FastAPIEventService(event_handler_id)

output_folder=os.path.abspath(os.path.join(os.path.dirname(__file__),'../../../../outputs'))

images=DiskImageStorage(output_folder)

services=InvocationServices(

generate=generate,

events=events,

)

db_location=os.path.join(output_folder,'invokeai.db')

invoker_services=InvokerServices(

queue=MemoryInvocationQueue(),

graph_execution_manager=SqliteItemStorage[GraphExecutionState](filename=db_location,table_name='graph_executions'),

processor=DefaultInvocationProcessor()

)

ApiDependencies.invoker=Invoker(services,invoker_services)

defshutdown():

ifApiDependencies.invoker:

ApiDependencies.invoker.stop()

fromqueueimportEmpty,Queue

fromfastapi_events.dispatcherimportdispatch

from ..services.eventsimportEventServiceBase

classFastAPIEventService(EventServiceBase):

event_handler_id:int

__queue:Queue

__stop_event:threading.Event

def__init__(self,event_handler_id:int)->None:

self.event_handler_id=event_handler_id

self.__queue=Queue()

self.__stop_event=threading.Event()

asyncio.create_task(self.__dispatch_from_queue(stop_event=self.__stop_event))

super().__init__()

defstop(self,*args,**kwargs):

self.__stop_event.set()

self.__queue.put(None)

defdispatch(self,event_name:str,payload:Any)->None:

self.__queue.put(dict(

event_name=event_name,

))

asyncdef__dispatch_from_queue(self,stop_event:threading.Event):

whilenotstop_event.is_set():

try:

event=self.__queue.get(block=False)

ifnotevent:# Probably stopping

dispatch(

event.get('event_name'),

payload=event.get('payload'),

middleware_id=self.event_handler_id)

exceptEmpty:

awaitasyncio.sleep(0.001)

exceptasyncio.CancelledErrorase: