Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

How to implement Asynchronous Processing?#1012

Unanswered
jokiefer asked this question inQ&A
Discussion options

Recommendations from the JSON:API specs

JSON:API defines the following to handle async processing for long running tasks:

POST /photos HTTP/1.1

The request SHOULD return a status 202 Accepted with a link in the Content-Location header.

HTTP/1.1 202 AcceptedContent-Type: application/vnd.api+jsonContent-Location: https://example.com/photos/queue-jobs/5234{"data": {"type":"queue-jobs","id":"5234","attributes": {"status":"Pending request, waiting other process"   },"links": {"self":"/photos/queue-jobs/5234"   } }}

--json:api

My implementation to handle long running create endpoint

To follow that recommendation from the json:api specs if implemented it as following:

Long running create endpoint

fromdjango_celery_results.modelsimportTaskResultfromrest_framework_json_api.viewsimportModelViewSetclassOgcServiceViewSet(ModelViewSet):queryset=OgcService.objects.all()serializer_classes= {'default':OgcServiceSerializer,'create':OgcServiceCreateSerializer    }filterset_fields= {'id': ('exact','lt','gt','gte','lte','in'),'title': ('icontains','iexact','contains'),    }search_fields= ('id','title',)defget_serializer_class(self):returnself.serializer_classes.get(self.action,self.serializer_classes['default'])defcreate(self,request,*args,**kwargs):serializer=self.get_serializer(data=request.data)serializer.is_valid(raise_exception=True)task=build_ogc_service.delay(data=serializer.data)task_result,created=TaskResult.objects.get_or_create(task_id=task.id)serialized_task_result=TaskResultSerializer(task_result)serialized_task_result_data=serialized_task_result.data# meta object is None... we need to set it to an empty dict to prevend uncaught runtime exceptionsmeta=serialized_task_result_data.get("meta",None)ifnotmeta:serialized_task_result_data.update({"meta": {}})headers=self.get_success_headers(serialized_task_result_data)returnResponse(serialized_task_result_data,status=status.HTTP_202_ACCEPTED,headers=headers)defget_success_headers(self,data):try:return {'Content-Location':str(data[api_settings.URL_FIELD_NAME])}except (TypeError,KeyError):return {}

Create serializer with different field set

classOgcServiceCreateSerializer(ModelSerializer):# TODO: implement included serializer for ServiceAuthentication# included_serializers = {#     'auth': ServiceAuthentication,# }classMeta:model=OgcServicefields= ("get_capabilities_url", )

TaskResult serializer

fromdjango_celery_results.modelsimportTaskResultfromregistry.models.jobsimportRegisterOgcServiceJobfromrest_framework_json_api.serializersimportModelSerializerclassTaskResultSerializer(ModelSerializer):classMeta:model=TaskResultfields="__all__"

Response

The create enpoint is creating the celery job fine and response with the content like bellow:

Response Body

{"data": {"type":"OgcService","id":"8","attributes": {"task_id":"23e945e2-4c63-45bf-a76d-b64062db930e","task_name": null,"task_args": null,"task_kwargs": null,"status":"PENDING","worker": null,"content_type":"","content_encoding":"","result": null,"date_created":"2021-11-22T09:34:11.940341+01:00","date_done":"2021-11-22T09:34:11.940404+01:00","traceback": null,"meta": {}    }  }}

Response headers

 allow: GET,POST,HEAD,OPTIONS  content-encoding: gzip  content-language: en  content-length: 238  content-type: application/vnd.api+json  referrer-policy: same-origin  server-timing: SQLPanel_sql_time;dur=3.191709518432617;desc="SQL 2 queries"  vary: Accept-Language,Cookie,Accept-Encoding,Origin  x-content-type-options: nosniff  x-frame-options: DENY

Quesion

  1. As you can see in the response body, the type differs to the resource type of the used serializer. How can dynamically set the type to the correct"type": "TaskResult" ?

  2. How can i addurl field to provide resource urls as meta information on theTaskResultSerializer without naming all fields explicit?

You must be logged in to vote

Replies: 2 comments 4 replies

Comment options

I did not notice before that there is a recommendation from JSON:API spec on async calls... Great!

  1. As far as I understand your example it would be enough if you setresource_name in the create serializer like the following.
classOgcServiceCreateSerializer:classMeta:resource_name='TaskResult'
  1. I hope I understand you correctly here. DJA removes the URLs from fields as this is presented in the links (seehere). You can write a custom metadata class which overwritesget_serializer_info though. To follow the JSON:API spec recommendation though you would need to overwriteget_links in your view to set the correctself link.
You must be logged in to vote
4 replies
@jokiefer
Comment options

  1. As far as I understand your example it would be enough if you set resource_name in the create serializer like the following.

That did not properly work. I defined theresource_name onOgcServiceCreateSerializer and/or onTaskResultSerializer. The endpoint will always response with:

{"errors": [    {"detail":"The resource object's type (OgcService) is not the type that constitute the collection represented by the endpoint (TaskResult).","status":"409","source": {"pointer":"/data"      },"code":"error"    }  ]}

on given POST data:

{"data": {"attributes": {"get_capabilities_url":"http://geo5.service24.rlp.de/wms/karte_rp.fcgi?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities"},"relationships": {"owned_by_org": {"data": {"id":"b2fdc463-87cb-4deb-b39b-187f7616d1df","type":"Organization"}}},"type":"OgcService"}}

This is correct, causeJSON:API spec defines the following:

A server MUST return 409 Conflict when processing a POST request in which the resource object’s type is not among the type(s) that constitute the collection represented by the endpoint.

But i did not find a constraint in the JSON:API spec that does not allow on a endpoint to response with a different resource object type (TaskResult). Here is a discussion on thejsonapi.org forum, which handles about that.

So for my request the response looks like (with mismatching resource object type) :

{"data": {"type":"OgcService","id":"9","attributes": {"task_meta": null,"task_id":"d8146e14-d1ef-46d5-a6d2-be0de7017703","task_name": null,"task_args": null,"task_kwargs": null,"status":"PENDING","worker": null,"content_type":"","content_encoding":"","result": null,"date_created":"2021-11-24T09:12:35.056767+01:00","date_done":"2021-11-24T09:12:35.056860+01:00","traceback": null    },"links": {"self":"http://testserver/api/v1/registry/task-results/9/"    }  }}

Here is the link to the current implemented code:https://github.com/mrmap-community/mrmap/blob/a215b1c11bfff1f17d13c27d3e7b3ca88c3e3f87/backend/registry/api/views/service.py#L108

  1. I hope I understand you correctly here. DJA removes the URLs from fields as this is presented in the links (see here). You can write a custom metadata class which overwrites get_serializer_info though. To follow the JSON:API spec recommendation though you would need to overwrite get_links in your view to set the correct self link.

OK - i understood.
My solution was it to add the field asHyperlinkedIdentityField:

classWebMapServiceSerializer(ModelSerializer):url=HyperlinkedIdentityField(view_name='registry:taskresult-detail',    )classMeta:model=WebMapServicefields="__all__"

I struggled a bit with the view_name, cause i forgot to pass the app_name as well.

@sliverc
Comment options

True this error message makes sense. If we wanna support this async recommendation we properly would need a special case when response status code is 202 which than also handlesContent-Location. I haven't investigated how this could be accomplished yet. Suggestions are welcome.

@jokiefer
Comment options

I think the handling of theresource_name andContent-Location could be done with a little switch in the renderer:

classJSONRenderer(renderers.JSONRenderer):    ...defrender(self,data,accepted_media_type=None,renderer_context=None):ifserializerisnotNone:# Extract root meta for any type of serializerjson_api_meta.update(self.extract_root_meta(serializer,serializer_data))ifgetattr(serializer,"many",False):                ...else:                ...ifresponse.status_code==202:# handle async processing as recommended https://jsonapi.org/recommendations/#asynchronous-processingresource_name=utils.get_resource_type_from_serializer(serializer)response.headers.update({'Content-Location':serializer_data[api_settings.URL_FIELD_NAME]})resource_instance=serializer.instancejson_api_data=self.build_json_resource_obj(fields,serializer_data,resource_instance,resource_name,serializer,force_type_resolution,                )
@sliverc
Comment options

Hmm not so sure. During parsing wouldn't this still lead to the same error as whenresource_name of the created serializer is set differently? Also in terms of features the renderer is already fairly complex and we need to work on that to make it easier and more maintainable. So I want to try to avoid making it more complex especially when adding optional features like async support. An idea (not thought through yet) but what about putting the support of async calls into aAsyncResponse class?

Comment options

Additionally as a workaround what you could do is setresource_name toFalse in yourOgcServiceCreateSerializer. This means DJA rendering will be skipped and the data will be returned as you pass it on toResponse. This means in your code you have to structure the data as it is required by the JSON:API async recommendation. Not ideal but should work as a workaround.

You must be logged in to vote
0 replies
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Category
Q&A
Labels
None yet
2 participants
@jokiefer@sliverc
[8]ページ先頭
©2009-2025 Movatter.jp