Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork2.1k
Description
Describe the bug
I built a request to upload files to a local cloud. I detected that if there is a page redirection (307 Temporary Redirect) during the request, the followingaiohttp request's body is built incorrectly, resulting in unexpected malformed requests.
For example,FastAPI servers by default accept URLs with and without trailing slashes, redirecting to the right path if a slash is miswritten. But I get 4xx errors after this redirect. If bytes are passed instead of a file stream or the right path is written (without the redirect), the server gets the correct file inside the request body.
To Reproduce
server.py - small FastAPI server
fromfastapiimportFastAPI,UploadFileapp=FastAPI()@app.api_route("/files",methods=["DELETE","PATCH","POST","PUT"])defupdate_file(file:UploadFile|None=None):# Working with the filereturnfile.sizeiffileelse0if__name__=="__main__":fromuvicornimportrunrun("server:app",port=8000,reload=True)
aiohttp_request.py - I added my tests as commented lines
fromasyncioimportrunfromioimportBufferedReader,BytesIO,FileIO,StringIO,TextIOWrapperfromtypingimportAsyncGeneratorfromaiohttpimportClientSession,FormDataasyncdeffile_chunks(*args,**kwargs)->AsyncGenerator[bytes,None]:importaiofilesasyncwithaiofiles.open(*args,**kwargs)asfile:whilechunk:=awaitfile.read(64*1024):yieldchunkasyncdefupload_file():filename="server.py"# url = "http://localhost:8000/files" # all good, always worksurl="http://localhost:8000/files/"# doesn't work after the FastAPI redirectwithopen(filename,"rb")asdata:# with FileIO(filename) as data:# with BufferedReader(FileIO(filename)) as data:# with TextIOWrapper(BufferedReader(FileIO(filename))) as data:# with BytesIO(b"test") as data: # works for both urls# with StringIO("test") as data: # works for both urlsformdata=FormData({"file":data})# doesn't work with file streams# formdata = FormData({"file": data.read()}) # works, but passing bytes is deprecated# formdata = FormData(); formdata.add_field("file", file_chunks(filename, "rb"), filename=filename) # doesn't work with redirects as it is a generator# formdata = None # works as there is no passed dataasyncwith (ClientSession()assession,session.post(url,data=formdata)asresp, ):print(awaitresp.text())resp.raise_for_status()if__name__=="__main__":run(upload_file())
After running the request, the output is[422 Unprocessable Entity] {"detail":[{"loc":["body","file"],"msg":"field required","type":"value_error.missing"}]} (or sometimes[400 Bad Request] Invalid HTTP request received.)
I even made a try to write an alternativeaiohttp web server implementation, but it still leads to errors, though they are different:
alternative_server.py
alternative_server.pyfromaiohttpimportwebasyncdefupdate_file(request):data=awaitrequest.post()print(data)file_field=data.get("file")# Working with the filesize=len(file_field.file.read())iffile_fieldelse0returnweb.Response(text=str(size))app=web.Application(middlewares=[web.normalize_path_middleware(remove_slash=True,append_slash=False)])app.add_routes([web.route("*","/files",update_file)])if__name__=="__main__":web.run_app(app,port=8000)
Expected behavior
Behaviour is expected to be the same as without redirects. The status code[200 OK] is expected when the passed file is sent to the server, and its size is printed.
Logs/tracebacks
Server output:INFO: 127.0.0.1:65045 - "POST /files/ HTTP/1.1" 307 Temporary RedirectWARNING: 127.0.0.1:65046 - "POST /files HTTP/1.1" 400 Bad RequestClient output:Invalid HTTP request received.Traceback (most recent call last): File"C:\Users\Mike\Desktop\aiohttp_request.py", line30, in<module> run(upload_file()) File"C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\runners.py", line44, inrunreturn loop.run_until_complete(main) File"C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line649, inrun_until_completereturn future.result() File"C:\Users\Mike\Desktop\aiohttp_request.py", line26, inupload_file resp.raise_for_status() File"C:\Users\Mike\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\client_reqrep.py", line625, inraise_for_statusraise ClientResponseError(aiohttp.client_exceptions.ClientResponseError:400, message='Bad Request', url='http://localhost:8001/files'
Python Version
$python --versionPython 3.10.11
aiohttp Version
$python -m pip show aiohttpVersion: 3.12.7
multidict Version
$python -m pip show multidictVersion: 6.0.5
propcache Version
$python -m pip show propcacheVersion: 0.3.2
yarl Version
$python -m pip show yarlVersion: 1.20.1
OS
Microsoft Windows 11 Pro (10.0.26100)
Related component
Client
Additional context
$python -m pip show fastapiVersion: 0.115.12
As I can guess from my own tests, the problem somehow lies withinaiohttp.payload.IOBasePayload:write_with_length method, as there are no problems with other payloads for binary data.
Code of Conduct
- I agree to follow the aio-libs Code of Conduct