Repository files navigation

Most modern networking is built either on slow and ambiguous REST APIs or unnecessarily complex gRPC.FastAPI, for example, looks very approachable.We aim to be equally or even simpler to use.

pip install fastapi uvicorn

pip install ucall

fromfastapiimportFastAPIimportuvicornserver=FastAPI()@server.get('/sum')defsum(a:int,b:int):returna+buvicorn.run(...)

fromucall.posiximportServer# from ucall.uring import Server on 5.19+server=Server()@serverdefsum(a:int,b:int):returna+bserver.run()

It takes over a millisecond to handle a trivial FastAPI call on a recent 8-core CPU.In that time, light could have traveled 300 km through optics to the neighboring city or country, in my case.How does UCall compare to FastAPI and gRPC?

How can a tiny pet-project with just a couple thousand lines of code compete with two of the most established networking libraries?UCall stands on the shoulders of Giants:

• io_uring for interrupt-less IO.

  • io_uring_prep_read_fixed on 5.1+.
  • io_uring_prep_accept_direct on 5.19+.
  • io_uring_register_files_sparse on 5.19+.
  • IORING_SETUP_COOP_TASKRUN optional on 5.19+.
  • IORING_SETUP_SINGLE_ISSUER optional on 6.0+.

• SIMD-accelerated parsers with manual memory control.

  • simdjson to parse JSON faster than gRPC can unpackProtoBuf.
  • Turbo-Base64 to decode binary values from aBase64 form.
  • picohttpparser to navigate HTTP headers.

You have already seen the latency of the round trip..., the throughput in requests per second..., want to see the bandwidth?Try yourself!

@serverdefecho(data:bytes):returndata

FastAPI supports native type, while UCall supportsnumpy.ndarray,PIL.Image and other custom types.This comes handy when you build real applications or want to deploy Multi-Modal AI, like we do withUForm.

fromucall.rich_posiximportServerimportuformserver=Server()model=uform.get_model('unum-cloud/uform-vl-multilingual')@serverdefvectorize(description:str,photo:PIL.Image.Image)->numpy.ndarray:image=model.preprocess_image(photo)tokens=model.preprocess_text(description)joint_embedding=model.encode_multimodal(image=image,text=tokens)returnjoint_embedding.cpu().detach().numpy()

We also have our own optionalClient class that helps with those custom types.

fromucall.clientimportClientclient=Client()# Explicit JSON-RPC call:response=client({'method':'vectorize','params': {'description':description,'image':image,    },'jsonrpc':'2.0','id':100,})# Or the same with syntactic sugar:response=client.vectorize(description=description,image=image)

Aside from the PythonClient, we provide an easy-to-use Command Line Interface, which comes withpip install ucall.It allow you to call a remote server, upload files, with direct support for images and NumPy arrays.Translating previous example into a Bash script, to call the server on the same machine:

ucall vectorize description='Product description' -i image=./local/path.png

To address a remote server:

ucall vectorize description='Product description' -i image=./local/path.png --uri 0.0.0.0 -p 8545

To print the docs, useucall -h:

usage: ucall [-h] [--uri URI] [--port PORT] [-f [FILE ...]] [-i [IMAGE ...]] [--positional [POSITIONAL ...]] method [kwargs ...]UCall Client CLIpositional arguments:  method                method name  kwargs                method argumentsoptions:  -h, --help            show this help message and exit  --uri URI             server uri  --port PORT           server port  -f [FILE ...], --file [FILE ...]                        method positional arguments  -i [IMAGE ...], --image [IMAGE ...]                        method positional arguments  --positional [POSITIONAL ...]                        method positional arguments

You can also explicitly annotate types, to distinguish integers, floats, and strings, to avoid ambiguity.

ucall auth id=256ucall auth id:int=256ucall auth id:str=256

We will leave bandwidth measurements to enthusiasts, but will share some more numbers.The general logic is that you can't squeeze high performance from Free-Tier machines.Currently AWS provides following options:t2.micro andt4g.small, on older Intel and newer Graviton 2 chips.This library is so fast, that it doesn't need more than 1 core, so you can run a fast server even on a tiny Free-Tier server!

In this case, every server was bombarded by requests from 1 or a fleet of 32 other instances in the same availability zone.If you want to reproduce those benchmarks, check out thesum examples on GitHub.

For Python:

pip install ucall

For CMake projects:

include(FetchContent)FetchContent_Declare(    ucall    GIT_REPOSITORY https://github.com/unum-cloud/ucall    GIT_SHALLOWTRUE)FetchContent_MakeAvailable(ucall)include_directories(${ucall_SOURCE_DIR}/include)

The C usage example is mouthful compared to Python.We wanted to make it as lightweight as possible and to allow optional arguments without dynamic allocations and named lookups.So unlike the Python layer, we expect the user to manually extract the arguments from the call context withucall_param_named_i64(), and its siblings.

#include<cstdio.h>#include<ucall/ucall.h>staticvoidsum(ucall_call_tcall,ucall_callback_tag_t) {int64_ta{},b{};charprinted_sum[256]{};boolgot_a=ucall_param_named_i64(call,"a",0,&a);boolgot_b=ucall_param_named_i64(call,"b",0,&b);if (!got_a|| !got_b)returnucall_call_reply_error_invalid_params(call);intlen=snprintf(printed_sum,256,"%ll",a+b);ucall_call_reply_content(call,printed_sum,len);}intmain(intargc,char**argv) {ucall_server_tserver{};ucall_config_tconfig{};ucall_init(&config,&server);ucall_add_procedure(server,"sum",&sum,NULL);ucall_take_calls(server,0);ucall_free(server);return0;}

• Batch Requests
• JSON-RPC over raw TCP sockets
• JSON-RPC over TCP with HTTP
• Concurrent sessions
• NumPyarray and Pillow serialization
• HTTPS support
• Batch-capable endpoints for ML
• Zero-ETL relay calls
• Integrating withUKV
• WebSockets for web interfaces
• AF_XDP and UDP-based analogs on Linux

Want to affect the roadmap and request a feature? Join the discussions on Discord.

• Transport independent: UDP, TCP, bring what you want.
• Application layer is optional: use HTTP or not.
• Unlike REST APIs, there is just one way to pass arguments.