Repository files navigation

This is a example of distributedJSON document database using a SQL-like query language created during a school lesson.

It takes inspiration fromCosmosDB. There are containers, and containers have documents.The distribution is achieved by using a partition key to split the data between different engines.

Everything run locally usingDocker andDocker Compose so it's not actually distributed in the current state.

The project is divided in two main components:

• Gateway
• Engine

• Responsible to store and query the data.
• One for each partition key value (independently of container)
• Each engine has a volume mounted to store the data
• Hidden from the clients

• Responsible to route the requests to the correct engine
• Creates engine containers when needed
• Exposed to the clients
• Use SQLite to store container metadata

From the root of the project, run the following commands:

docker build -t ddsql_engine -f src/Engine/Dockerfile.      docker compose up

TheGateway will be available athttp://localhost:8080.

TheOpenAPI definition can be found at/openapi/v1.json andaSwagger UI at/swagger.

Both projects areNET9minimal API. The use extension methods forRouteGroupBuilder to split the logic in different files, and to take ruoting more readable:

app.MapGroup("containers").MapCreateContainer().MapGetContainers().MapQueryContainer();app.MapGroup("containers/{container}/documents").MapUpsertDocument().MapGetDocument();

The query is converted to anAST (Abstract Syntax Tree) so that:

• TheGateway can extract the partition key value from the query
• TheEngine can execute the query

The gateway, after finding the partition key value, send theAST to the correct execute it.

TheGateway usesHttpClient to communicate with theEngine.TheGateway doesn't need to deserialize the answer, so it write the response stream directly to the client.

publicstaticasyncTask<IResult>ProxyAsync(thisHttpContexthttpContext,HttpResponseMessageresponse){httpContext.Response.StatusCode=(int)response.StatusCode;httpContext.Response.ContentType=response.Content.Headers.ContentType?.ToString();awaitresponse.Content.CopyToAsync(httpContext.Response.Body);returnTypedResults.Empty;}

It usesSHA256 to hash the partition key value and used to find the correct engine.