Repository files navigation

中文文档 |Documentation

Fast, Simple, Secure

Download compiled executablego-judge for your platform fromRelease and run.

Or, by docker

docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge criyle/go-judge

A REST service to run program in restricted environment (Listening onlocalhost:5050 by default).

• POST /run execute program in the restricted environment
• GET /file list all cached file id to original name map
  • POST /file prepare a file in the go judge (in memory), returns fileId (can be referenced in /run parameter)
  • GET /file/:fileId downloads file from go judge (in memory), returns file content
  • DELETE /file/:fileId delete file specified by fileId
• /ws WebSocket version for /run
• /stream WebSocket for stream run. Supports streaming interface
• GET /version gets build git version (e.g.v1.9.0) together with runtime information (go version, os, platform)
  • GET /config gets some configuration (e.g.fileStorePath,runnerConfig) together with some supported features

API Interface Structure Definition

Example Request & Response

• Linux Kernel Version >= 3.10
• Cgroup file system mounted at /sys/fs/cgroup. Usually done by systemd

+----------------------------------------------------------------------------------+| Transport Layer (HTTP / WebSocket / FFI / ...)                                   |+----------------------------------------------------------------------------------+| Sandbox Worker (Environment Pool w/ Environment Builder )                        |+-----------------------------------------------------------+----------------------+| EnvExec                                                   | File Store           |+--------------------+----------------+---------------------+---------------+------+| Linux (go-sandbox) | Windows (winc) | macOS (app sandbox) | Shared Memory | Disk |+--------------------+----------------+---------------------+---------------+------+

Server:

• The default binding address for the go judge islocalhost:5050. Can be specified with-http-addr flag.
• By default gRPC endpoint is disabled, to enable gRPC endpoint, add-enable-grpc flag.
  • The default binding address for the gRPC go judge islocalhost:5051. Can be specified with-grpc-addr flag.
• The default log level is info, use-silent to disable logs or use-release to enable release logger (auto turn on if in docker).
• -auth-token to add token-based authentication to REST / gRPC
• By default, the GO debug endpoints (localhost:5052/debug) are disabled, to enable, specifies-enable-debug, and it also enables debug log
• By default, the prometheus metrics endpoints (localhost:5052/metrics) are disabled, to enable, specifies-enable-metrics
• Monitoring HTTP endpoint is enabled if metrics / debug is enabled, the default addr islocalhost:5052 and can be specified by-monitor-addr

Sandbox:

• The default concurrency equal to number of CPU, Can be specified with-parallelism flag.
• -mount-conf specifies detailed mount configuration, please referFile System Mount as a reference (Linux only)
• -file-timeout specifies maximum TTL for file created in file store （e.g.30m)
• The default file store is in memory(/dev/shm/), local cache can be specified with-dir flag.
• -output-limit specifies size limit of POSIX rlimit of output (default 256MiB)
• -copy-out-limit specifies the default file copy out max (default 64MiB)

You can findmore available configuration here.

Downloadgo-judge-shell fromRelease and run. It will connect localgo-judge, and open an interactive shell in the container for debugging purpose.

• Accepted: Program exited with status code 0 within time & memory limits
• Memory Limit Exceeded: Program uses more memory than memory limits
• Time Limit Exceeded: (exitStatus usually have value9 as killed bySIGKILL after timeout)
  • Program uses more CPU time than cpuLimit
  • Or, program uses more clock time than clockLimit
• Output Limit Exceeded:
  • Program output more than pipeCollector limits
  • Or, program output more than output-limit
• File Error:
  • CopyIn file is not existed
  • Or, CopyIn file too large for container file system
  • Or, CopyOut file is not existed after program exited
• Non Zero Exit Status: Program exited with non 0 status code within time & memory limits
• Signalled: Program exited with signal (e.g.SIGSEGV)
• Dangerous Syscall: Program killed by seccomp filter (not enabled by default)
• Internal Error:
  • Program is not exist
  • Or, container create not successful (e.g. not privileged docker)
  • Or, other errors

For linux platform, the default mounts points are bind mounting host's/lib,/lib64,/usr,/bin,/etc/ld.so.cache,/etc/alternatives,/etc/fpc.cfg,/dev/null,/dev/urandom,/dev/random,/dev/zero,/dev/full and mounts tmpfs at/w,/tmp and creates/proc.

Tocustomize mount points, please look at examplemount.yaml file.

Ifmount.yaml is not specified, the size oftmpfs for/w and/tmp is configured through-tmp-fs-param with default valuesize=128m,nr_inodes=4k

If a file named/.env exists in the container rootfs, the container will load the file as environment variable line by line.

If a bind mount is specifying a target within the previous mounted one, please ensure the target exists in the previous mount point.

Prometheus Metrics Monitoring Endpoint

For cgroup v1, thego-judge need root privilege to createcgroup. Either creates sub-directory/sys/fs/cgroup/cpuacct/gojudge,/sys/fs/cgroup/memory/gojudge,/sys/fs/cgroup/pids/gojudge and make execution user readable or usesudo to run it.

For cgroup v2, systemd dbus will be used to create a transient scope for cgroup integration.

If no permission to create cgroup, the cgroup related limit will not be effective.

The cgroup v2 is supported bygo-judge now when running as root since more Linux distribution are enabling cgroup v2 by default (e.g. Ubuntu 21.10+, Fedora 31+). However, for kernel < 5.19, due to missingmemory.max_usage_in_bytes inmemory controller, the memory usage is now accounted bymaxrss returned bywait4 syscall. Thus, the memory usage appears higher than those who uses cgroup v1. For kernel >= 5.19,memory.peak is being used.

When running in containers, thego-judge will migrate all processed into/api hierarchy to enable nesting support.

When running in Linux distributions powered bysystemd, thego-judge will contactsystemd viadbus to create a transient scope as cgroup root.

When running with kernel >= 5.7, thego-judge will try fasterclone3(CLONE_INTO_CGROUP) andvfork method.

The controller usually consumes20M memory and each container usually consumes20M + size of tmpfs2 * 128M. For each request, it consumes as much as user program limit + extra limit (16k) + total copy out max. Notice that the cached file stores in the shared memory (/dev/shm) of the host, so please ensure enough size allocated.

For example, when concurrency = 4, the container itself can consume as much as60 + (20+32) * 4M = 268M + 4 * total copy out + total max memory of requests.

Due to limitation of GO runtime, the memory will not return to OS automatically, which could lead to OOM killer. A background worker was introduced to checks heap usage and invokes GC when necessary.

• -force-gc-target default20m, the minimal size to trigger GC
• -force-gc-interval default5s, the interval to check memory usage