Making kernels for Jupyter#
A ‘kernel’ is a program that runs and introspects the user’s code. IPythonincludes a kernel for Python code, and people have written kernels forseveral other languages.
At kernel startup, Jupyter passes the kernel a connection file. This specifieshow to set up communications with the frontend.
There are three options for writing a kernel:
You can reuse the IPython kernel machinery to handle the communications, andjust describe how to execute your code. This is much simpler if the targetlanguage can be driven from Python. SeeMaking simple Python wrapper kernels for details.
You can implement the kernel machinery in your target language. This is morework initially, but the people using your kernel might be more likely tocontribute to it if it’s in the language they know.
You can use thexeus library that isa C++ implementation of the Jupyter kernel protocol. Kernel authors only need toimplement the language-specific logic in their implementation(execute code, auto-completion…). This is the simplestsolution if your target language can be driven from C or C++: e.g. if it hasa C-API like most scripting languages. Check out thexeus documentation for more details.Examples of kernels based on xeus include:
Connection files#
Your kernel will be given the path to a connection file when it starts (seeKernel specs for how to specify the command line arguments for your kernel).This file, which is accessible only to the current user, will contain a JSONdictionary looking something like this:
{"control_port":50160,"shell_port":57503,"transport":"tcp","signature_scheme":"hmac-sha256","stdin_port":52597,"hb_port":42540,"ip":"127.0.0.1","iopub_port":40885,"key":"a0436f6c-1916-498b-8eb9-e81ab9368e84"}
Thetransport,ip and five_port fields specify five ports which thekernel should bind to usingZeroMQ. For instance, theaddress of the shell socket in the example above would be:
tcp://127.0.0.1:57503
New ports are chosen at random for each kernel started.
signature_scheme andkey are used to cryptographically sign messages, sothat other users on the system can’t send code to run in this kernel. SeeThe Wire Protocol for the details of how this signature is calculated.
Handling messages#
After reading the connection file and binding to the necessary sockets, thekernel should go into an event loop, listening on the hb (heartbeat), controland shell sockets.
Heartbeat messages should be echoed back immediatelyon the same socket - the frontend uses this to check that the kernel is stillalive.
Messages on the control and shell sockets should be parsed, and their signaturevalidated. SeeThe Wire Protocol for how to do this.
The kernel will send messages on the iopub socket to display output, and on thestdin socket to prompt the user for textual input.
See also
- Messaging in Jupyter
Details of the different sockets and the messages that come over them
- Creating Language Kernels for IPython
A blog post by the author ofIHaskell,a Haskell kernel
- simple_kernel
A simple example implementation of the kernel machinery in Python
Kernel specs#
A kernel identifies itself to IPython by creating a directory, the name of whichis used as an identifier for the kernel. These may be created in a number oflocations:
Unix | Windows | |
|---|---|---|
System |
|
|
Env |
| |
User |
|
|
The user location takes priority over the system locations, and the case of thenames is ignored, so selecting kernels works the same way whether or not thefilesystem is case sensitive.Since kernelspecs show up in URLs and other places,a kernelspec is required to have a simple name, only containing ASCII letters,ASCII numbers, and the simple separators:- hyphen,. period,_underscore.
Other locations may also be searched if theJUPYTER_PATH environmentvariable is set.
Inside the kernel directory, three types of files are presently used:kernel.json,kernel.js, and logo image files. Currently, no otherfiles are used, but this may change in the future.
Inside the directory, the most important file iskernel.json. This should be aJSON serialised dictionary containing the following keys and values:
argv: A list of command line arguments used to start the kernel. The text
{connection_file}in any argument will be replaced with the path to theconnection file.display_name: The kernel’s name as it should be displayed in the UI.Unlike the kernel name used in the API, this can contain arbitrary unicodecharacters.
language: The name of the language of the kernel.When loading notebooks, if no matching kernelspec key (may differ across machines)is found, a kernel with a matching
languagewill be used.This allows a notebook written on any Python or Julia kernel to be properly associatedwith the user’s Python or Julia kernel, even if they aren’t listed under thesame name as the author’s.interrupt_mode (optional): May be either
signalormessageandspecifies how a client is supposed to interrupt cell execution on this kernel,either by sending an interruptsignalvia the operating system’ssignalling facilities (e.g.SIGINTon POSIX systems), or by sending aninterrupt_requestmessage on the control channel (seeKernel interrupt). If this is not specifiedthe client will default tosignalmode.env (optional): A dictionary of environment variables to set for the kernel.These will be added to the current environment variables before the kernel isstarted. Existing environment variables can be referenced using
${<ENV_VAR>}andwill be substituted with the corresponding value. Administrators should note that useof${<ENV_VAR>}can expose sensitive variables and should use only in controlledcircumstances.metadata (optional): A dictionary of additional attributes about thiskernel; used by clients to aid in kernel selection. Metadata addedhere should be namespaced for the tool reading and writing that metadata.
For example, the kernel.json file for IPython looks like this:
{"argv":["python3","-m","IPython.kernel","-f","{connection_file}"],"display_name":"Python 3","language":"python"}
To see the available kernel specs, run:
jupyterkernelspeclist
To start the terminal console or the Qt console with a specific kernel:
jupyterconsole--kernelbashjupyterqtconsole--kernelbash
The notebook offers you the available kernels in a dropdown menu from the ‘New’button.
Packaging#
To release your kernel as a Python package, we recommend following the patternused in theecho_kernel, which uses thehatch build backend anda build file that creates the kernel directory with thekernel.json andkernel icons, which is included asshared-data, ending up in theshare/jupyter/kernels/ folder in the user’s installed environment.Seepyproject.toml andhatch_build.py for more details.