New child process communication involves 3 different things:

1. Lifetimes
2. Rd entities
3. Rdgen

Let's dive in each of them:

Imagine an object having some resources that should be freedwhen object dies. For this purpose Java introduced interfacesCloseable\AutoCloseable. Also, 2 helper functions were introduced: try-with-resources and Kotlin'sCloseable.use. There are several problems:

1. Object's lifetime can be more complicated thatCloseable.use scope.
2. If function parameter isCloseable - should you close it?
3. Multiple closes.
4. Concurrent closes.
5. If you have several objects that depends on another object's lifetime - how to correctly close all of them with respect to issues 1-4? How to make it simple, convenient and fast?

And so Lifetime was introduced.

Lifetime is a class, where you can register callbacks and which can be terminated once, thus executing all registered callbacks.

Lifetime is an abstract class, it's inheritor -LifetimeDefinition. The only difference - onlyLifetimeDefinition can be terminated. Though allLifetime are instances ofLifetimeDefinition, there are some conventions:

1. Do not castLifetime toLifetimeDefinion unless you are the one who createdLifetimeDefinition.
2. If you introduce somewhereLifetimeDefinition - either attach it to anotherLifetime or provide code that terminates it.

UsefulLifetime methods:

• onTermination - Executes lambda/closeable when lifetime terminates. If already terminated - executes instantly. Termination will proceed on thread that calledLifetimeDefinition.terminate(). Callbacks will be executed inreversed order, that is LIFO - last added callback will be executed first.
• onTerminationIfAlive - same asOnTermination, but callback will not be executed if lifetime is not alive.
• executeIfAlive - executes lambda if lifetime is alive. This method guarantees that lifetime will not be terminated until lambda completes.
• createdNested - creates child LifetimeDefinition that will be terminated if parent does.
• usingNested - same ascreateNested, but likeCloseable.use pattern.
• Eternal - lifetime that never terminates.
• Terminated - lifetime that already terminated.
• status - seeLifetimeStatus.kt in RD repo. There are 3 convenient method:IsAlive, IsNotAlive, IsTerminated.

• terminate - terminatesLifetime and calls all callbacks. Sometimes if multiple concurrent terminations occurred - method will return before executing all callbacks because some other thread is doing this.

Rd is a cross-language, cross-platform, light-weight, reactive, one-to-one rpc protocol. Can work either on the same or different machines via internet.

Useful entities:

• Protocol - encapsulation of all logic regarding rd communication. All entities should be bound to protocol before using. ContainsIScheduler which executes runnables on different thread.

• RdSignal - entity forfire and forget. You can add callback for every received message viaadvise(lifetime, callback) method. There are 2 interfaces -ISink which allows only to advise for messages, andISignal which can alsofire events. Also, there is justSignal class with same behaviour, but without remote communication.

  Important: if you advise and fire from the same process - your callback will receivenot only messages from another process, but also the ones you fire.

• RdProperty - stateful property. You can get current value, you can advise - advised callback will be executed on current value and on every change.

• RdCall - remote procedure call.

Also there areRdSet,RdMap and many other.

There isasync property that allows you tofire entities from any thread. Otherwise you would need to do it fromProtocol.scheduler thread. All rd entities should be at first

Generates custom classes and requests which can be bound to protocol and advised. There is special model DSL for it.

Eexample:

1. Korifey - quite simple
2. Rider Unity plugin - complicated one

First you need to defineRoot object - only one instance of each root can be assigned to protocol.

Then there is root extensionExt(YourRoot) where you can define your own types and model entities. You can assign multiple extensions of root for the protocol. Auxillary structures can be defined as direct fields - they will be also generated.

DSL:

• structdef - structure with fields that cannot be bound to protocol, but can be serialized. Can be open for inheritace -openstruct, can be abstract -basestruct. Can have onlyfield as member.

• classdef - class that can be bould to model. Can also haveproperty,signal,call e.t.c. as members. It is possible to do inheritance -openclass,baseclass.

• interfacedef - to define interfaces. Usemethod to create signature.

  You can useextends andimplements to work with inheritance.

  N.B. - rdgen can generate models also for C# and C++. Their structs and classes have a little different behaviour.

• Rd entities - only in bindable models(Ext,classdef):

  • property
  • signal
  • source
  • sink
  • array andimmutablelist

• Useful properties in dsl entities:

  • async - asasync in RD entities
  • docs - provides kdoc/javadoc for generated entity

Example

RdGenExtension configurates Rdgen, useful properties:

• sources - folders with dsl .kt files. If not present, scan classpath for inheritors ofRoot andExt.
• hashfile - folder to store hash file.rdgen for incremental generation.
• packages - java package names to search toplevels, delimited by ','. Example:com.jetbrains.rd.model.nova,com,org.
• Configuring model generation with methodRdGenExtension.generator:

  • root - for which root you are declaring this generator

  • namespace - which namespace should be used in generated source. In kotlin it configures generated packaged name.

  • directory - where to put generated files.

  • transform - can besymmetric,asis andreversed. This allows to configure model interface differently for client-server scenarios.

    P.S. This is legacy from distant past, in 99% of time you should usesymmetric. In 1% chance - consult with somebody if you really need another option.

  • language - can bekotlin,cpp andcsharp.

There is another gradle projectutbot-rd which contains model sources inrdgenModels sources. Look fororg.utbot.rd.models.ProtocolRoot.

Usefull:

1. if you need to use rd somewhere - add following dependencies:

   implementation group: 'com.jetbrains.rd', name: 'rd-framework', version: 'actual.version'implementation group: 'com.jetbrains.rd', name: 'rd-core', version: 'actual.version'

2. There are some usefull classes to work with processes & rd:

   • LifetimedProcess - bindsLifetime to process. If process dies - lifetime terminates and vice versa. You can terminate lifetime manually - this will destroy process.
   • ProcessWithRdServer - also starts Rd server and waits for connection.
   • UtInstrumentationProcess - encapsulates logic for preparing child process for executing arbitary commands. ExposesprotocolModel for communicating with child process.
   • ConcreteExecutor is convenient wrapper for executing commands and managing resources.

3. How child communication works:

   • Choosing free port
   • Creating child process, passing port as argument
   • Both processes create protocols and bind model
   • Child process setups all callbacks
   • Parent process cannot send messages before child creates protocol, otherwise messages will be lost. So child process needs to signal that he is ready.
   • Child proces creates special file in temp dir, that is observed by parent process.
   • When parent process spots file - he deletes it, and then sends special message for preparing child proccess instrumentation
   • Only then process is ready for executing commands

4. How to write custom commands for child process

   • Add newcall inProtocolModel
   • Regenerate models
   • Add callback for newcall inChildProcess.kt
   • UseConcreteExecutor.withProcess method
   • Important - do not addRdgen as implementation dependency, it breaks some.jars as it containskotlin-compiler-embeddable.

5. Logs

   There isUtRdLogger where you can configure level vialog4j2.xml.

6. Custom protocol marshalling types

   Do not spend time on it until:

   • Cyclic dependencies removed from UtModels
   • Kotlinx.serialization is used