Summary

Provide an interactive tool to evaluate declarations, statements, andexpressions of the Java programming language, together with an API sothat other applications can leverage this functionality.

Goals

The JShell API and tool will provide a way to interactively evaluatedeclarations, statements, and expressions of the Java programminglanguage within the JShell state. The JShell state includes an evolvingcode and execution state. To facilitate rapid investigation and coding,statements and expressions need not occur within a method, andvariables and method need not occur within a class.

Thejshell tool will be a command-line tool with features to easeinteraction including: a history with editing, tab-completion, automaticaddition of needed terminal semicolons, and configurable predefinedimports and definitions.

Non-Goals

A new interactive language is not the goal: All accepted input must matchgrammar productions in the Java Language Specification (JLS). Further,within an appropriate surrounding context, all accepted input must bevalid Java code (JShell will automatically provide that surroundingcontext -- the "wrapping"). That is, ifX is an input that JShellaccepts (as opposed to rejects with error) then there is anA andBsuch thatAXB is a valid program in the Java programming language.

Out of scope are graphical interfaces and debugger support. The JShell APIis intended to allow JShell functionality in IDEs and other tools, but thejshelltool is not intended to be an IDE.

Motivation

Immediate feedback is important when learning a programming language andits APIs. The number one reason schools cite for moving away from Java as ateaching language is that other languages have a "REPL" and have far lowerbars to an initial"Hello, world!" program. A Read-Eval-Print Loop (REPL) is an interactive programming tool which loops, continually reading user input, evaluating the input, and printing the value of the input or a description of thestate change the input caused. Scala, Ruby, JavaScript,Haskell, Clojure, and Python all have REPLs and all allow small initialprograms. JShell adds REPL functionality to the Java platform.

Exploration of coding options is also important for developersprototyping code or investigating a new API. Interactive evaluation isvastly more efficient in this regard than edit/compile/execute andSystem.out.println.

Without the ceremony ofclass Foo { public static void main(String[]args) { ... } }, learning and exploration is streamlined.

Description

Functionality

The JShell API will provide all of JShell's evaluation functionality.The code fragments that are input to the API are referred to as"snippets". Thejshell tool will also use the JShell completion API todetermine when input is incomplete (and the user must be prompted formore), when it would be complete if a semicolon were added (in which case thetool will append the semicolon) and also how to complete input whencompletion is requested with a tab. The tool will have a set of commandsfor query, saving and restoring work, and configuration. Commands willbe distinguished from snippets by a leading slash.

Documentation

The JShell module API Specifications can found here:

• http://download.java.net/java/jdk9/docs/api/jdk.jshell-summary.html

Which includes the primary JShell API (jdk.jshell package) Specification:

• http://download.java.net/java/jdk9/docs/api/jdk/jshell/package-summary.html

Thejshell tool reference:

• https://docs.oracle.com/javase/9/tools/jshell.htm

is part of the Java Platform, Standard Edition Tools Reference:

• https://docs.oracle.com/javase/9/tools/tools-and-command-reference.htm

Terms

In this document, the term“class” is meant in the sense used in the Java Virtual MachineSpecification (JVMS), which includes Java Language Specification (JLS) classes, interfaces, enums,and annotation types. The text makes it clear if a different meaning is intended.

Snippets

A snippet must correspond to one of the following JLS syntax productions:

• Expression
• Statement
• ClassDeclaration
• InterfaceDeclaration
• MethodDeclaration
• FieldDeclaration
• ImportDeclaration

In JShell, a "variable" is a storage location and has an associated type. A variable is created explicitly with aFieldDeclaration snippet:

int a = 42;

or implicitly by an expression (see below). Variables have a small amount of field semantics/syntax (for example,thevolatile modifier is allowed). However, variables have no user-visible class enclosing themand will be generally viewed and used like local variables.

All expressions are accepted as snippets. This includes expressions without side effects, such as constants, variable accesses, and lambda expressions:

1a2+2Math.PIx -> x+1(String s) -> s.length()

as well as expressions with side effects, such as assignments and method invocations:

a = 1System.out.println("Hello world");new BufferedReader(new InputStreamReader(System.in))

Some forms of expression snippet implicitly create a variable to store the expression's value so it can be referred to later by other snippets.By default, an implicitly created variable has the name$X whereX is the snippet identifier.A variable is not implicitly created if the expression is void (theprintln example),or if the value of the expression can already be referred to by asimple name (as in the case of 'a' and 'a=1' above; all other examples have variables implicitly created for them).

All statements are accepted as snippets, except for 'break', 'continue', and 'return'. However, a snippet may contain 'break', 'continue', or 'return' statements where they meet the usual rules of the Java programming language for enclosing context. For example, the return statement in this snippet is valid because it is enclosed in a lambda expression:

() -> { return 42; }

A declaration snippet (ClassDeclaration,InterfaceDeclaration,MethodDeclaration, orFieldDeclaration) is a snippet that explicitly introduces a namethat can be referred to by other snippets. A declaration snippet is subject to the following rules:

• the access modifiers (public,protected, andprivate) are ignored (all declaration snippets are accessible to all other snippets)
• the modifierfinal is ignored (future changes/inheritance are permitted)
• the modifierstatic is ignored (there is no user-visible containing class)
• the modifiersdefault andsynchronized are not allowed
• the modifierabstract is allowed only on classes.

All snippets, except for those of the formImportDeclaration, may contain nested declarations. For example, a snippet that is a class instance creation expression may specify an anonymous class body with nested method declarations. The usual rules of the Java programming language apply to modifiers on nested declarations, rather than the rules above. For example, the class snippet below is accepted, and the private modifier on the nested method declaration is respected, so the snippet "new C().secret()" would not be accepted:

class C {  int answer() { return 2 * secret(); }  private int secret() { return 21; }}

A snippet may not declare a package or a module. All JShell code is placed in a single package in an unnamed module. The name of the package is controlled by JShell.

Within thejshell tool, the terminal semicolon of a snippet can be omitted if that semicolon would be the last character of the input(excluding whitespace and comments).

State

The JShell state is held in an instance ofJShell. A snippet isevaluated in aJShell with theeval(...) method, producing anerror, declaring code, or executing a statement or expression. In thecase of a variable with an initializer, both declaration and executionoccur. An instance ofJShell contains previously-defined andmodified variables, methods, and classes, previously-defined importdeclarations, the side-effects of previously-entered statements andexpressions (including variable initializers), and external code bases.

Modification

Since the desired use is exploration, the declarations (variables,methods, and classes) must be able to evolve over time while, at the sametime, preserving evaluated data. One choice would be to make a changeddeclaration a new additional entity in some or all cases, but that iscertain to be confusing and does not play well with exploring theinteraction between declarations. In JShell, each unique declaration keyhas exactly one declaration at any given time. For variables and classesthe unique declaration key is the name, and,the unique declaration key for methods is the name and the parametertypes (to allow for overloading). As this is Java, variable, methods,and classes each have their own name spaces.

Forward reference

In the Java programming language, within the body of a class,references to members which will appear latercan appear; this is a forward reference. As code is enteredand evaluated sequentially in JShell, these references will betemporarily unresolved. In some cases, for example mutual recursion,forward reference is required. This can also occur in exploratoryprogramming while entering code, for example, realizing that another (sofar unwritten) method should be called. JShell supports forwardreferences in method bodies, return type, and parameter types, in variable type, and, within a class.Since the semantics requires them to be immediatelyexecuted, forward references in variable initializers is not supported.

Snippet dependencies

The code state is kept up-to-date and consistent; that is, when a snippetis evaluated, any changes to dependent snippets are immediatelypropagated.

When a snippet is successfully declared, the declaration will be one ofthree kinds:Added,Modified, orReplaced. A snippet isAdded ifit is the first declaration with that key. A snippet isReplaced ifits key matches a previous snippet, but their signatures differ. Asnippet isModified if its key matches a previous snippet and theirsignatures match; in this case, no dependent snippets are impacted. Inboth theModified andReplaced cases the previous snippet is nolonger part of the code state.

When a snippet isAdded it may be providing an unresolved reference.When a snippet isReplaced it may update an existing snippet. Forexample, if a method's return type is declared to be of classC andthen classC isReplaced then the method's signature has changed andthe method must beReplaced. Note: This can cause previously-validmethods or classes to become invalid.

The desire is that user data persist whenever possible. This is attainedexcept in the case of variableReplace. When a variable is replaced,either directly by the user or indirectly via a dependency update, thevariable is set to its default value (null since this can only occurwith reference variables).

When a declaration is invalid, either because of a forward-reference orbecoming invalid through an update, the declaration is "corralled". Acorralled declaration can be used in other declarations and code,however, if an attempt is made to execute it a runtime exception willoccur which will explain the unresolved references or other issues.

Wrapping

In the Java programming language, variables, methods, statements,and expressions must be nested within other constructs, ultimately a class.When the implementation of JShell compiles a variable, method,statement, and expression snippet asJava code an artificial context is needed, as follows:

• Variables, Methods, and Classes
  • As static members of a synthetic class
• Expressions and Statements
  • As expressions and statements within a synthetic static method within asynthetic class

This wrapping also enables snippet update, so, note that a snippetclass is also wrapped in a synthetic class.

Modular Environment Configuration

Thejshell tool has the following options for controlling the modularenvironment:

• --module-path
• --add-modules
• --add-exports

The modular environment can also be configured by direct additionto the compiler and runtime options.Compiler flags may be added with the-C option. Runtime flagsmay be added with the-R option.

Alljshell tool options are documented in the Tool Reference (seeabove).

The modular environment can be configured at the API level withthecompilerOptions andremoteVMOptions methods onJShell.Builder.

The set of modules read by JShell's unnamed module is the same as the default set of root modules for the unnamed module, as established by JEP 261 "Root modules":

• http://openjdk.java.net/jeps/261

Naming

• Module
  • jdk.jshell
• Tool launcher
  • jshell
• API package
  • jdk.jshell
• SPI package
  • jdk.jshell.spi
• Execution engine "library" package
  • jdk.jshell.execution
• Tool launching API package
  • jdk.jshell.tool
• Tool implementation package
  • jdk.internal.jshell.tool
• OpenJDK Project
  • Kulla

Alternatives

A simpler alternative is just to provide a batch scripting wrapperwithout interactive/update support.

Another alternative is to maintain the status quo: Use another languageor use a third-party REPL such asBeanShell,though that particular REPL has been dormant for many years, is based onJDK 1.3, and makes arbitrary changes to the language.

Many IDEs, for example the NetBeans debugger and BlueJ's CodePad, providemechanisms to interactively evaluate expressions. Preserved context andcode remains class-based, and method granularity is not supported. Theyuse specially crafted parsers/interpreters.

Testing

The API facilitates detailed point testing. A test framework makeswriting tests straight-forward.

Because the evaluation and query functionality of the tool is built onthe API, most testing is of the API. Command testing and sanity testingof the tool is, however, also needed. The tool is built with hooks for atesting harness, which is used for tool testing.

Tests are comprised of three parts:

1. Tests for the API. These tests cover both positive and negativecases. Each public method must be covered by tests which includeadding variables, methods, and class, redefining them, etc.

2. Testing of the jshell tool. These tests check thatjshell commands andcompilation, and execution of Java code, have correct behavior.

3. Stress testing. To ensure that JShell can compile all allowed Javasnippets, correct Java code from the JDK itself will be used. Thesetests parse the sources, feed code chunks to the API, and testthe behavior of the API.

Dependences

The implementation will make every effort to leverage the accuracy andengineering effort of the existing language support in the JDK. TheJShell state is modeled as a JVM instance. Code analysis and theproduction of executable code (jdk.jshell API) will be performed by the Java Compiler(javac) through the Compiler API. Code replacement (jdk.jshell.execution) will use the JavaDebug Interface (JDI).

Parsing of raw snippets (i.e., snippets that have not been wrapped) willbe done using the Compiler API with a small subclassing of the parser toallow raw snippets. The resulting information will be used to wrap thesnippet into a valid compilation unit including a class declaration withimports for previously evaluated code. Further analysis and generationof the class file will be done with unmodified instances of the Javacompiler. Generated class files will be kept in memory and never writtento storage.jdk.jshell.spi SPI exists to configure the execution engine. The defaultexecution engine behaves as follows. Class files will be sent over asocket to the remote process. A remote agent will handle loading andexecution. Replacement will be done via the JDIVirtualMachine.redefineClasses() facility.

Tab-completion analysis (jdk.jshell API) will also use the Compiler API. Completiondetection will use thejavac lexer, custom and table-driven code.

Thejshell tool (jdk.internal.jshell.tool) will use 'jline2' for console input, editing, and history.jline2 has been, privately, rolled into the JDK.