Here's a status list for progress with JuliaInterpreter, running Julia's own test suite. It's organized by the number of passes, failures (ideally should be 0), errors (ideally 0), broken (these are not JuliaInterpreter's problem), and aborted blocks (tests that took too long, given the settings). Some tests error outside a@test (marked by "X" in the table below) and others cause Julia itself to exit (marked by ☠️)
The tests below were run on a multiprocessor server from the Linux command line with

$ JULIA_CPU_THREADS=8 julia --startup-file=no juliatests.jl --nstmts 1000000 --skip compiler

The--nstmts 1000000 allows you to control the maximum number of interpreter statements per lowered block of code; tests that require more than this are marked as being "aborted." The default setting is 10000 (10^4). The higher you make this number, in general the more tests that should finish, but of course also the longer the suite will take to run. On my laptop, running with 2 worker processes the entire suite takes less than 5 minutes to complete using the default settings.

The remaining arguments are the same as given to Julia's owntest/runtests.jl: you can either provide a list of tests you want to run (e.g.,julia --startup-file=no juliatests.jl ambiguous), or you can list some to skip (here, all thecompiler/* tests). "Blank" indicates that one is running all the tests, so the line above runs everything except those incompiler/*.

The key point of having a status list is that it allows us to discover issues with JuliaInterpreter; consequently, the next step is to use these results to fix those problems. Help is very much wanted! Here are good ways to help out:

• (moderate) investigate failures and file an issue with a MWE. Highest priority should probably go to ones that caused errors or process exit(note: with the possible exceptions ofchannels,worlds, andarrayops, it appears that most such errors are due to a single cause,MWE of char crash #28; deletingthis block and rebuilding Julia fixes them) (EDIT: all of these appear to be fixed now). Then would be error that occurs outside of tests (theXs), errors that occur inside a@test (those marked as Errors by the test suite), failures, and of lowest priority the aborted blocks. Note that aborted blocks can lead to test failures due to repeating work (seeCompiled resumers #44), so many of these may go away if you increasenstmts. However, note that aborted blocks could indicate that the interpreter has incorrectly gotten itself stuck in an infinite loop (yes, the author has seen that happen), and as a consequence it's possible that some of these too are actually errors.
• (hard) fix the bugs.

A good way to get started is to pick one test that's exhibiting problems, anduncomment these lines. Then, the easiest way to dive into this is to run tests in a REPL session, e.g.,

include("utils.jl")const juliadir=dirname(dirname(Sys.BINDIR))const testdir=joinpath(juliadir,"test")configure_test()nstmts=10000run_test_by_eval("ambiguous",joinpath(testdir,"ambiguous.jl"), nstmts)# replace ambiguous with whatever test you want to run

from within JuliaInterpreter'stest/ directory. If you get failures, make sure you first check whether they go away if you increasenstmts (typically by 10x or more).

When you see test errors, the expression printed right above it is the one causing the problem. Go into the source code and copy the relevant source lines into aquote block. Once you have a minimal expressionex that triggers a problem, do this:

modexs, _= JuliaInterpreter.split_expressions(m, ex)for modexin modexs    frame= JuliaInterpreter.prepare_thunk(modex)    nstmtsleft= nstmtswhiletrue        ret, nstmtsleft=evaluate_limited!(frame, nstmtsleft,true)ifisa(ret, Aborted)run_compiled(frame)breakelseifisa(ret, Some)breakendendend

wherem is the module you want to execute this in. You may want to do

module JuliaTestsusing Test, Randomendm = JuliaTests

to isolate the tests from your current session.

To diagnose problems in greater detail, uncommentingthese lines can be a great first start.

Without further ado, here's the current list (note the time of the run to determine how current this is):

Julia Version 1.1.1-pre.0
Commit a84cf6f56c (2019-01-22 04:33 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Test run at: 2019-02-26T11:36:49.456

Maximum number of statements per lowered expression: 1000000