@joe-warren in thedemo, you state that the current doctests approach isn't very ergonomic because it requires you to write a comment and to import functions that you're going to use.

Not a HLS contributor, but I'd like to discuss the design that you propose.

Local values, result types, types on hover

New plugin

With your plugin, one can write a number of top-level functions that use local values that are unavailable in the configuration file.

One will then be able to run some of these top-level functions using code lenses that callIO () functions.

One will also be able to see the types of expressions on hover.

t = 3f = idf1 = f t

Doctests

When writing doctests, one can also write top-level functions that use local values that are unavailable in the configuration file.

Doctests may produce not onlyIO ().

Unfortunately, one can't interactively view the types of expressions in doctests. Hence, one needs to create auxiliary functions likef1 and Evaluate them.

t = 3f = idf1 = f t-- >>> f1

Unnecessary function imports

New plugin

New plugin avoids imports by loading the configuration file into a GHCi session.

However, unlike hls-eval-plugin, it's not triggered by doctest comments, instead it takes a "configuration" file, containing a number of Haskell functions, and for each combination of "value in the current module" and "function in the config", if the result of applying the function to the value is IO () it generates a code lens which runs that result.

From thedemo:

I'm doing the equivalent of copying and pasting the contents of that file into the GHCi session.

Doctests

Doctests do require imports.

This fact leads to two problems:

1. If an import isn't used anywhere else apart from doctests, a warning is displayed that this import is unused.
2. Imports from local modules in the$setup section don't work #4612

After the second problem is resolved, one will be able to write imports that are necessary only for doctests into a$setup section. Hence, it will be possible to resolve the first problem.

Wrong code lenses

New plugin

The new plugin may display code lenses for combinations of render and ordinary functions that don't make sense. Such code lenses may create a lot of visual noise.

Moreover, each part of the codebase may require own set of render functions. It may not be feasible to put all render functions into a single configuration because the visual noise from code lenses will increase even more.

Doctests

When using the doctests approach, one will see Evaluate code lenses above each doctest.
This visual noise seems acceptable.

Relation to a Haskell package

New plugin

It's unclear whether the configuration file should belong to the Haskell package where the file is used (i.e., be listed in the package's.cabal file). Can HLS show types on hover in the configuration file when you're composing that file?

Doctests

Doctests in a module use functions imported from local and non-local modules (e.g. modules from dependencies on Hackage). Local modules and dependencies are tracked in.cabal files. HLS shows types on hover in local modules.

Conclusion

To sum up, the new plugin provides a slightly less verbose way for composing and runningIO () actions using definitions from a special module (a configuration file), but the plugin may introduce significant visual noise and issues with maintainability of the code in the configuration file.

Despite some overhead in the form of comments and auxiliary functions, doctests might be a more general, more precise, and less noisy solution to the problem of quickly composing and runningIO () actions using definitions from local modules when#4612 is fixed.

The new plugin may display code lenses for combinations of render and ordinary functions that don't make sense. Such code lenses may create a lot of visual noise.

I don't know if I'm too concerned about visual noise from render code lenses, because:

1. the plugin can be disabled (may be disabled by default)
2. noisy render actions (actions that typecheck when applied to a lot of values, but don't make sense to evaluate), seems more like a hypothetical issue with a bad configuration, rather than a fundamental issue with the approach