Repository files navigation

Tools for managing namespaces in Clojure. Parsens declarations fromsource files, extract their dependencies, build a graph of namespacedependencies within a project, update that graph as files change, andreload files in the correct order.

This is only about namespace dependencieswithin a single project.It has nothing to do with Leiningen, Maven, JAR files, orrepositories.

This project follows the version scheme MAJOR.MINOR.PATCH where each component provides some relative indication of the size of the change, but does not follow semantic versioning. In general, all changes endeavor to be non-breaking (by moving to new names rather than by breaking existing names).

Change Log

All Released Versions

Latest stable release is1.5.0

CLI/deps.edn dependency information:

org.clojure/tools.namespace {:mvn/version"1.5.0"}

Leiningen stable dependency information:

[org.clojure/tools.namespace "1.5.0"]

Maven stable dependency information:

<dependency>  <groupId>org.clojure</groupId>  <artifactId>tools.namespace</artifactId>  <version>1.5.0</version></dependency>

Git master branch is at1.5.1-SNAPSHOT

All Snapshot Versions

Leiningen dependency information for development snapshots:

:dependencies [[org.clojure/tools.namespace "1.5.1-SNAPSHOT"]]:repositories [["sonatype-oss-public"                "https://oss.sonatype.org/content/groups/public/"]]

See alsoMaven Settings and Repositories on dev.clojure.org.

API Documentation

tools.namespace consists of several parts:

clojure.tools.namespace.parse: A parser for namespace declarationsin Clojure source files. Given a stream of characters from a Clojuresource file, it can find thens declaration and parse the:requireand:use clauses to find the names of other namespaces that filedepends on. This is all syntactic analysis: it does notevaluate any code.

clojure.tools.namespace.find: Utilities to search for Clojurenamespaces on the filesystem, in directories or JAR files. Combinedwithjava.classpath, itcan search for namespaces on the Java classpath. This namespacecontains most of the functions in clojure.tools.namespace version0.1.x.

clojure.tools.namespace.repl: Utilities to load and reload codebased on the namespace dependency graph. This takes some explaining,see below. c.t.n.repl is built out of smaller parts:

• c.t.n.dependency - generic dependency graph data structure
• c.t.n.track - namespace dependency tracker
• c.t.n.file - file-reader extension to tracker
• c.t.n.dir - directory-scanner extension to tracker
• c.t.n.reload - namespace-reloading extension to tracker

You can recombine these parts in other ways, but c.t.n.repl is theprimary public entry-point to their functionality.

clojure.tools.namespace.move: Utilities to aid in moving andrenaming Clojure namespaces. This code is still ALPHA, and it modifiesyour source files, so be careful.

New in version 0.3.0-alpha1

These namespaces are.cljc files usable from both Clojure(JVM) andClojureScript:

• c.t.n.dependency
• c.t.n.track
• c.t.n.parse

These namespaces are usable on Clojure(JVM) only but can analyze bothClojure(JVM) and ClojureScript source files:

• c.t.n.file
• c.t.n.dir
• c.t.n.find

Most functions now take an optional "platform" argument, which is oneof the constant values defined in c.t.n.find:clj orcljs. Thedefault isclj.

These namespaces are still Clojure(JVM) only:

• c.t.n.reload
• c.t.n.repl
• c.t.n.move.

c.t.n.repl is a smarter way to reload code.

The traditional way to reload Clojure code without restarting the JVMis(require ... :reload) or:reload-all or an editor/IDE featurethat does the same thing. This has several problems:

• If you modify two namespaces which depend on each other, you mustremember to reload them in the correct order to avoid compilationerrors.

• If you remove definitions from a source file and then reload it,those definitions are still available in memory. If other codedepends on those definitions, it will continue to work but willbreak the next time you restart the JVM.

• If the reloaded namespace containsdefmulti, you must also reloadall of the associateddefmethod expressions.

• If the reloaded namespace containsdefprotocol, you must alsoreload any records or types implementing that protocol and replaceany existing instances of those records/types with new instances.

• If the reloaded namespace contains macros, you must also reload anynamespaces which use those macros.

• If the running program contains functions which close over values inthe reloaded namespace, those closed-over values are not updated.(This is common in web applications which construct the "handlerstack" as a composition of functions.)

Often the only surefire way to reload Clojure code is to restart theJVM. A large Clojure application can take 20 seconds or more just tocompile. I wrote tools.namespace to help speed up this developmentcycle.

For more detail on how I use tools.namespace in my developmentworkflow, see the articleMy Clojure Workflow, Reloaded.

There's only one important function,refresh:

user=> (require '[clojure.tools.namespace.repl :refer [refresh]])niluser=> (refresh):reloading (com.example.util com.example.app com.example.app-test):ok

Therefresh function will scan all the directories on the classpathfor Clojure source files, read theirns declarations, build a graphof their dependencies, and load them in dependency order. (You canchange the directories it scans withset-refresh-dirs.)

Later on, after you have changed and saved a few files in your editor,run it again:

user=> (refresh):reloading (com.example.app com.example.app-test):ok

Based on file modification timestamps and the graph of dependencies,therefresh function will reloadonly the namespaces that havechanged, in dependency order. But first, it willunload (remove) thenamespaces that changed to clear out any old definitions.

This is quite unlike(require ... :reload). Callingrefresh willblow away your old code. Sometimes this is helpful: it can catchtrivial mistakes like deleting a function that another piece of codedepends on. But sometimes it hurts when you have built-up applicationstate stored in a Var that got deleted byrefresh.

This brings us to the next section:

Being able to safely destroy and reload namespaces without breakingyour application requires some discipline and careful design. It won't"just work" on any Clojure project.

The first rule for making your application reload-safe isno globalstate. That means you should avoid things like this:

(def state-of-world (ref {}))(def object-handle (atom nil))

c.t.n.repl/refresh will destroy those Vars when it reloads thenamespace (even if you useddefonce).

Instead of storing your state in global Vars, store itlocally in anobject that represents the running state of your application. Thenprovide a constructor function to initialize that state:

(defn create-application []  {:state-of-world (ref {})   :object-handle (atom nil)})

You can choose what representation works best for your application:map, vector, record, or even just a single Ref by itself.

Typically you'll still need one globaldef somewhere, perhaps in theREPL itself, to hold the current application instance. See the nextsection.

The second rule for making your application reload-safe is to have aconsistent way tostart and stop the entire system.

The "start" function should:

• Acquire stateful resources such as sockets, files, and databaseconnections

• Start threads or background processes

• Initialize application state such as caches or counters

• Return an object encapsulating the state of the application

The "stop" function should take the state returned by "start" as anargument and do the opposite:

• Close or release stateful resources

• Stop all background processes

• Clear out application state

It might take a few tries to get it right, but once you have workingstart and stop functions you can have a workflow like this:

Step 1. Start up a REPL.

Step 2. Load the app:

user=> (require '[clojure.tools.namespace.repl :refer [refresh]])user=> (refresh)user=> (def my-app (start-my-app))

Step 3. Test it out.

Step 4. Modify some source files.

Step 5. Restart:

user=> (stop-my-app my-app)user=> (refresh)user=> (def my-app (start-my-app))

(You could also combine all those steps in a single utility function,but see warnings below.)

After that, you've got a squeaky-clean new instance of your apprunning, in a fraction of the time it takes to restart the JVM.

If an exception is thrown while loading a namespace,refresh stops,prints the namespace that caused the exception, and returns theexception. You can print the rest of the stacktrace withclojure.repl/pst; the exception itself is bound to*e.

user=> (refresh):reloading (com.example.app com.example.app-test):error-while-loading com.example.app#<IllegalArgumentException java.lang.IllegalArgumentException:  Parameter declaration cond should be a vector>user=> (clojure.repl/pst)IllegalArgumentException Parameter declaration cond should be a vector        clojure.core/assert-valid-fdecl (core.clj:6567)        clojure.core/sigs (core.clj:220)        clojure.core/defn (core.clj:294)        clojure.lang.Var.invoke (Var.java:427)        ...

Remember that any namespaces which depend on the namespace that causedthe exceptiondo not exist at this point: they have been removedbut not yet reloaded.

After you fix the problem, callrefresh again and it will resumereloading where it left off.

NOTE: If your current REPL namespace is one of those that has notyet been reloaded, then none of the functions you defined in thatnamespace will exist! Starting with version 0.2.8, tools.namespacewillattempt to restore aliases to the namespaces which weresuccessfully loaded.

So, for example, if your current REPL namespace is nameddev andcontains this ns declaration:

(ns dev  (:require [com.example.foo :as foo]            [com.example.bar :as bar]            [clojure.tools.namespace.repl :as tns]))

And you get an error on refresh like this:

dev=> (tns/refresh):reloading (com.example.bar dev):error-while-loading com.example.bar#<CompilerException ... compiling:(com/example/bar.clj:1:21)>

Then the functions incom.example.foo should still be available inthedev namespace via the aliasfoo.

ns syntax: Clojure'sns macro is notoriously lax in whatsyntax it accepts. tools.namespace.parse is somewhat liberal, but itcannot handle every possible variation of syntax thatns does. Stickto the docstrings ofns andrequire and everything should be fine.

AOT-compilation: Reloading code does not work in the presence ofAOT-compiled namespaces. If you are using AOT-compilation in yourproject, make sure it is disabled and you have deleted anyAOT-compiled.class files before starting a REPL developmentsession. (In Leiningen, runlein clean.)

Note that the presence of:main in project.clj triggersAOT-compilation in some versions of Leiningen.

Conflicts: Other libraries which also do code-reloading mayconflict with tools.namespace. One known example is ring-devel (as ofRing version 1.1.6) which usesns-tracker, which uses an olderversion of tools.namespace.

REPL namespace: Be careful when reloading the namespace in whichyou run your REPL. Because namespaces are removed when reloading, allyour past definitions are lost. Either keep your REPL in a namespacewhich has no file associated with it, such asuser, or put all yourREPL definitions in a file so that they can be reloaded.

Fully-qualified names: Be careful when using fully-qualifiedsymbol names without namespace aliases (require with no:as). Ifthe namespace happens to be loaded already, it will not necessarilycause an error if you forget torequire it, but the dependency graphof namespaces will be incorrect.

Old definitions: Beware of code which has references to olddefinitions, especially references to things you created in the REPL.

Rolling your own: If you create your own instance of thedependency tracker, do not store it in a namespace which getsreloaded.

Be careful defining a helper function in a namespace which callsrefresh if that namespace also could get reloaded. For example, youmight try to combine the stop-refresh-start code from the "ManagedLifecycle" section into a single function:

(def my-app nil)(defn restart []  (stop-my-app my-app)  (refresh)  (alter-var-root #'my-app (constantly (start-my-app))))

This won't work if the namespace containingrestart could getreloaded. Afterrefresh, the namespace containingrestart has beendropped, but the function continues to run in theold namespace andrefer to old Vars.

If you want to run some code afterrefresh, you can pass an optionnaming a function you want to runafter a successful reload. Thevalue of this option must be a symbol, and it must be fullynamespace-qualified. The previous example could be correctly written(assuming these functions are defined in thedev namespace):

(def my-app nil)(defn start []  (alter-var-root #'my-app (constantly (start-my-app))))(defn restart []  (stop-my-app my-app)  (refresh :after 'dev/start))

Namespace aliases created at the REPL will still refer to theold namespace afterrefresh. For example:

user=> (require '[com.example.foo :as foo])user=> foo/baruser=> (refresh):reloading (com.example.foo):okuser=> foo/bar   ; this is the *old* foo/bar

If you try to recreate the alias with the new namespace, you will get an error:

user=> (require '[com.example.foo :as foo])IllegalStateException Alias foo already exists innamespace user, aliasing com.example.fooclojure.lang.Namespace.addAlias (Namespace.java:224)

The only way out is to remove the alias before recreating it:

user=> (ns-unalias *ns* 'foo)niluser=> (alias 'foo 'com.example.foo)

When reloading namespaces which contain protocols, be careful that youdo not leave any old instances of records or types implementing thoseprotocols.

For example, if you have a namespace like this:

(ns com.example.foo)(defprotocol IFoo  (foo [this]))(defrecord FooRecord []  IFoo (foo [this] nil))

And you do something like the following at the REPL:

user=> (def my-foo (->FooRecord))user=> (clojure.tools.namespace.repl/refresh)user=> (foo my-foo)

You will get a confusing error message like this:

IllegalArgumentExceptionNo implementation of method: :fooof protocol: #'com.example.foo/IFoofound for class: com.example.foo.FooRecordclojure.core/-cache-protocol-fn (core_deftype.clj:527)

That's becausemy-foo is aninstance of theold version ofFooRecord, implementing theold version ofIFoo. As far as theJVM is concerned, the oldIFoo and the newIFoo are completelydifferent classes.

To avoid this problem, always create new instances of records after arefresh.

Callingprefer-method is a global side-effect. If you modify a calltoprefer-method and reload the namespace containing it, Clojure maythrow "java.lang.IllegalStateException: Preference conflict in multimethod."The workaround is to callremove-method before reloading.tools.namespace cannot detect this situation automatically. See [TNS-23].

In rare cases, reloading a lot of code may lead to out-of-memoryerrors from the JVM likejava.lang.OutOfMemoryError: PermGen space.

You may be able to mitigate this by increasing the size of the"Permanent Generation" where the JVM stores compiled classes. To dothis, add the following command-line argument to your JVM startup:

-XX:MaxPermSize=<N>

where<N> is a number with a suffix likem for megabytes.

To find the default MaxPermSize for your JDK, runjava -XX:+PrintFlagsFinal and search the results for "MaxPermSize".Try doubling it.

The Permanent Generation was removed in JDK 1.8 ([JEP 122]) so thissection no longer applies.

In some older JDKs (1.5) the default garbage collector did not collectthe Permanent Generation at all unless it was explicitly enabled with-XX:+CMSPermGenSweepingEnabled.

Some projects have a "project REPL" or a "scratch" namespace where youwant keep state during development. You can use the functionsdisable-unload! anddisable-reload! inclojure.tools.namespace.repl to preventrefresh from automaticallyun/reloading those namespaces.

Use this feature sparingly: it exists as a development-timeconvenience, not a work-around for code that is not reload-safe. Also,see thewarnings about aliases, below. Aliases to reloaded namespaceswill break if the namespacecontaining the alias is not reloadedalso.

After an error,refresh willnot attempt to recover symbolmappings and aliases for namespaces withdisable-unload! ordisable-reload! set.

• Change Log
• GitHub project
• How to contribute
• Bug Tracker
• Continuous Integration

Copyright © Rich Hickey, Alessandra Sierra, and contributors

All rights reserved. The use anddistribution terms for this software are covered by theEclipse Public License 1.0 which can be found in the fileepl-v10.html at the root of this distribution. By using this softwarein any fashion, you are agreeing to be bound by the terms of thislicense. You must not remove this notice, or any other, from thissoftware.