Jun 26, 2020 • Arnout Roemers
During the development of another library, a new lifecycle managment library for Clojure “revealed” itself. Without getting too entangled in Heidegger’s philosophy, this blog post discusses this new library.
Redelay is a Clojure library, offering first class
State objects for state lifecycle management.
You can think of
State objects like Clojure’s
Delay objects, but tracked and resettable.
State objects can be reset in reverse realization order by calling
Because of its first class nature you get to decide how you like to create, pass around and use your state.
State lifecycle management in Clojure is something that highly interests me.
I started with Component way back when.
Heck, we even had something custom similar to Component when it didn’t exist yet at my first professional Clojure job.
Later I discovered mount and I liked it very much.
So much actually, that I created a spin-off called mount-lite.
I also tried Integrant and I blogged about how a small change to Component’s
Lifecycle protocol solves the problems I have with Component.
Of these solutions, I still like mount(-lite) the most. Adding a lifecycle around some stateful part of the application is incredibly easy and starting and stopping the system just works. It goes out of your way and it is hardly a framework.
However, in software development there are always trade-offs, and mount(-lite) is no exception. One of the advantages of the solutions that use maps with components - or “system maps” - over global components is their unit testing story. Inside a unit test, one can simply create a system map that contains just enough components and has “test” versions of them, and pass it to the unit under test. Both Component and Integrant work this way. For example, a unit test in Component is like this:
With mount(-lite) however, you possibly have to “substitute” states and start the system, ensuring only the applicable states are started. And at the end of the test you have to ensure that the system is stopped again. It works fine, but it is a bit of a hassle. For example, in mount-lite it could look like this:
Of course, this is only the case when the
(unit-under-test) refers to the global state directly.
In most cases however, you’d have designed your functions such that they receive the state as a parameter.
This would make the unit test more like the Component example again.
A related downside of mount is that the state is always global. This implies that only one system can be started at any given time. Mount-lite 2.0 does allow multiple systems, but it’s API was not the most intuitive.
Now, I could continue this post by introducing mount-lite 3.0, because version 3.0 fixes the former two downsides. Actually, this new version of mount-lite is as good as finished (see 3.x branch). And while it may see its release not too far from now, it is not the reason for this blog post, as you’ve guessed from the title. This is because one issue was still nagging me.
Like mentioned before, state in mount is global. I obviously do not have a problem with that per se, as long as one does not refer to it from every corner of the application. However, there is something else global as well. It is its internal registry of which global states actually exist.
When you define a mount
(defstate ...), the global state is added to this registry if it wasn’t already known.
When you invoke mount’s
(start), it uses this internal registry to determine what states to start and in what order.
The same goes for
While this makes mount’s start/stop so easy to work with, it can cause trouble.
For example when you introduce a new
defstate while your application is already fully loaded.
Because mount and mount-lite rely on the load order of the Clojure compiler, it cannot determine the correct order for this new
The only solution now is fiddling with the internal registry, or restarting the application.
The latter defeats the purpose of these kind of libraries.
Mount-lite 3.0 currently toys with a function called
(forget!), which cleans the internal registry.
Afterwards one can reload the application, fixing any inconsistencies in the registry.
The name of the function or this entire functionality may change or vanish though.
Because, I don’t really like having it.
It complicates things.
Can’t I fix this as well, just as I fixed the other two downsides?
What is actually the underlying problem here?
Back to the drawing board.
Turns out, the problem is not the fact that the states are global.
I mean, take Clojure namespaces for example.
Those can be seen as global registries with vars.
Mount’s registry is not the problem either.
Again, look at the clojure.spec library.
That library has a global registry as well, yet does not need something like a
The difference between those global registries and the one in mount(-lite) lies in the way they are queried.
A namespace does not need to keep track of some sort of order for its vars, or which ones are new or old, because it will get queried precisely at the moment a particular var is needed. Likewise, the clojure.spec registry doesn’t have this problem either. A spec definition is either needed at some point, or not. Put simply, it is usage-driven. One function needs another function, so it queries the namespace. One spec needs another spec, so it queries the specs registry. If a new function or spec has been added, it just works. If an old function or spec is not used anymore, that’s fine.
However, with mount(-lite) it is not usage-driven.
(start) starts all the states.
Therefore it needs to know the order, it cannot recognize how a new state fits in this order, and it cannot recognize that a state is actually old and not used anymore (except with some tricks).
Now the question is, how can we design a mount-like solution to be usage-driven?
Easy, just remove
Let me explain.
What if we design our state construct to only be activated when it is required, so it becomes usage-driven?
In other words, only run the state’s start logic when requested.
Clojure already has a construct that does this: a Delay.
The expression you pass to
delay is lazy; it is only realized once it is dereferenced using
However, once a Delay is realized, it stays that way.
What if we create a Delay that can be reset, possibly by running some stop logic?
That’s exactly what the redelay library is about.
Instead of Delay objects, it lets you create
State objects take a start expression, but optionally also a stop expression.
By dereferencing a State object the first time, it is realized by executing the start expression.
The result is cached and returned when dereferenced again.
A State object also implements Java’s
.close on it, the stop logic is run and its cache is cleared.
This makes the State ready to be realized again.
Adding a lifecycle around a stateful part of the application using an object like this makes it first-class.
In other words, we can create, keep and pass around State everywhere we like, however we like.
This is also the reason that redelay does not have some sort of
It simply does not know where your state is, and it does not need to know.
As long as your functions receive or can retrieve the State objects, the “system” starts itself.
But what about resetting the system?
Next to offering a State object, the redelay library has one more trick up it’s sleave.
It keeps track of State objects that have been realized and in what order.
(status) function enumerates the realized State objects.
And while a
start function is not part of the library, a
stop function is.
(stop) will close all the realized State in reverse order, wherever they may be.
Even if the application or your REPL lost track of the reference to a State object, redelay can still reset it.
(stop) your system is ready to be started again.
It may now start with redefined States and/or it may ignore some old ones.
Anything goes, now that State has become first class.
Time for some source code!
Let’s have a look at how the redelay API works.
First, creating a State object is done using the
The body of the macro consists of any number of forms.
These forms can be qualified using a keyword.
Valid qualifiers are
If no qualifier is given,
:start is assumed.
As you can see, the
state macro is flexible.
:start and the
:stop expression can consist of multiple forms.
Also note that the
:stop expression has an implicit
this parameter to refer to its own value.
A State object also supports metadata (
with-meta), which can also be set using the
:name expression takes a (optionally namespaced) symbol.
This makes recognizing State objects easier.
Notice how the State objects are printed differently depending on whether a name is specified:
If you want your State objects to be global, there is an extra macro called
The following expressions are all the same:
To use a state, it must be dereferenced.
If it is dereferenced for the first time or after it has been reset, it is realized by running
The result of the expression is cached and the State is now tracked by redelay.
(status) function returns which States are realized.
To stop the system, one can call
All the tracked states are closed in reverse realization order.
Continuing the last example, it would look like this:
And that’s almost the full API:
state macro, a
defstate convenience macro, a
status and a
There is also a
state? function that returns true if the object given to it is a State object.
The last part of the API is the
While it simply points to itself, you can call Clojure’s
add-watch on it.
The watchers receive notifications of realized or closed State objects.
This allows you to add logging for example, but also to create registries of your own.
For example, this adds simple logging:
stop functions are implemented using the
In that sense, only the
state macro and the
watchpoint var make up the core of the library.
Now that any stateful part of your application can be wrapped in a first class object, one is completely free where and how to create this state, where to keep it, when to realize it, how to pass it around or refer to it, et cetera.
If you like your state to be global - as it is with mount - you can use
defstate as in the examples above.
For unit tests, you can use
with-redefs to substitute “production” state with test variants.
No special “substitution” API is needed here.
And if you like a
(start) function, you can simply create such a function yourself.
Either explicitly naming the states:
Or, more implicitly using the metadata that is set by the
But maybe you like your components bundled together in a system map - as it is with Component.
You can easily create such a map yourself.
However, I recently blogged about the updated rmap library.
In short, its
rmap macro let’s you create recursive maps, where values can refer to other entries in the same map, using the
The values are not evaluated yet at that point.
Only after calling
valuate! the values are evaluated and a realized/normalized map is returned.
Let’s use this to create a system map:
In the example above, the
production map is not a started system yet.
It is only when calling
valuate! that the system map is really started.
Creating a variant on this map - say for an integration test - is also easy.
For example, we could update the config before creating the system map:
Again, these are just examples. The point is that redelay is hopefully flexible enough that it lets you deal with state and its lifecycle in your preferred way. Be it globally or in a system map, be it programmatically or using data, be it lazily or started explicitly, it’s your choice. There is no fixed approach, no rules to follow, no framework.
State management distilled to just a resettable Delay.
As always, have fun!