component


20
Apr 16

The Way Nature Creates Things

Dogma From The Top


The best thing about the Clojure community is a large number of independent thinkers. The second best thing that allows this community to exist is the Clojure language itself, that have the hammock philosophy and great tools to enable thinkers to think and create. Often I notice myself following a “90 / 10” rule: where I spend 90% of the time thinking (brain, REPL, google, papers, people, sleep, etc..) and 10% of the time creating once thoughts are solidified.

As any vibrant community Clojure has “the powers that be” that provide guidance to people who need it. This guidance could be either: very helpful or lacking context + real world examples => let’s call it “dogma from the top”.

While I am not a biologist, I like to draw a parallel between software programs created by developers and things “naturally” created by nature. When I am in the beautiful garden or diving in the ocean, I find a lot less dogma, and a lot more fruits of mutation and centuries of trial and error development. It’s amazing. So much bigger and more honest than… “singletons are bad”.

Yes, this short story is about Mount, or rather it is about perception and understanding.

Singletons are Bad


If the context is not provided, “singletons”, or “global singletons” when referring to Clojure vars, have no quality of being bad or good. It’s just the word. Is “paper” bad? How about “inheritance”, is it bad? Well, what if it’s in millions of dollars?

The claim from the top is that keeping state in Clojure vars is bad, because global singletons are bad. Ok more context, but still no context to measure the goodness. When I ask for a concrete example, or two, that would “teach me” why Clojure var is such a horrible place to keep state, the only concrete example I get is:

“Because you can develop and run tests within the same REPL / runtime”

It turns out that some people develop and run tests within the same REPL.

I like my tests run in a separate REPL, moreover I like to run a watcher in that REPL which will rerun tests on any change I make and will notify me with a pass/fail sound / color in that terminal. This can be easily done with Boot: boot watch speak test. Can also be done with Leiningen via lein-test-refresh: lein test-refresh.

Another benefit of running tests in a separate REPL / runtime: it ensures a clean, isolated environment where tests are run. And since most people still use Leiningen this saves a lot of time.

But I do respect that people could have different workflows and they like running tests within the same REPL they develop. And there is Yurt that allows to create as many different yurts as needed in the same runtime.

False Dmitry I


… “the real Dmitriy died in Uglich” [source]

All other “examples” of why keeping state in vars is bad are simply.. not cutting it. Here are some.

“Because there are instances I will have more than one database”

– “Why would I ever have more than one database?”
– “Why would I not!?”

And the example usually goes like this:

(component/system-map
 ;;...
 :user-db (database "sql://user-db")
 :product-db (database "sql://product-db")
 ;;...)

“I can’t do that with namespaces and vars”. Yes you can. Namespaces and vars are beautiful, very capable things. Here is a simple example:

(defn database [uri]
  (connect-to uri))
 
(defstate user-db :start (database "sql://user-db"))
(defstate product-db :start (database "sql://product-db"))
 
;; + :stop functions

notice how “database” is just a function, can be reused as many times as needed. user and product DBs can live within different namespaces if needed.

Yes, if “defstate” was a simple “def”, “def user-db” would be limiting, since how would you restart it for example. But “defstate” plugs “user-db” into a lifecycle aware Mount where “user-db” becomes a managed component (in a good sense) of an application.

Context is King


And here we arrived at the context: (defstate state :start (f1) :stop (f2)) is not just some global singleton which is claimed to be bad, but it is a living component that can be started, restarted, referenced and healed when it’s abused. Is it bad? Not from my experience, it’s actually quite excellent 🙂

“Because if you recompile a namespace you end up with a stale state”

Now with the context provided, it is simply not true with Mount.

When a namespace is recompiled Mount will stop/restart/clean all the states in this namespace. It will also clean the deleted states. Here is a lot more details about it: Recompiling Namespaces with Running States and Cleaning up Deleted States

Just Ask


Again, the Clojure community is full of people who think and reason independently. The next time you hear something like:

” Whenever you’re about to make a singleton, ask yourself “When will I need 2 of these?” ”

or

” More than one instance of a resource in an app? Inconceivable! ”

Just ask for more context, you’ll see that most of the time this is just “dogma from the top” and has nothing to do with the way nature creates things.


31
Jan 16

Yurt: Mount’s Local Real (e)State

Components or states in mount are kept in Clojure vars, which means that they are accessible from any other namespaces that :require them. They can be made private of course but nothing stops a developer from accessing a private var in Clojure by its full name: i.e. #’any.namespace/any-var.

The flip side is that they “lie near”: they are really easy to use. Can usage of these vars be abused? Of course. Can any “other state management solutions” be abused? Of course. I like my fine balance between “easy” things for development and “simple” things for the architecture.

Several, Local and Simultaneous


In several reddit discussions, people pointed out that since mount keeps components in Clojure vars, those are singletons, hence you can’t have more than one.

While I honestly don’t know how often I would want to have more than one database connection to

* the same database
* with the same host / port / sid
* the same credentials and
* the same schema

(because if any of the above is not the same it would be a different resource / component all together) The example people gave would always come down to this one use case where you want to run a development “system” and a test “system” (potentially more than one) in the same REPL.

The way I do it today is simply running:

boot watch speak test

in a different REPL.

The need to run multiple systems in the same REPL might have something to do with the limitation of the framework (i.e. component), since you can’t just start and stop parts of a system, which means you can’t iterate quickly with sub systems in the REPL. So instead you end up predefining and running several sub systems simultaneously. But this is just an assumption.

I personally never needed to run multiple “systems” within the same REPL. But I truly believe there are multiple great ways to do things, an I simply can’t just dismiss the fact that people have different development flows.

So I sat down and started to think.

Yurts are Comfy


Mount relies on the Clojure compiler to tell it what states are defined and what order they should be started / stopped in. So, I thought, I can just use this intel, create a “system” and simply detach it from vars. In other words, using the intel mount has and only using the vars for the bootstrap I can spawn as many local systems as needed.

I don’t think this should be mount’s core functionality, plus I really like the way things work with vars and namespaces, but it would be really cool to have these local systems that people can use for testing, whether it is from REPL or for running tests in parallel.

That’s how Yurt was born. The docs go over Yurt’s API (i.e. blueprint, build, destroy) and have a couple of examples of running multiple Yurts simultaneously in the same REPL.

This of course comes down to a choice of either using vars directly or encapsulating components in Yurts, or maybe even both (?) in the same application. But what’s cool about either choice, there is still “nothing to buy”: no full app buy in.