cassiel

Recently we’ve been working on several digital art projects using Field as a development and presentation platform but with Clojure running the core, domain-specific algorithmic code. This choice is, admittedly, partly because Clojure is new and shiny, but we also like the Emacs- and Leiningen-based development environment (complete with continuous integration testing), and Clojure’s clean functional semantics lends itself to realtime, evolutionary artworks. Since Field works at the level of Python-on-Java (via Jython), and Clojure runs in the JVM, the Python and Clojure worlds inevitably collide.

Field hosts Clojure directly – in fact, it hosts several JVM-based languages – using something called Text Transforms. Activate the Clojure transform, as in the splash image above, and execution flips from Python to Clojure and back again. The transform passes Python bindings into the Clojure namespace, and bindings made within Clojure are passed out again (within a dedicated variable called _clojure). Both Python and Clojure have access to anything the JVM can see via its classpath. There are one or two wrinkles (such as the hack, shown above, of making sure a variable is retrieved from the Python world before its first use in the Clojure world, since Field’s dynamic lookup machinery only works for Python), but everything pretty much just works.

If we’re developing in two languages at once, we need to be able to pass information between them, not least configuration information from Python into Clojure. In Python, we organise our manifest constants without polluting the top-level namespace by declaring things like this:

m_Configuration = Manifest(
    FULL_CANVAS = False,
    DO_VORONOI = True,
    TREE_DEPTH = 5,
    FIELD_TIMER_INTERVAL = 0.033,
    ASSETS_DIR = "/Users/nick/CASSIEL/FIELDWORKSPACE/"
)
 
m_Nested = Manifest(
    A = Manifest(B = 6, C = "Hello"),
    D = Manifest(E = False),
    L = [1, 4, 9, Manifest(Z = 14)],
    L_L = [[1, 2], [3, 4, [5]], 6, []],
    N = None
)

This lets us use names like m_Configuration.TREE_DEPTH (value 5) or (if we use our rather implausible test declaration) m_Nested.A.C ("Hello"). The Python for Manifest is pretty simple:

class Manifest:
    def __init__(self, **kw):
        for k in kw:
            setattr(self, k, kw[k])

So what of the Clojure world? Passing a Manifest into Clojure exposes an object constructed from numerous Jython components, whereas what we really want is Clojure hash maps with keywords, so that we can write things like (:TREE_DEPTH m_Configuration) or (:C (:A m_Nested)). So, here’s an implementation of something we call hashify:

(ns oncotrees.util
  (:import [org.python.core PyFloat PyInteger PyBoolean PyString
            PyList PyInstance PyNone]
           [java.util NoSuchElementException]))
 
(defn hashify [py-object]
  (cond (instance? PyNone py-object) nil
        (instance? PyInstance py-object)
        (letfn [(retr [key] (.__getitem__ (.__dict__ py-object) key))
                (de-iter [iter result]
                  (let [n (try
                            (.next iter)
                            (catch NoSuchElementException _ nil))]
                    (if (nil? n)
                      (reverse result)
                      (recur iter (cons n result)))))
                (lookup [key]
                  (let [p (retr key)]
                    (condp = (class p)
                        PyFloat (.asDouble p)
                        PyInteger (.asInt p)
                        PyBoolean (.__nonzero__ p)
                        PyString (.asString p)
                        PyList (map hashify (de-iter (.iterator p) nil))
                        (hashify p))))]
          (reduce (fn [m k] (assoc m (keyword k) (lookup k)))
                  (hash-map)
                  (.keys (.__dict__ py-object))))
        :else py-object))

We scan through the keys, turning each one into a Clojure keyword and converting its value (recursively, so that we can handle nested manifest objects and lists). The conversion from Java iterator into Clojure list is rather ugly – I’m a bit surprised that there’s no pre-existing support for this.

Looking back at the Field listing at the top: we pass the various manifest objects as arguments to top-level Clojure functions which are higher-order and hence return the actual functions we call from Python, having hashified the manifests. Here’s the main function from a current artwork:

(defn kick
  [m_Configuration m_Geometry]
  (let [{:keys [DO_VORONOI TREE_DEPTH BRANCH_BEND_PATHS]}
        (u/hashify m_Configuration)
        {:keys [CELL_ROWS CELL_COLS CELL_SIZE DELTA
                INITIAL_LEN LEN_MUL]}
        (u/hashify m_Geometry)]
    (fn [t]
      (let [v-fn (if DO_VORONOI
                   (voronoi-with (* CELL_COLS CELL_SIZE 1/2)
                                 (* CELL_ROWS CELL_SIZE 1/2))
                   no-voronoi)]
        (v-fn t
              (d/fixed CELL_ROWS CELL_COLS CELL_SIZE)
              (fn [t] (let [depth-here TREE_DEPTH
                            tip 1
                            tree0 ((d/tree DELTA LEN_MUL tip BRANCH_BEND_PATHS)
                                   t depth-here 0 [0 0] INITIAL_LEN nil)
                            tree1 (map rotate-R tree0)
                            tree2 (map flip-seg-Y tree0)
                            tree3 (map (comp flip-seg-X flip-seg-Y) tree0)]
                        (cons [[0 0] [0 0]] tree1))))))))

The big win here is the ability to map-bind the entries of a manifest in one go. Using the :keys form here binds the names directly, at the cost of a pile of ugly upper-case identifiers in the middle of the code – but there’s no reason why we couldn’t modify hashify to downcase everything.

The final push: it makes sense to pass composite data objects from Clojure back to Python in the same form (rather straining the original intent of “manifest”), but how do we turn Clojure’s persistent hash maps back into manifest objects? It would be great if we could overload the Manifest constructor to take a hash map; since we can’t, this is the closest I managed to get: an optional non-keyword argument which can be a hash map. Manifest now looks like this:

from java.util import Map, List
 
class Manifest:
	def __init__(self, h=None, **kw):
		for k in kw:
			setattr(self, k, kw[k])
 
		if h is not None:
			for k in h.iterator():
				n = k.getKey().getName()
				v = k.getValue()
				setattr(self, n, self.__unpack(v))
 
	def __unpack(self, x):
		if isinstance(x, Map):
			return Manifest(x)
		elif isinstance(x, List):
			return map(self.__unpack, x)
		else:
			return x

This is effectively “unhashify” on the Python side (and again it’s recursive to handle nested structures). The instance check for Map gets us hash maps; the instance check for List gets us Clojure lists, vectors and sequences. The only real Clojure-dependent part of this is the .getName() call on the keywords to turn them back into strings. (Again, we’d probably want to upcase these to be typographically correct.)