Klipse - The Magic of Self-Host ClojureScript in Any Web Page

ClojureX 2017

London - Dec 4, 2017

Klipse


		(str (js/Date.))

Yehonathan Sharvit

@viebel, viebel@gmail.com

Agenda

Klipse: why and what
Getting familiar with Self-Hosted ClojureScript
The internals of Klipse
How you can contribute

Who am I?

Yehonathan Sharvit @viebel, viebel@gmail.com, LinkedIn
A mathematician
A coder
A pragmatic theorist
A freak of interactivity
Founded Audyx in 2013 - an Audiology Startup with 30K LOCs in Clojurescript
Author of Klipse - a simple client-side code evaluator pluggable on any web page

KLIPSE

A Web consultant: Full-Stack, clojure, clojurescript, javascript, node.js, react
Blogger about functional programming at blog.klipse.tech

Klipse - it’s all about Code interactivity

Klipse - Evaluating data

(map inc [1 2 3])

Klipse - Printing data

(println "Hello ClojureX")
(+ 1 2 3)

Klipse - Interactive UI (DOM) manipulations

(set! (.-innerHTML js/klipse-container)
      "<div style='color:blue;'>
          Hello <strong>World</strong>!
      </div>")
nil

Klipse - Interactive UI à la Clojure

[:div {:style {:color "blue"}}
"Hello " [:strong "World"] "!"]

Klipse - macros

(ns my.m$macros)

(defmacro dbg [x]
  `(let [x# ~x]
     (println (str '~x " => " x#))
     x#))

(my.m/dbg (map inc [1 2 3]))

Here is a detailed explanation why we need the extra $macros suffix.

Klipse: numbers and facts

1950 Github stars (and growing…)

Klipse

77 forks
849 commits
Started on March 2016
13 languages: clojure, ruby, javascript, python, scheme, php, brainfuck, c++, lua, ocaml, reasonml, common lisp, prolog
100+ websites
105 clojure articles on http://blog.klipse.tech/
Integration with blogging engines: Cyrogen, Jekyll, Ghost, Gatsby
Other integrations: Gitbook, deck.js (thanks @SevereOverfl0w from JUXT)
Klipse the app

Self-Hosted ClojureScript

Self-Hosted ClojureScript

Definition: Bootstrapping is the process of writing a compiler in the same language that it intends to compile

Example: Eval written in LISP

Application: Once a language that transpiles to javascript is bootstrapped it can run on a web page

Fact: ClojureScript has been bootstrapped in July 2015

Self-Hosted ClojureScript: Evaluation

(ns my.sandbox)
(require '[cljs.js :as cljsjs])

(def cljsjs-state (cljsjs/empty-state))

(cljsjs/eval-str cljsjs-state
               "(map inc [1 2 3])"
               ""
               {:eval cljsjs/js-eval}
               println)

state (atom) - the compiler state
source (string) - the ClojureScript source
name (symbol or string) - optional, the name of the source
opts (map) - compilation options.
cb (function) - callback, will be invoked with a map. If succesful the map will contain a :value key with the result of evaluation and :ns the current namespace. If unsuccessful will contain a :error key with an ex-info instance describing the cause of failure.

options details:

:def-emits-var - sets whether def (and derived) forms return either a Var (if set to true) or the def init value (if false). Default is false.
:eval - eval function to invoke.
:load - library resolution function.
:static-fns - employ static dispatch to specific function arities in emitted JavaScript, as opposed to making use of the call construct. Defaults to false.

Self-Hosted ClojureScript: Transpilation

(cljsjs/compile-str cljsjs-state
                  "(let [a 1]
                      (+ a 2))"
                  ""
                  { }
                  (comp println :value))

state (atom) - the compiler state
source (string) - the ClojureScript source
name (symbol or string) - optional, the name of the source
opts (map) - compilation options.
cb (function) - callback, will be invoked with a map. If succesful the map will contain a :value key with the result of evaluation and :ns the current namespace. If unsuccessful will contain a :error key with an ex-info instance describing the cause of failure.

options details:

:def-emits-var - sets whether def (and derived) forms return either a Var (if set to true) or the def init value (if false). Default is false.
:eval - eval function to invoke.
:load - library resolution function.
:static-fns - employ static dispatch to specific function arities in emitted JavaScript, as opposed to making use of the call construct. Defaults to false.

Pitfalls

With :statement context (default)

(cljsjs/eval-str cljsjs-state
               "(if 1 2 3)"
               ""
               {:eval cljsjs/js-eval
	        :context :statement}
               println)

With :expr context (default)

(cljsjs/eval-str cljsjs-state
               "(if 1 2 3)"
               ""
               {:eval cljsjs/js-eval
	        :context :expr}
               println)

The proper way of evaluating several expressions

Split the code into S-expressions
Evaluate each S-expression

(defn eval [exp]
  (cljsjs/eval-str cljsjs-state
                   exp
                   ""
                   {:eval cljsjs/js-eval
                    :context :expr}
                   println))

(require '[viebel.gist-983676a98aee0991cfb002a67676602f.raw.split-expressions :refer [split-expressions]])
(defn eval-several-expressions [code]
  (loop [exps (split-expressions code)
         res nil]
    (if (seq exps)
      (recur (rest exps) (eval (first exps)))
      res)))

(eval-several-expressions "(def a 4) (if a 1 2)")

Namespace resolution

Namespace resolution

The implementor needs to provide a custom mechanism for namespace resolution

(cljsjs/eval-str cljsjs-state
               "(require 'my.ns)"
               ""
               {:eval cljsjs/js-eval}
               println)

The load function is called with two arguments - a map and a callback function:
The map has the following keys:

:name - the name of the library (a symbol)
:macros - modifier signaling a macros namespace load
:path - munged relative library path (a string)

(cljsjs/eval-str cljsjs-state
               "(require 'my.ns)"
               ""
               {:eval cljsjs/js-eval
                :load (fn [data callback] (println data))}
               println)

Namespace resolution (cont.)

It is up to the implementor to correctly resolve the corresponding .cljs, .cljc, or .js resource (the order must be respected).

Upon resolution, the callback should be invoked with a map containing the following keys:

:lang - the language, :clj for clojurescript or :js for javascript
:source - the source of the library (a string)

If the resource could not be resolved, the callback should be invoked with nil.

(cljsjs/eval-str cljsjs-state
                 "(require '[my.bar :refer [foo]])
                  (+ foo 10)"
                 ""
                 {:eval cljsjs/js-eval
                  :load (fn [data callback] (callback {:lang :clj
                                                       :source "(ns my.bar)
                                                                (def foo 12)"}))}
                 println)

Klipse internals

Namespace resolution
Infinite loop prevention
Reagent

Klipse internals (1/3) - namespace resolution

Clojure libs from any hosted repository (e.g. Github)
Clojure libs from analysis cache hosted by Klipse
Code from a gist

All the details are in Github klipse/lang/clojure/io.cljs.

Klipse internals (1/3) - Clojure libs from Github

Super cool String manipulation library: superstring.


(require '[superstring.core :as superstring])
(superstring/swap-case "Hello ClojureX!")

data-external-libs="https://raw.githubusercontent.com/expez/superstring/279f722e5e61167ac11b27d6017c2d9d239f8343/src"

Klipse internals (1/3) - Clojure libs from analysis cache

Many popular clojure libs are available out-of-the box in Klipse:

clojure: clojure.spec, core.match, math.combinatorics, core.async, test.check, clojure.data
clojurescript: reagent, re-frame, om.next, cljs-time, re-frisk, cljs.tools.reader, cljs.test

(require '[clojure.math.combinatorics :refer [permutations]])

(permutations [1 2 3])

If you want to add a lib to Klipse analysis cache, follow this guide - (using Lumo).

Klipse internals (1/3) - Clojure code from a gist

(require
  '[goog.dom :as dom])
(require
  '[viebel.gist-3800b8ebae5292921c7d6fcb6c995c1f.raw.body-color :refer [set-bg-color-element]])

(let [colors ["blue" "red" "yellow" "magenta" "cyan" "green" "purple" "coral" "dodgerblue" "pink"]]
  (set-bg-color-element (dom/getElement "klipse-color-me") (rand-nth colors)))

Klipse internals (2/3) - infinite loop prevention (trivial)

(map inc (range))

Klipse internals (2/3) - infinite loop prevention


(defn foo [x] (if x 1 2))

(defn foo [x] (if x 1 2))

#_(loop []
  (foo 9)
  (recur))

Klipse internals (3/3) - Reagent

Klipse internals (3/3) - Reagent

Can you see the differences?
How do we get from the 1st code snippet to the second?

(defn hello [name]
  [:span "Hello "
   [:strong name]])

[hello "ClojureX"]

(defn hello [name]
  [:span "Hello "
   [:strong name]])

(reagent.core/render [hello "ClojureX"] js/klipse-container)

We split the expressions
We take all but the last expression
We read the last expression
We wrap the read last expression into a reagent.core/render call

Thank you Timothy Pratley!

Reagent - Let’s do it!

The eval function

(defn eval-code [exp]
  (cljsjs/eval-str cljsjs-state
                   exp
                   ""
                   {:eval cljsjs/js-eval}
                   println))

The code as a string

(def code "(defn hello [name]
  [:span \"Hello \"
  [:strong name]])
  [hello \"ClojureX\"]")

Separate the component from the rest of the code

(def expressions (split-expressions code))
(def component (cljs.reader/read-string (last expressions)))

Rewrite the code

(def rewritten-code (str (first expressions) "\n"
                         `(reagent.core/render ~component js/klipse-container)))
rewritten-code

Evaluate the rewritten code

(eval-code rewritten-code)

Joining the party

Write an interactive blog post

It’s super simple: it’s nothing more than adding a javacript tag!

Get inspired by the best blog posts out there:

Use Klipse for the slides of your next talk

Write interactive docs of your Clojure library

Klipse Codox Theme

Add new languages to Klipse

As of November 2019, Klipse supports 13 languages: clojure, ruby, javascript, python, scheme, php, brainfuck, c++, lua, ocaml, reasonml, common lisp, prolog.

It takes less than 10 lines of code to add a new language. See the guide.

Pull Requests are more than welcome!!!

Evaluate Code inside a web worker (Solve security issues + make the evaluation interruptible)
Connect with npm hosted repository
source special form
add a button to trigger evaluation
Solve CSS issues on reveal.js
etc… See Klipse issues

Klipse - The Magic of Self-Host ClojureScript in Any Web Page