Klipse - The Magic of Self-Host ClojureScript in Any Web Page

Klipse - The Magic of Self-Host ClojureScript in Any Web Page

qr code
ClojureX 2017
London - Dec 4, 2017




		(str (js/Date.))
		



Yehonathan Sharvit
@viebel, viebel@gmail.com

Agenda

  • Klipse: why and what
  • Getting familiar with Self-Hosted ClojureScript
  • The internals of Klipse
  • How you can contribute

Who am I?

  • Yehonathan Sharvit @viebel, viebel@gmail.com, LinkedIn
  • A mathematician
  • A coder
  • A pragmatic theorist
  • A freak of interactivity
  • Founded Audyx in 2013 - an Audiology Startup with 30K LOCs in Clojurescript
  • Author of Klipse - a simple client-side code evaluator pluggable on any web page
KLIPSE
  • A Web consultant: Full-Stack, clojure, clojurescript, javascript, node.js, react
  • Blogger about functional programming at blog.klipse.tech
me

Klipse - it’s all about Code interactivity

interactivity

Klipse - Evaluating data

(map inc [1 2 3])

Klipse - Printing data

(println "Hello ClojureX")
(+ 1 2 3)

Klipse - Interactive UI (DOM) manipulations

(set! (.-innerHTML js/klipse-container)
      "<div style='color:blue;'>
          Hello <strong>World</strong>!
      </div>")
nil

Klipse - Interactive UI à la Clojure

[:div {:style {:color "blue"}}
"Hello " [:strong "World"] "!"]

Klipse - macros

(ns my.m$macros)

(defmacro dbg [x]
  `(let [x# ~x]
     (println (str '~x " => " x#))
     x#))



(my.m/dbg (map inc [1 2 3]))


Here is a detailed explanation why we need the extra $macros suffix.

Klipse: numbers and facts

  • 1450 Github stars (and growing…​)
Klipse
  • 77 forks
  • 849 commits
  • Started on March 2016
  • 11 languages: clojure, ruby, javascript, python, scheme, php, brainfuck, c++, lua, ocaml, reasonml
  • 100+ websites
  • 105 clojure articles on http://blog.klipse.tech/
  • Integration with blogging engines: Cyrogen, Jekyll, Ghost, Gatsby
  • Other integrations: Gitbook, deck.js (thanks @SevereOverfl0w from JUXT)
  • Klipse the app

Self-Hosted ClojureScript

inception

Self-Hosted ClojureScript

Definition: Bootstrapping is the process of writing a compiler in the same language that it intends to compile

Example: Eval written in LISP

Application: Once a language that transpiles to javascript is bootstrapped it can run on a web page

Fact: ClojureScript has been bootstrapped in July 2015

sicp

Self-Hosted ClojureScript: Evaluation

(require '[cljs.js :as cljsjs])



(def cljsjs-state (cljsjs/empty-state))
(cljsjs/eval-str cljsjs-state
               "(map inc [1 2 3])"
               ""
               {:eval cljsjs/js-eval}
               identity)
  1. state (atom) - the compiler state
  2. source (string) - the ClojureScript source
  3. name (symbol or string) - optional, the name of the source
  4. opts (map) - compilation options.
  5. cb (function) - callback, will be invoked with a map. If succesful the map will contain a :value key with the result of evaluation and :ns the current namespace. If unsuccessful will contain a :error key with an ex-info instance describing the cause of failure.

options details:

  • :def-emits-var - sets whether def (and derived) forms return either a Var (if set to true) or the def init value (if false). Default is false.
  • :eval - eval function to invoke.
  • :load - library resolution function.
  • :static-fns - employ static dispatch to specific function arities in emitted JavaScript, as opposed to making use of the call construct. Defaults to false.

Self-Hosted ClojureScript: Transpilation

(cljsjs/compile-str cljsjs-state
                  "(let [a 1]
                      (+ a 2))"
                  ""
                  { }
                  identity)
  1. state (atom) - the compiler state
  2. source (string) - the ClojureScript source
  3. name (symbol or string) - optional, the name of the source
  4. opts (map) - compilation options.
  5. cb (function) - callback, will be invoked with a map. If succesful the map will contain a :value key with the result of evaluation and :ns the current namespace. If unsuccessful will contain a :error key with an ex-info instance describing the cause of failure.

options details:

  • :def-emits-var - sets whether def (and derived) forms return either a Var (if set to true) or the def init value (if false). Default is false.
  • :eval - eval function to invoke.
  • :load - library resolution function.
  • :static-fns - employ static dispatch to specific function arities in emitted JavaScript, as opposed to making use of the call construct. Defaults to false.

Pitfalls

With :statement context (default)

(cljsjs/eval-str cljsjs-state
               "(if 1 2 3)"
               ""
               {:eval cljsjs/js-eval
	        :context :statement}
               identity)

With :expr context (default)

(cljsjs/eval-str cljsjs-state
               "(if 1 2 3)"
               ""
               {:eval cljsjs/js-eval
	        :context :expr}
               identity)

The proper way of evaluating several expressions

  • Split the code into S-expressions
  • Evaluate each S-expression
(defn eval [exp]
  (cljsjs/eval-str cljsjs-state
                   exp
                   ""
                   {:eval cljsjs/js-eval
                    :context :expr}
                   identity))
(defn eval-several-expressions [code]
  (loop [exps (klipse.lang.clojure/split-expressions code)
         res nil]
    (if (seq exps)
      (recur (rest exps) (eval (first exps)))
      res)))
(eval-several-expressions "(def a 4) (if a 1 2)")

Namespace resolution

internals

Namespace resolution

The implementor needs to provide a custom mechanism for namespace resolution

(cljsjs/eval-str cljsjs-state
               "(require 'my.ns)"
               ""
               {:eval cljsjs/js-eval}
               identity)

The load function is called with two arguments - a map and a callback function:
The map has the following keys:

  • :name - the name of the library (a symbol)
  • :macros - modifier signaling a macros namespace load
  • :path - munged relative library path (a string)
(cljsjs/eval-str cljsjs-state
               "(require 'my.ns)"
               ""
               {:eval cljsjs/js-eval
                :load (fn [data callback] (println data))}
               identity)

Namespace resolution (cont.)

It is up to the implementor to correctly resolve the corresponding .cljs, .cljc, or .js resource (the order must be respected).

Upon resolution, the callback should be invoked with a map containing the following keys:

  • :lang - the language, :clj for clojurescript or :js for javascript
  • :source - the source of the library (a string)

If the resource could not be resolved, the callback should be invoked with nil.

(cljsjs/eval-str cljsjs-state
                 "(require '[my.bar :refer [foo]])
                  (+ foo 10)"
                 ""
                 {:eval cljsjs/js-eval
                  :load (fn [data callback] (callback {:lang :clj
                                                       :source "(ns my.bar)
                                                                (def foo 12)"}))}
                 identity)

Klipse internals

internals

Klipse internals

  1. Namespace resolution
  2. Infinite loop prevention
  3. Reagent

Klipse internals (1/3) - namespace resolution

  1. Clojure libs from any hosted repository (e.g. Github)
  2. Clojure libs from analysis cache hosted by Klipse
  3. Code from a gist

All the details are in Github klipse/lang/clojure/io.cljs.

Klipse internals (1/3) - Clojure libs from Github

Super cool String manipulation library: superstring.


(require '[superstring.core :as superstring])
(superstring/swap-case "Hello ClojureX!")


data-external-libs="https://raw.githubusercontent.com/expez/superstring/279f722e5e61167ac11b27d6017c2d9d239f8343/src"

Klipse internals (1/3) - Clojure libs from analysis cache

Many popular clojure libs are available out-of-the box in Klipse:

  • clojure: clojure.spec, core.match, math.combinatorics, core.async, test.check, clojure.data
  • clojurescript: reagent, re-frame, om.next, cljs-time, re-frisk, cljs.tools.reader, cljs.test
(require '[clojure.math.combinatorics :refer [permutations]])

(permutations [1 2 3])


If you want to add a lib to Klipse analysis cache, follow this guide - (using Lumo).

Klipse internals (1/3) - Clojure code from a gist

(require
  '[goog.dom :as dom])
(require
  '[viebel.gist-3800b8ebae5292921c7d6fcb6c995c1f.raw.body-color :refer [set-bg-color-element]])

(let [colors ["blue" "red" "yellow" "magenta" "cyan" "green" "purple" "coral" "dodgerblue" "pink"]]
  (set-bg-color-element (dom/getElement "klipse-color-me") (rand-nth colors)))



Klipse internals (2/3) - infinite loop prevention (trivial)

(map inc (range))

Klipse internals (2/3) - infinite loop prevention


(defn foo [x] (if x 1 2))


(defn foo [x] (if x 1 2))

#_(loop []
  (foo 9)
  (recur))

Klipse internals (3/3) - Reagent

party

Klipse internals (3/3) - Reagent

Can you see the differences?
How do we get from the 1st code snippet to the second?

(defn hello [name]
  [:span "Hello "
   [:strong name]])

[hello "ClojureX"]



(defn hello [name]
  [:span "Hello "
   [:strong name]])

(reagent.core/render [hello "ClojureX"] js/klipse-container)


  1. We split the expressions
  2. We take all but the last expression
  3. We read the last expression
  4. We wrap the read last expression into a reagent.core/render call

Thank you Timothy Pratley!

Reagent - Let’s do it!

The eval function

(defn eval-code [exp]
  (cljsjs/eval-str cljsjs-state
                   exp
                   ""
                   {:eval cljsjs/js-eval}
                   identity))

The code as a string

(def code "(defn hello [name]
  [:span \"Hello \"
  [:strong name]])
  [hello \"ClojureX\"]")

Separate the component from the rest of the code

(def expressions (klipse.lang.clojure/split-expressions code))
(def component (cljs.reader/read-string (last expressions)))

Rewrite the code

(def rewritten-code (str (first expressions) "\n"
                         `(reagent.core/render ~component js/klipse-container)))
rewritten-code

Evaluate the rewritten code

(eval-code rewritten-code)

Joining the party

party

Write an interactive blog post

It’s super simple: it’s nothing more than adding a javacript tag!

Get inspired by the best blog posts out there:

Use Klipse for the slides of your next talk

ted

Write interactive docs of your Clojure library

Add new languages to Klipse

As of December 2017, Klipse supports 11 languages: clojure, ruby, javascript, python, scheme, php, brainfuck, c++, lua, ocaml, reasonml.


It takes less than 10 lines of code to add a new language. See the guide.

questions

Pull Requests are more than welcome!!!

  • Evaluate Code inside a web worker (Solve security issues + make the evaluation interruptible)
  • Upgrade to latest clojurescript (deal with clojure.spec.alpha)
  • Connect with npm hosted repository
  • source special form
  • doc special form
  • autocomplete
  • add a button to trigger evaluation
  • Solve CSS issues on reveal.js
  • etc…​ See Klipse issues

Questions

questions

Meanwhile, you can give a github star to Klipse...

powered by Klipse /