Ekaitz's tech blog:
I make stuff at ElenQ Technology and I talk about it

The previous post introduced termios as a native interface to configure the terminal input processing. With termios we managed to make our C programs get input character by character processing them as they came with no buffering but we didn’t integrate that with our Clojure code. Now it’s time to make it.

Run before it’s too late

Before we dig in the unknown, I have to tell you there are other alternatives for the terminal configuration. The simplest one I can imagine is using stty¹ as an external command. I learned this from Liquid, a really interesting project I had as a reference. If you want to see this work check the adapters/tty.clj file in the src directory of the project.²

Of course, it has some drawbacks. stty is part of the GNU-Coreutils project and you have to be sure your target has it installed if you want to rely on that. I’m not sure about if it’s supported in non-GNU operating systems³.

In my case, I decided to stay with termios interface to deal with all this because I didn’t really want to rely on external commands and it’s supposed to be implemented in any POSIX OS. The good (bad?) thing is it made me deal with native libraries from Clojure and had to learn how to do it.

The floor is Java

When dealing with low-level stuff we have to remember Clojure is just Java, and most of the utilities we need to use come from it. This means the question we have to answer is not really “how to call native code from Clojure?” because if we are able to call native code from Java, we will be able to do it from Clojure too (if we spread some magic on top).

So, how to call native code from Java?

First I checked the Java Native Interface (aka JNI), but I thought it was too much for me and I decided to check further. Remember there are only a couple of calls to make to termios from our code, so we don’t really want to mess with a lot of boilerplate code, compilations and so on.

My research made me find Java Native Access (aka JNA) library. If you check the link there you’ll find that the Wikipedia⁴ describes it as:

JNA’s design aims to provide native access in a natural way with a minimum of effort. No boilerplate or generated glue code is required.

Sounds like right for me. Doesn’t it?

I encourage you to check the full Wikipedia entry and, if you have some free time at the office or something, to check the implementation because it’s really interesting. But I’ll leave that for you.

A lantern in the dark

JNA is quite easy to use for the case of Clopher, even easier if you realize there is lanterna, the TUI library, out there, using it internally so you can steal⁵ the implementation from it. Lanterna is a great piece of software I took as a reference for many parts of the project. Digging in the internals of large libraries is a great exercise and you can learn a lot from it.

First of all, like many Java projects, the amount of abstractions it has is crazy. It takes some time to find the actual implementation of what we want. This isn’t like this for no reason, the reality is they need to create this amount of abstractions because the part of the library that handles the widgets can work on top of many different terminal implementations, including a Swing based one that comes with Lanterna itself.

Clopher only targets POSIX compatible operating systems so we can go directly to what we want and read the termios part directly discarding all the other compatibility code. This code is quite easy to find if you see the directory tree of Lanterna: there’s a native-integration folder in the root directory. If you follow that you’ll arrive to PosixLibC.java that uses JNA to interact with termios.

The implementation provided by Lanterna is quite complete, they declare a library with the functions they need and the data structure introduced in the previous chapter. Once the library interface and the necessary data structures are defined from Java they can be called with JNA, like they do in the file: NativeGNULinuxTerminal.java.

How to call JNA from Clojure, then?

Calling Java code from Clojure is quite simple because Clojure have been designed with that in mind, but this is not only that. Thanks to the Internet, there’s a great blogpost by Nurullah Akkaya describing a simple way to use JNA from Clojure. From that, we can move to our specific case.

termios has its own data structure so we need to define it so the JNA knows how to interact with it. The problem is that Clojure doesn’t have enough OOP tools to do it directly so we need to make it in plain Java. The good thing is that we don’t really need to create anything else.

If we remove some unneeded code from Lanterna’s termios structure implementation it will look like the implementation I made at src/java/clopher/Termios.java:

package clopher.termios;
import com.sun.jna.Structure;

import java.util.Arrays;
import java.util.List;


/**
 * Interface to Posix libc
 */
public class Termios extends Structure {
    private int NCCS = 32;

    public int c_iflag;           // input mode flags
    public int c_oflag;           // output mode flags
    public int c_cflag;           // control mode flags
    public int c_lflag;           // local mode flags
    public byte c_line;           // line discipline
    public byte c_cc[];           // control characters
    public int c_ispeed;          // input speed
    public int c_ospeed;          // output speed

    public Termios() {
        c_cc = new byte[NCCS];
    }

    // This function is important for JNA, because it needs to know the
    // order of the fields of the struct in order to make a correct Java
    // class to C struct translation
    protected List<String> getFieldOrder() {
        return Arrays.asList(
                "c_iflag",
                "c_oflag",
                "c_cflag",
                "c_lflag",
                "c_line",
                "c_cc",
                "c_ispeed",
                "c_ospeed"
                );
    }
}

Once the struct is defined, it’s time to use it from Clojure. clopher.term namespace has the code to solve this. Summarized here:

(ns clopher.term
  (:import [clopher.termios Termios]
           [com.sun.jna Function]))

(def ^:private ICANON 02)
(def ^:private ECHO   010)
(def ^:private ISIG   01)
(def ^:private ECHONL 0100)
(def ^:private IEXTEN 0100000)

(def ^:private VTIME 5)
(def ^:private VMIN  6)

; The macro we saw at the blogpost by Nurulla Akkaya
(defmacro jna-call [lib func ret & args]
  `(let [library#  (name ~lib)
         function# (Function/getFunction library# ~func)]
     (.invoke function# ~ret (to-array [~@args]))))

; Wrapper for the tcgetattr function
(defn get-config!
  []
  (let [term-conf (Termios.)]
    (if (= 0 (jna-call :c "tcgetattr" Integer 0 term-conf))
      term-conf
      (throw (UnsupportedOperationException.
               "Impossible to get terminal configuration")))))

; Wrapper for the tcsetattr function
(defn set-config!
  [term-conf]
  (when (not= 0 (jna-call :c "tcsetattr" Integer 0 0 term-conf))
    (throw (UnsupportedOperationException.
             "Impossible to set terminal configuration"))))

; Example to set the non-canonical mode using the flags at the top of the
; file
; Yeah, binary operations.
(defn set-non-canonical!
  ([]
   (set-non-canonical! true))
  ([blocking]
  (let [term-conf (get-config!)]
    (set! (.-c_lflag term-conf)
          (bit-and (.-c_lflag term-conf)
                   (bit-not (bit-or ICANON ECHO ISIG ECHONL IEXTEN))))
    (aset-byte (.-c_cc term-conf) VMIN  (if blocking 1 0))
    (aset-byte (.-c_cc term-conf) VTIME 0)
    (set-config! term-conf))))

Pay attention to all the mutable code here! aset-byte function helps a lot when dealing with all that.

Be also sure to check termios’ documentation because the calls act in a very C-like way, returning a non-zero answer when they fail.

We need an extra point in our code to solve the Java-Clojure interoperability: we have to tell our project manager that we included some Java code in there. If our project manager is Leiningen, we can just tell it where do we store our Java code. Be careful because Leiningen doesn’t like if you mix Java and Clojure in the same folder.

(defproject
  ; There's more blablabla in here but these are the keys I want you to
  ; take in account
  :source-paths      ["src/clojure"]
  :java-source-paths ["src/java"]
  :javac-options     ["-Xlint:unchecked"])

Look back!

Now you can configure your terminal to act non-canonically and serve you the characters one by one as they come. It’s cool but you’ll see there are some problems to come for the next chapters. Don’t worry! They’ll come.

This is like a heroic novel where the character (in this case you) fights monsters one by one leaving their dead corpses in the dungeon floor. Looking back will let you remember how many monsters did you slaughter in your way to the deep where the treasure awaits. Remember to take rest and sharpen your sword. This is a long travel.

Prepare yourself for the next monster. Let the voice of the narrator guide you to the unknown.

Why don’t you mix what you learned on the previous chapter with what you learned from this one and try to make an interactive terminal program yourself?

I’ll solve that in the next chapter, but there’s some code of that part already implemented in the repository. You can check it while I keep writing and coding. Here’s the link to the project:

https://gitlab.com/ekaitz-zarraga/clopher

See you in the next episode!