Ekaitz's tech blog:
I make stuff at ElenQ Technology and I talk about it

Some months later, it’s time to talk about the post I made about writing software for windows, without windows because I put some of those things in practice.

I made a simple application with one external audio library (OpenAL) and simple networking. No GUI this time, but some complexity was there.

The program

The program I’ll discuss here is Karkarkar, a tool to read a Twitch chat out loud in Basque. I made this for the Basque streaming community of gamers, which is really cool but they had to rely on Text-To-Speech systems for other languages as the main services for streamers don’t include Basque in their service, and the closest one, Spanish, is not good at some words’ pronunciation defer.

Most of these people are gamers, and they use Windows, which I don’t like, but in the end they are also people and they deserve Free Software regardless of the Operating System they decide to (or are forced to) use.

I took this as a cool exercise to test all we talked about in the past, and as a learning experience for the times I need to do something for a client that requires Windows support or anything like that.

The Text-To-Speech system

I used a TTS library by Aholab, a research group from the Bilbao School of Engineering, the university where I studied. They released AhoTTS (ahots means voice in Basque) some years ago in Github, a working TTS library for Basque and Spanish, with all the extra data it needs to work.

The only problem it has is the AhoTTS codebase is a mess, with tons of horrible decisions and undercover bugs. I had to fork the lib and make it work more or less before adding it to the project (I won’t discuss the issues here), but I did.

Connection to the Twitch chat

This simply uses IRC to connect to Twitch chat. The protocol is simple, and I only implemented an embarrassingly minimal amount of it. Enough to make it kind of work. I hope to make more of this in the future.

Playing the audio

I started with libao, a simple audio library and then moved to OpenAL for reasons I’ll mention next.

All together

The program listens to IRC, when a message arrives it sends it to AhoTTS, receives the samples to play and sends them to OpenAL which does its magic to make it play out loud.

Everything is the simplest thing possible as I didn’t have a lot of time to spend on this and also wanted to focus on the release process and being able to make a package for windows, and some linux distributions. Adding more code on top of that is easy once the problem of the distribution is solved.

The tooling

My strongest dependency¹, AhoTTS, is written in C++, so I decided to go for binary file distribution. And as I discussed in aforementioned post, I was looking for an excuse to make a project in Zig, and its cross-compilation capabilities could help me in this case, so I went for that. I made everything with Zig 0.10.1, as that is the latest version packaged for Guix, but I was forced to move to my Zig 0.11.0 package (see Testing).

For binary distribution, I didn’t have many ideas when I wrote the post, but a person in the Lisp Game Jam of that time suggested me to use NSIS for the Windows installer. It was already packaged in Guix, my distro of choice, so I just went for that, as it looked the simplest way to solve this.

For testing all this, I relied in wine, what else could I do?

So tl;dr:

• I’m coding on Guix.
• Programming in Zig 0.10.1 but then moved to Zig 0.11.0.
• Installer using NSIS.
• Testing done with Wine64.

Keeping it small

First of all, publishing software for Windows from Linux is painful if you have to compile everything for it. Guix helps a little bit with that as you can use --target= with mingw and have some luck. Sadly, many packages (most of them) need bash-minimal, which is not buildable for mingw at the moment.

In many software projects cross-compilation is not even possible.

Knowing that I decided to keep my project as small as possible, because I wanted to actually deliver something without losing my sanity.

Each library you depend on you have to compile and deliver to your target, too. It’s not that common to have Windows users to install things by themselves as a GNU/Linux person would.

Audio library

libao is the smallest audio library I could find, but I didn’t manage to build it for Windows myself, so I had to rely on something else. The OpenAL-Soft maintainers have a binary distribution for windows, so I decided to go for that one instead. It’s way harder to use, and I had to do some weird stuff to make it work, but it’s easier for me: no need to build it myself.

This is even more important in the case of having an audio library, interacting with the system is always a pain in the ass, and trying to cross-compile a library like this is not always easy, as their configure scripts are complex as they need to check for many things.

This is why I also avoided a GUI for the moment. Too much.

AhoTTS

AhoTTS is written in C++, it has zero dependencies and uses CMake as a build system.

At the beginning I compiled it using Guix to mingw, because it doesn’t have dependencies, it just worked. Later I encountered some issues though: libstdc++ and libgcc were not found at runtime and I had to statically link them (-static-libgcc -static-libstdc++ ftw).

Later I started digging in its code, fixing some horrible things inside and then I managed to add it as a submodule (yes, I hate that too) and compile it directly with Zig. This is the best option so far: you can statically link the library, it is built with the same process and it is cross-compiled by Zig. No more missing library problems.

Extra: In order to interact with AhoTTS from Zig, I had to make a small C++ to C bridge, that I never did before. It’s easy stuff, just convert the API a little bit, put some extern "C" on it and everything will go fine. void * is your friend.

The rest of it

That’s just writing some actual code and making the program run. That’s the easiest part.

Bringing it to the users

One thing is making something that builds and runs in your computer and a very different thing is making it work in other people’s computers. And this is the main problem that I wanted to discuss here.

I have some requirements:

• I made a terminal application, but I don’t expect my users to know how to run it. I need to make something that is click and run.
• I need it to have some icon in the desktop/startup-menu.
• The AhoTTS library needs some extra data files. These are searched by the library and need to have some specific structure. These need to be installed properly, too.
• I need to provide a simple way to uninstall the application.

Windows

The cool thing of this is I don’t own a windows machine since ~2010 and I have no access to any (living the good life!). I don’t have any plans to change that neither I have plans to really learn about Windows. So we have to be clever to solve this.

First things first:

zig build -Dcpu=baseline -Dtarget=x86_64-windows-gnu

Damn! That was actually very easy to do. Why isn’t this the norm in other languages?

Of course, in order to do this I needed to provide a proper build.zig file that was able to find the DLLs I was linking against and the header files. That’s not that difficult after all². But has to be done. Still, easy. Kudos for Zig.

Now, knowing where Windows searches the DLLs we depend on is important as our installation process will depend on it. They have some good documentation for that which has a very interesting point:

Standard search order for unpackaged apps:
…
7. The folder from which the application loaded.

This seems good enough for my purpose, so I can just install my binary and the libraries in the very same folder. I thought I would need to install stuff mixed somewhere else and learn a lot about windows, but I didn’t have to. Simple!

The extra data can be stored anywhere I want, because I am the one that controls the search algorithm, but I still need to have easy access to it. I decided to go for LOCALAPPDATA, but I’m thinking on putting it in the same folder as the rest of the program. At the time of writing that’s not done yet.

NSIS

Having clear where everything should be installed, it’s time to make it actually install it.

NSIS is a great tool. The look of the vanilla installer is old-school but I didn’t even bother to use the modern interface because that required me to think, activity I like to reserve to special occasions (like when I’m paid extreme amounts of money for it or the time I’m in bed before falling asleep).

NSIS in a nutshell: you write a script, run makensis with the script as an input and, boom! You have a .exe that installs your stuff.

It took me a little while to understand the structure of the script but once you learn it is really easy (for the basic things, after that you can go as hard as you want). It has two concepts you need to understand: Pages and Sections.

• Pages: define the different pages the user will navigate through during the process. There are many pages pre-defined, and they cover all the basic functionality: license agreement, component selection, installation directory selection…
• Sections: define the different parts of your installation process. You can mark some as optional or include them in different installation profiles (all, minimal, recommended… You’ve seen this before). Then, if you use components page, the sections will be listed to the user to choose which one they want to install. You can also add sections for the uninstaller, which will only be run when the user uninstalls the program.

EXAMPLE: I decided to include the sources also in the installer, but they are not required to run the program. It’s as simple as adding a Section with the /o flag and they won’t be automatically checked in the components step. Really cool stuff!

The rest of the thing is just commands you can read in the documentation. You can do many-many things with it. It has environment variables for almost anything you’ll need, so you don’t need to hardcode things in the script.

In my case I decided to ask the user for a configuration (the Twitch username) during the installation using a custom page (this requires some digging in the plugins’ documentation, but it’s not hard either), and created a launcher that automagically inserts the username in the call to the program (not great for later configuration, I know). This is done with a Batch file, which also makes the dirty job of opening the terminal when it’s double clicked.

Here’s the installer script I did, if you want to read it:

https://github.com/ekaitz-zarraga/karkarkar/blob/master/windows/installer.nsi

GNU/Linux

GNU/Linux world is really diverse, so it’s not easy to know about every single system’s requirements. At the moment I stayed with Guix and Debian because they are the only distros I use and I’m more familiar with them.

In Windows I was asking to the user to add their username in the installation process, but in Linux I don’t have any simple (for the user) way to do it, so for the moment I ask for the username when the program is called with no input arguments. Ugly, but works for the moment. The goal was to deliver this thing, not to make it perfect. I can do that later.

The cool part is they provide different approaches for packaging: source vs binary distribution.

XDG standard: Desktop file and icons

We need, of course, to arrange the same things we arranged in Windows: the desktop icon and launching the terminal automatically. That’s not hard to do using XDG specification!

Just add the Terminal=true line and it should open a terminal emulator when clicked³. The Exec= line in the desktop file has the program you want to run, and it has to point to it correctly.

Once the .desktop file is done, we realize we need to deal with the icons. I went just for a SVG icon, but I could add the rest. The only thing I needed to do is put everything in the correct folder. Something like this:

 usr
 └── share
     ├── applications
     │   └── karkarkar.desktop
     └── icons
         └── hicolor
             └── scalable
                 └── apps
                     ├── karkarkar.svg
                     └── karkarkar-symbolic.svg

Both Guix and Debian detect them properly once they are installed in the folder where they expect them.

Guix

I wrote quite a few Guix packages lately so I’m pretty comfortable with this.

I just need to tell Guix how it should build the program and put some files in the proper directory.

Also, I added the zig-build-system myself to Guix not that long ago. It’s pretty straightforward to use.

The icons and the desktop file needs to put everything in the correct place, #$output/share/..., and patch the Desktop file to point to the correct binary, #$output/bin/... that is. For this, I kept a desktop file as a template, with some reasonable defaults and just patched it in the Guix package. That’s easy.

Not the best Guix package ever but it simply works, and that’s everything I want at this point.

Debian

Debian packages can be exported from Guix package definitions, but it exports its file structure with every single dependency. In our case, that meant hundreds of MegaBytes. That’s too much so I did it manually, and the final size was around 10 MegaBytes. Not bad.

I never made a Debian package before, and I did the most minimalistic thing I could think of.

First, we need to build the thing:

zig build -Dcpu=baseline

And then just place everything in place in a folder, and call dpkg-deb on top of it. In order to do that, I just wrote a bash script that did this whole thing, it explains what I did way better than I can write in English:

# Run me in the linux/ folder
# I need the version in an argument, which should be Major.Minor-Revision
# for example 0.1-3.
version=$1
outfolder="karkarkar-$version"
mkdir "$outfolder"
mkdir -p "$outfolder/usr/bin"
mkdir -p "$outfolder/usr/share/applications"
mkdir -p "$outfolder/usr/share/AhoTTS/"
mkdir -p "$outfolder/usr/share/icons/hicolor/scalable/apps"

cp -r "../AhoTTS/data_tts"       "$outfolder/usr/share/AhoTTS/"
cp    "karkarkar.desktop"        "$outfolder/usr/share/applications"
cp    "../icons/karkarkar.svg"   "$outfolder/usr/share/icons/hicolor/scalable/apps"
cp    "../zig-out/bin/karkarkar" "$outfolder/usr/bin/"
patchelf "$outfolder/usr/bin/karkarkar" --set-interpreter "/lib64/ld-linux-x86-64.so.2"

mkdir -p "$outfolder/DEBIAN"
cat > $outfolder/DEBIAN/control <<EOF
Package: karkarkar
Version: $version
Section: base
Priority: optional
Architecture: amd64
Depends: libopenal1 (>=1.19.1)
Maintainer: Ekaitz Zarraga <blablablah>
Description: Karkarkar
 Listen to a Twitch chat in Basque.
EOF

dpkg-deb --root-owner-group --build "$outfolder"
rm -rf "$outfolder"

I need to highlight here the zig build I did automatically adds the Guix dynamic linker to the binary, but that is not where the dynamic linker is in Debian. I decided to patch the binary (patchelf) instead of trying to configure the compilation process, I thought this would be easier. I don’t know if it was easier or not, but it was easy, so that’s ok.

Also note that the DEBIAN/control file has the bare minimum fields, but it’s enough to work. In Debian, everything is installed in /usr/whatever, but that’s the only detail I changed.

Something I want to write down to remember later is the debian packages (.deb files) can be extracted with ar -x and they have a couple of tar.xz files inside. The data.tar.xz file has the file structure that will later installed in the system.

Testing

Yeah, you have to do it too. I tested it in Wine, but still needed to be tested in Windows. I had a working version done in Zig 0.10.0 that was running well in Wine, but it exploded in Windows because of this issue, that didn’t happen in Wine. I needed to use Zig 0.11 because of this error, which wasn’t a big deal anyway, because I already had it more or less packaged.

So, yes, you have to test in Windows just in case, if you can. I have to thank my friend (you know who you are!) for testing this program in his computer and reporting the error.

Conclusions

The program itself it’s really underdeveloped, but I actually made a lot of progress understanding how to make a program reach users in different systems easily. In the end, being a solo developer forces me to be clever, and do everything as simple as I can.

I don’t normally have many dependencies because I believe software has to be simple and tailor-made. This makes me control every aspect of the projects I do but also greatly simplify the distribution. That’s my context.

I already talked about this in the previous post on the subject, but in the end, being able to do this relieves me from the “Web is Cross-platform” mentality, which I don’t think is a silver bullet, even though in some cases might be a simple solution.

I think developers today tend to avoid making native applications and software quality suffers from that. There are many reasons for this (corporate control, subscription systems, easy deployments, controlled environments…) but one might be that the code people is used to write has too many dependencies, and it is hard to package and distribute. I don’t usually have that problem. I already said I don’t like having many dependencies.

Now I have an easy way to do this, I can simply focus on writing the actual code, which is, in the end, the most important part.

Key points:

• Zig happened to make this process really simple, just because the developers decided to make it simple and they had the engineering skill to back the decision.

• NSIS gave me all I needed to put my application in a windows machine without learning almost anything about Windows. I have the enough information to make the thing work, not more. NSIS was the key I was missing.

• Guix lets me cross-compile some of the dependencies to mingw, and I did that at the beginning (and it worked!) so it’s a powerful tool even for that.

• Wine is a good way to see everything was working well, but I found some discrepancies between it and Windows.

The full problem is not completely solved yet. Finding a GUI toolkit I can cross compile easily is still in the TODO list. But I’m pretty satisfied when the result until now.

My colleague Andrius Štikonas talked to me about MXE. I may try it in the future but I don’t like the fact that it downloads things by itself. I leave it here just in the case I need it in the future.

In fact, I think I can try other programming languages, even interpreted ones. NSIS is surely capable of dealing with it. This opens a whole world of possibilities for me.

In the end, everything has been a very interesting process, making applications targeted to the non-programmer (and non-gnu/linux) user has always been in my mind. Now, I can say I almost solved the problem and I have the base to work more on this in the future.

Finally, the program is released in a very alpha stage in itch.io where you can find the installers and packages and the code is hosted in Github until I get completely mad and I delete my account entirely (which I have no doubt it will happen one day or another).

And that’s mostly it, now I can focus on extending the behavior.