Finding A Regular Expression Tool

Once again this year I will be teaching a “bash Beginners” workshop at the AUC’s World conference.

I’ve started re-working and improving the workshop and I’ve decided to spend at least some time on regular expressions in sed and awk as they are so often the hard bits of shell programming. Sorry about that cough, must have been some terrible syntax caught in my throat.

Do you know how hard it is to find a good regular expression experiment and demonstration tool that is compatible with those venerable tools? Well nigh impossible. Perl, Ruby and cough PHP compatible you can find but not for those two.

What I did find was an excellent web app, stream-editor. The biggest problem with this solution is that a web app isn’t the sort of thing you want to recommend to beginners. Getting it working can be fun.

The other problem with it was that I couldn’t get it to work on either my Mac or an Ubuntu VM.

Given that it was written in node this might take some fixing. The biggest step in fixing it was when my big brother told me that sometimes node is installed as node and sometimes nodejs. It turns out that the Ubuntu package installs it as nodejs and stream-editor was calling it as node so I created a link – sudo ln -s /usr/bin/nodejs /usr/bin/node and that problem went away.

The second problem was with the GitHub repo for stream-editor. Git does not create empty directories in a repo and the code for stream-editor relied on writing a file to a directory called “dist”. This meant that if you just cloned the repo the code failed to run. That one was also easy to fix when I figured it out.

(By the way, if you need an empty directory then the best thing to do is to put a README file in there saying why the directory exists and then git will have the directory and the README in the repo.)

So now I had a version that worked fine on my Mac and in the Ubuntu VM. Still not easy for a beginner to install and run.

Docker to the rescue. Could I create a Docker container to do all the hard work?
Sure I could. The process was quite easy.

First, create a brand new Ubuntu VM. Now install all the requirements for the web app, documenting them as I went. Get it running, documenting those steps. Here are the exact notes:

sudo apt install git
sudo apt install npm
sudo apt install python-pip3
sudo ln -s /usr/bin/nodejs /usr/bin/node
mkdir dev
cd dev
git clone
cd stream-editor
pip3  install -r requirements.txt
python3 &&
cd stream_editor/static
mkdir dist
npm install && npm start &&

Then to write the Dockerfile

# DockerFile for stream-editor
# v1.0
# 2 May 2018
# Tony Williams (ARW)

FROM ubuntu:16.04


COPY . /usr/local/stream-editor

# required software
RUN apt-get update && apt-get install -y \
    npm \

RUN ln -s /usr/bin/nodejs /usr/bin/node
RUN cd /usr/local/stream-editor ; pip3 install -r requirements.txt
RUN mkdir /usr/local/stream-editor/stream_editor/static/dist

# run the software
WORKDIR /usr/local/stream-editor

CMD ["--editor"]

You can see how the Dockerfile pretty much writes itself from my notes on how to get the application installed. The other important bit is that – it’s a trick I learnt from “Best practices for writing Dockerfiles”. Here it is :-

set -e

if [ "$1" = '--editor' ]; then
    python3 &
    cd stream_editor/static
    npm install
    npm start
    exit 0

exec "$@"

The advantage of using this simple shell script it is allows you to do such things as run the container and drop into bash easily. Just docker run -it stream-editor bash and we get bash instead of our application. Putting that facility into our container makes life easier if we have to debug a problem.

So now I can offer my students a tool for learning and experimenting with regular expressions. It isn’t as easy as running an app, but it’s certainly easier than futzing with python and npm.


Some iTerm2 hacking

The first thing I discovered is that when you go to install iTerm2’s shell integration it checks to see what shell you are using by reading the variable $SHELL, which is your accounts default shell, not necessarily the shell you are running.

Since my corporate Active Directory account sets my default shell to /bin/ksh (don’t ask, just don’t ask) this caused me a problem. In iTerm my default profile is set to run the command /bin/bash rather than my default shell. So to get shell integration installing properly I now set SHELL='/bin/bash' at the bottom of my bash profile.

Continue reading

Ulysses As A Blogging Tool

So I’m now using Ulysses for writing. Not coding, for that I’m still with BBEdit, but writing text of any sort. It seems quite an attractive editor, it supports MarkDown and things seem to work well.

I’m writing this as a test of it as a blogging platform. Of course the first thing to do is see what text looks like when I export it to WordPress from within Ulysses. Here, for example is some emphasised texrt and here is some strong test Let’s try a list

  • We have the first item in an unordered list
  • Now we have the second

That was the whole list.

  1. An ordered list
  2. So can you pick up items and reorder them
  3. We will see

If you delete or reorder it doesn’t update the numbers in your document but it is correct on export.

So how about we throw in some code

Code block
That continues.
I don't like Ulysses needs a marker at the beginning of every code line.

Setting Up A Cloud Server

I recently decided to set myself up with a Unix box in the cloud. I want it so I can use it for things I can’t do on my iPad, most notably using pandoc for converting MarkDown documents to various other formats.

There are any number of places that offer an incredibly cheap box. I found a special offer at one and bought myself a year for less than $20. I then fired up an Ubuntu 16 server.

Here’s what I did to get it all working in a (fairly) secure manner.

Continue reading

A Little Python Makes Dad Happy

My daughter, Jessica was born the Thursday before Father’s Day in 1992, the 3rd of September. That means this year her birthday occurs on Father’s Day.

That begs the question, what other years will this occur? Here’s some Python code that answers the question.

# Find all the years when my daughter's
# birthday and Father's day coincide
# from the year of her birth till 2100

import time

for i in range(1992, 2100):
	dt = "3 09 " + str(i)
	tt = time.strptime(dt, "%d %m %Y")
    	if tt.tm_wday == 6:
    		print i

I love you, Jessica. My favourite Father’s Day gift ever, always.

Writing A Web App With Python

I have spent my spare time in the past two weeks writing a web app. I couldn’t find a good web site that offered an easy personal journal. I wanted something that was a bit like a blog but with less fuss and totally private.

So I decided to try writing one. The language choice was obvious, Python. The next question was which framework to use. I did a web search and discovered that the choice quickly came down to two.

Django is the older giant in the room compared to Flask. The two also have a different philosophies. Django has it all built-in while Flask seems to be a thin wrapper over Werkzeug and Jinja2 mostly providing request and session handling while leaving almost everything else to extensions. Continue reading