Containers Rock! Why I’m A Docker Fan

Docker for the Macintosh has recently emerged from beta and I’m ecstatic.

Docker implements a way of walling off a piece of software from the underlying operating system using a tech they call “containers”.

This is an absolute godsend for deploying services. One of the problems in system administration is the cost and complexity of spinning up a new service and then removing it from a computer once it is no longer required.

Software when it is installed and run can spray pieces of itself all over the computer’s file system and getting it out again is difficult.

Previously we have used virtual machines to isolate this problem. That has it’s own costs, a virtual machine means you are running (at least) two complete operating systems on the hardware. It also has a cost in memory and hard disk space.

Containers lower the cost considerably. They have all the advantages of virtual machines but share the operating system kernel with each other and the underlying OS. This makes them smaller and consuming considerably less resources than virtual machines. This also makes them quicker to download and deploy.

Since Docker is open source it means that there is now a huge community around it. Docker containers are easily available for a huge range of applications, a quick visit to Dockerhub will show you how large.

Docker containers may well be the holy grail of app deployment. They certainly tick all the boxes system administrators require.

Using Docker

So how easy is it to use? Installing it is trivial, just download the install package and copy the Docker application to your Applications folder. You might also want to download
Kitematic which provides a GUI interface to Docker, it also just requires downloading and copying the app to your Applications folder. It is just as easily installed on a Linux box.

You can also install bash completion for docker with this

curl -XGET https://raw.githubusercontent.com/docker/docker/master/contrib/completion/bash/docker > brew --prefix/etc/bash_completion.d/docker

I wish I could tell you how easy it is to build a Docker container from scratch but every time I searched DockerHub for a container I wanted someone else had already built it, or built a large chunk of it.

As an example, I wanted a container running Python 3, Jupyter and the add-on for bash notebooks. Sure, I could have built it from scratch but Continuum, the Anaconda people, already have a Docker container with Python 3 and Jupyter (along with a bunch of other useful Python libraries) installed so :-

docker run -it continuumio/anaconda3 /bin/bash

which will download and run the Python 3 version of Anaconda in a container. Then when the container runs (the -it makes it an interactive container) :-

pip install bash_kernel
python -m bash_kernel.install

then exit the container and at the terminal prompt

docker ps -a
docker commit <container_name> tonyw/jupyter

The ps -a lists all the containers so I know which one to commit and the commit saves the changed container with (optionally) a new name. Now we can run the new container.

docker -d -p 8888:8888 -v /Users:/Users -rm tonyw/jupyter 
 jupyter notebook --ip='*' --port=8888 
 --notebook-dir /Users/tonyw/dev/Notebooks 
 --no-browser

This runs the Docker container in ‘daemon’ mode and when the container starts runs the command at the end, in this case Jupyter in notebook mode.

Of course if I just want to run Python 3.5 instead of Jupyter I can always replace the -d with -it and the jupyter command with bash and I get a shell in the container.

Docker Magic

Now all the Docker gurus out there are screaming at me that I should use a Dockerfile to build my custom container and define all sorts of magical stuff like the default command to run when the container starts and the working directory and all the rest so I didn’t need them all in my long command line. Frankly, while that would probably be a good idea I haven’t quite managed to learn how to do all that automated magic and it almost seems like too much work.

Perhaps for my next blog post.

Further Reading

Macadmins Dockerhub
Pepijn Bruienne’s talk on Docker from PSU MacAdmins 2015

BBEdit Really Doesn’t Suck

Recently, with version 11, BBEdit introduced a demo mode so I thought to take another look at the big brother of TextWrangler. I have to say BareBones Software’s tag line for BBEdit is true “BBEdit – It doesn’t suck!”.

There are two tasks that I use an editor for, writing Python and writing Markdown so those are the two that I looked at.

There are a number of things you can do to improve BBEdit as a Python IDE. The first is to install Dash. This is a brilliant tool for searching documentation sets and can be easily searched from BBEdit. Just select a library call and choose “Find In Reference…” under the Search menu and BBEdit will pass the search to Dash. Dash will search across all your documentation sets but it is easy to set the sort order so the Python entries are close to the top and in the Dash results window there is a little Python icon next to the Python results.

The other neat item under the Search menu is “Find Definition”, this will find where in your file a function is defined – useful if you have a long source file.

But how does that work if our project is in multiple source files? Well, Unix has long known of that problem and had a solution. It’s a tags file, first used in vi. This is a file that lists all the function definitions and variables used in all the files in a directory tree. Not only can BBEdit use a tags file but it can (using the open source utility cats) generate them. At the top of your project directory tree, on the command line bbedit --maketags will generate a tags file and now “Find Definition” will work across all the Python files in the tree.

BBEdit can also run a syntax check across your source. You will find “Check Syntax under the ”#!“ menu which also allows you to run your Python code. The final entry in this menu ”Show Module Documentation” displays a new text window with the output from running pydoc across your file. I love this, it encourages me to properly document my code as I write with pydoc strings for each function. The output is extremely useful as a memory aide for large programs and modules.

Next up is running a lint across our Python source. BBEdit comes with another command line tool, bbresults which turns formatted error output from Unix command-line tools into a BBEdit results windows. This is an exceptionally neat trick. At the command line flake8 example.py | bbresults will give you a window in BBEdit with each of the errors and warnings listed and a click on one will take you to the exact spot in your source. If you don’t have flake8 installed then you can install it with conda or pip.

By the way, this works because the bbedit and bbresults command line tools understand the +n argument syntax for going to line n in a file. Sublime Text and other editors on the Mac could learn this.

A final tip for programmers, BBEdit recommends setting the $EDITOR shell variable to bbedit -w where the -w flag has the bbedit command line tool wait till you close the window before exiting. If you add the --resume flag as well then when you close the window in BBEdit it will return the Terminal to the front. Exceptionally handy.

Markdown

One complaint I would make, and I make it about a number of editors, is that the Markdown syntax highlighting is on the stupid side. This is generally due to the flaws in using nothing but regular expressions to do the highlighting. The most obvious flaw is that underlines in such things as a URL will trigger highlighting for italics.

If you want you can “lint” your prose using proselint and bbresults. Personally I find proselint rarely throws up something I actually want to change but your mileage might vary, it’s a good tool for looking at prose text.

BBEdit has no special facilities for writing Markdown such as inserting the codes for text styles or formatting but it does have the ability to use “Clippings”, a short piece of text, and clippings can be kept in sets and a clipping can have a keyboard shortcut. I don’t use it, I have a few Keyboard Maestro macros for such things as web links and otherwise just type the few extra keystrokes.

BBEdit also has “Text Filters”, which allow you to run the current selection through a script. For Markdown I have one that turns tab separated text into a Markdown table, incredibly useful for tables copied from a spreadsheet. Not sure where I got it but I suspect it was from Brett Terpstra’s blog.

BBEdit is a good editor, well worth the $50 purchase price and has a number of advantages over it’s free little brother TextWrangler. As both a general purpose editor and an editor for programming I’d have to say that it is the best editor available on the Mac at the moment though Sublime Text comes close.

Bash Notebooks in Jupyter

So IPython now has a back end called Jupyter that supports a large number of kernels as well as Python.

This means we can write Jupyter notebooks with bash for example.

Here’s how.

Installing Python 3

The bash kernel requires Python 3 and I’ve only installed Anaconda using Python 2.7. How do we easily get Python 3.4 or later as the kernel requires?

Fortunately conda can create a Python environment with Python 3 in one easy step.

conda create -n py3k python=3 ipython notebook

That creates the environment using Python 3 and adds the IPython and Juypter system with notebook support.

This will give us all we need so then source activate py3k will turn it on.

You can tell you are in a conda environment as the bash prompt is prepended with the name of the environment like so py3k)Air:~/dev/Notebooks tonyw$ Continue reading

Why Alex Is Wrong And What Are Apple’s real Software Problems

OK, that headline is a bit inflammatory and an insult to a perfectly good tech journalist but Alex Kidman here is the latest in a long line of commentators to take a swipe at Apple for the quality of its software. Unfortunately he grabs the wrong end of the clue stick.

I am not going to argue that there aren’t some things that Apple could do to improve software quality, indeed I will point out some of them before I conclude. What I would like to do is look at some of the reasons there are problems. I will also point out some of the things that won’t fix the problems and why. As I do I’ll show why Alex is partly wrong though partly correct at the core of his arguments.

First, software is hard. I started out my professional life as a programmer, mostly writing code for accounting software, most of the time in C. It is impossible for anyone who has not spent several years writing software to understand the complexity of what you do and the difficulties you find in proving it’s correct. Continue reading

Keyboard Maestro Updated To 6

Keyboard Maestro 6 has just arrived and this already marvellous utility has just improved a bunch. I could tell you about the new features or you could read the press release from Stairways.

The two big gee whizz new features for me are browser actions and the ability to write my own actions. I will have more on it when I have some time to rejig all my existing macros and work on adding some more.

It’s not a huge reworking of the entire application but it is a decent improvement so if you haven’t tried then visit the website for more details and download a free trial.

(Re)Building a Mac Media Centre

Mac mini

This week I decided to bite the bullet and re-install everything on the Mac Mini that I use as a media centre. The little silver box attached via an HDMI cable to my honking big Yamaha A/V receiver and Sony TV.

It’s a great little box. I got the Server version so it has two 500Gb drives powered by a 2Ghz i7 and 16Gb of third party RAM. With all that it’s a fast little box. I can’t imagine how fast it would be with an SSD to boot off.

It had to be shifted to 10.8 from 10.7 anyway so I decided the entire system and all the apps had to go. I’d installed a bit too much cruft and fiddled a bit too much for the health of the system. I had it set up with the System and apps on the first drive, all the server software using the second and all the media on a 4Tb USB external. I’d love to have a Thunderbolt external but a 4Tb Thunderbolt drive is still too expensive for this little black duck and the current drive already has more than 1.5Tb of media on it.

Continue reading

Foldify Gets An Excellent Update

Me as "SuperLover" - OK, you can stop laughing now. (Click to enlarge the image.)

Me as “SuperLover” – OK, you can stop laughing now. (Click to enlarge the image.)

Foldify, the papercraft app for the iPad I previously wrote up was updated to version 1.1 today and they have hit the ball out of the park.

The biggest new feature and high on the wish list of many users, including myself, is the ability to import a photo to place on your papercraft figure. This works extremely well as you select or take a photo then you can crop it before placing it on the papercraft just like a stamp so it can then be resized and rotated to place it perfectly.

They have also added a paint bucket tool which I found extremely useful in re-colouring an existing model. It works just as you would expect it to work though I think it would be nice if it didn’t go over a fold line. The final tool added is an eye dropper so you can pick up a colour, extremely useful when working with one of the example models.

The developers have also added to the blank templates, 6 more to take the total to 16 including a heart shape for Valentine’s Day. Also for Valentine’s Day are two new packs of extras, a free assortment of 28 elements (you can see most in my picture above) and 35 elements of underwear, most only suitable for female figures but seven or eight could be used for male figures.

The final change is the addition of a “Trending” tab on top of the previous “New” and “Best” tabs in the online template “store”.

There are still some things I’d like to see added. The biggest would be some more examples already painted and ready to alter. I’d also like it if you could touch up a model you download from the online ones rather than just be able to print them. To really nitpick I’d like to be able to adjust the size of the paintbrush and eraser — now you can add a photo I found myself erasing large areas of a model and this can be tiring with the tiny tools as they are. Even the ability to choose three or four sizes would be good.

That said this little app is an essential for iPad owners who like to do some cutting and folding. Kids will love it now they can have their face on a model.