Designing Scriptorium

I just released Scriptorium, a small console program. Here are some notes on how I used argparse to do that.

We need a function to parse our arguments. Parsing is taking the line of words from the command line and processing them to extract the structure and meaning. The term ‘word’ can be fraught with complexity on the shell command line but a simple definition is any set of characters delimited by a space or matching quotes.

Scriptorium has a simple structure, scriptorium <command> [<argument>]. Some commands don’t have any arguments, for some arguments are entirely optional and for others arguments are required. But let’s start building that parser.

    """ build our command line parser """
    parser = argparse.ArgumentParser(
        epilog="for command help: `scriptorium <command> -h`"
    )
    subparsers = parser.add_subparsers(description="", required=True)

You can also see we are starting to build some help in ‘epilog’ – this is the final line printed when you run scriptorium --help. Then we need to have some code to parse each individual commands arguments. This is a subparser.

How about we look at a simple command? scriptorium commit

We create the subparser for the command:

    #
    # create parser for `commit`
    #
    parser_commit = subparsers.add_parser("commit", help="git commit")

Notice that as we add the command we also add more help.

Time for the arguments.

    parser_commit.add_argument(
        "-p",
        "--push",
        help="do a git push after commit",
        action="store_true",
    )
    parser_commit.add_argument(
        "-m",
        "--message",
        help="set commit message",
    )

You see how easily we add an argument. The short and the long argument and the help are obvious, the action not so much. What that does is tell the subparser what to do if it sees the argument. In this case it will set a class variable push to true. What happens with the message parameter? In this case the parser performs the default action. It stores the next ‘word’ after --message in a class variable message.

At this point I suggest you run scriptorium -h and scriptorium commit -h to see how all the help hangs together.

There is one more thing we need to do, we need to give our parser some code to run when it finishes the parsing.

    parser_commit.set_defaults(func=Scripts.do_commit)

I defined a class to contain all the required functions called Scripts and from the above example you can guess what I named each function. A function is an object so we can assign it to a variable.

How do we call all this and get it to do it’s thing?

    fred = Parser()  # I just love fred, he does all the dirty work
    # handle no arguments on command line, fred doesn't do this well
    if len(argv) == 1:
        print("Missing subcommand")
        fred.parser.print_help()
        exit(1)
    args = fred.parser.parse_args()
    if args:
        args.func(args, jpc)
    exit()

Essentially we create a parser object and the __init__ function in that object runs our code that adds the command and arguments. Then we call the parser and get back a dictionary that contains all the words in our command line. We pass that array and a jpc object to the default function we specified when we created the command parser. The jpc object is just a class that carries useful variables such as the URL needed to access our Jamf server. jpc is an acronym for Jamf Pro Cloud. Since we control the function we can decide what it should get passed.

This is the really smart bit, all the logical flow of our Python script is built in to our parser so all we need to do is call it. I have to stress it’s not my smarts, it’s the smarts in argparse.

How about we have a look at do_push to understand how a do_ function works? It is ridiculously simple.

    def do_push(args, jpc):
        command = ["git", "push"]
        Scripts.both_repos(args, jpc, command)

Because a git push is simple all it does is create a command string with two words and passes it to both_repos which contains all the logic to run a shell command in both the XML and text directories along with some error handling. Every command (apart from list and verify) calls this at least once.

Obviously our other do_ functions are more complex but only because they are doing a more complex task. The design is all here.

Have a look at something more complex.

    def do_commit(args, jpc):
        """ do a git commit """
        """ this commit, and optionally a push, on both directories """

        lst = Scripts.do_up(args, jpc)
        command = ["git", "add", "*"]
        Scripts.both_repos(args, jpc, command)
        msg = args.message if args.message else lst
        command = ["git", "commit", "-m", msg]
        Scripts.both_repos(args, jpc, command)
        if args.push:
            command = ["git", "push"]
            Scripts.both_repos(args, jpc, command)

For logical reasons we do an ‘up’. The up function returns a list of scripts processed. Then to make sure everything is ready to commit there is a git add. Then the line msg = args.message if args.message else lst, this has a look to see if the argument ‘message’ has been set to anything and uses it if it is. You can see just below it how we check to see if ‘push’ is true, remember it will be set to true if you have ‘–push’ as an argument. All the hard part is done by argparse.

There is one final trick I’d like to show you. When you add a script to Jamf Pro you need to set the priority to one and only one of ‘after’, ‘before’ or at reboot. How do we handle a mutually exclusive group?

    priority = parser_add.add_mutually_exclusive_group()
    priority.add_argument(
        "-a",
        "--after",
        help="run script with priority 'after'",
        action="store_true",
    )
    priority.add_argument(
        "-b",
        "--before",
        help="run script with priority 'before'",
        action="store_true",
    )
    priority.add_argument(
        "-r", "--reboot", help="run script at reboot", action="store_true"
    )

As it happens argparse has a method that allows you to quickly and easily define a mutually exclusive group. What happens if the user tries to use two of the mutually exclusive arguments:

scriptorium add -a -b
usage: scriptorium add [-h] [-f FILENAME] [-c CATEGORY] [-n NOTES] [-p | -d] [-m MESSAGE] [-a | -b | -r] [-z]
scriptorium add: error: argument -b/--before: not allowed with argument -a/--after

argparse sees the error and gives the user some help before pointing out the exact error. This took absolutely no code by me once I had defined the mutually exclusive group. Our parser even returns an error to the shell.

Go and have a look at the source code now and I hope you can see the magic that is argparse. I don’t think as Python programmers we need to be scared of writing command line tools.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s