PatchBot – Zero Touch Patch Management #2

Last post I detailed the first steps taken by PatchBot, building and uploading a new version of an application package.

This post I will explain the next step, updating the testing patch policy.

First thing I should explain is why we don’t do this when we build and upload the package. It boils down to the reliability of our patch definition feed. If every time a new version was available the patch definition feed was updated at exactly the same time we could have done it all in JPCImporter. Unfortunately the patch definitions are only updated every 12 hours (I think) and that’s enough of a window for Murphy. Kinobi keep on decreasing the window but no matter how narrow you know Murphy will have his way, so defensive design and coding.

Having a separate custom processor is also appealing for development reasons and to allow others to use JPCImporter on it’s own.

We will also need a recipe for our custom processor. Here’s one example, pretty simple.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"&gt;
<plist version="1.0">
<dict>
<key>Description</key>
<string>Move Abstract package into testing</string>
<key>Identifier</key>
<string>com.honestpuck.ptch.Abstract</string>
<key>Input</key>
<dict></dict>
<key>MinimumVersion</key>
<string>1.0.0</string>
<key>Process</key>
<array>
<dict>
<key>Arguments</key>
<dict>
<key>title</key>
<string>Abstract</string>
<key>patch</key>
<string>Abstract</string>
</dict>
<key>Processor</key>
<string>com.honestpuck.PatchBot/PatchManager</string>
</dict>
</array>
</dict>
</plist>
view raw Abstract.ptch.recipe hosted with ❤ by GitHub
Abstract.ptch.recipe

The title field is the name of the application used in the package name and the patch field is the display name of the title in the “Patch Management” section of Jamf Pro. These are often the same but not always and as the default display name is provided as part of the patch definition feed I don’t want to change it and possibly cause confusion.

PatchManager.py

So now we introduce our second AutoPkg custom processor, PatchManager.py. It goes in to exactly the same folder as JPCImporter.py

#!/usr/bin/env python3
#
# PatchManager v2.0
#
# Tony Williams 2020-03-10
#
# ARW 2020-06-22 Major code clean and refactor
#
"""See docstring for PatchManager class"""
from os import path
import subprocess
import plistlib
import xml.etree.ElementTree as ET
import datetime
import logging.handlers
from time import sleep
import requests
from autopkglib import Processor, ProcessorError
APPNAME = "PatchManager"
LOGLEVEL = logging.DEBUG
__all__ = [APPNAME]
class Package:
"""A package. This exists merely to carry the variables"""
title = "" # the application title matching the test policy
patch = "" # name of the patch definition
name = "" # full name of the package '<title>-<version>.pkg'
version = "" # the version of our package
idn = "" # id of the package in our JP server
class PatchManager(Processor):
"""Custom processor for autopkg that updates a patch policy
and test policy for a package"""
description = __doc__
input_variables = {
"title": {"required": True, "description": "Application title"},
"patch": {"required": True, "description": "Patch name"},
}
output_variables = {
"patch_manager_summary_result": {"description": "Summary of action"}
}
pkg = Package()
def setup_logging(self):
"""Defines a nicely formatted logger"""
LOGFILE = "/usr/local/var/log/%s.log" % APPNAME
self.logger = logging.getLogger(APPNAME)
# we may be the second and subsequent iterations of JPCImporter
# and already have a handler.
if len(self.logger.handlers):
return
ch = logging.handlers.TimedRotatingFileHandler(
LOGFILE, when="D", interval=1, backupCount=7
)
ch.setFormatter(
logging.Formatter(
"%(asctime)s %(levelname)s %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
)
)
self.logger.addHandler(ch)
self.logger.setLevel(LOGLEVEL)
def policy(self):
"""Download the TEST policy for the app and return the version string"""
self.logger.warning("******** Starting policy %s *******" % self.pkg.title)
# Which pref format to use, autopkg or jss_importer
autopkg = False
if autopkg:
plist = path.expanduser("~/Library/Preferences/com.github.autopkg.plist")
fp = open(plist, "rb")
prefs = plistlib.load(fp)
self.base = prefs["JSS_URL"] + "/JSSResource/"
self.auth = (prefs["API_USERNAME"], prefs["API_PASSWORD"])
else:
plist = path.expanduser("~/Library/Preferences/JPCImporter.plist")
fp = open(plist, "rb")
prefs = plistlib.load(fp)
self.base = prefs["url"] + "/JSSResource/"
self.auth = (prefs["user"], prefs["password"])
policy_name = "TEST-{}".format(self.pkg.title)
url = self.base + "policies/name/{}".format(policy_name)
self.logger.debug("About to make request URL %s, auth %s" % (url, self.auth))
ret = requests.get(url, auth=self.auth)
if ret.status_code != 200:
self.logger.debug(
"TEST Policy %s not found error: %s" % (policy_name, ret.status_code)
)
raise ProcessorError(
"Policy get for: %s failed with code: %s" % (url, ret.status_code)
)
self.logger.debug("TEST policy found")
root = ET.fromstring(ret.text)
self.pkg.idn = root.find("package_configuration/packages/package/id").text
self.pkg.name = root.find("package_configuration/packages/package/name").text
self.logger.debug(
"Version in TEST Policy %s " % self.pkg.name.split("-", 1)[1][:4]
)
# return the version number
return self.pkg.name.split("-", 1)[1][:4]
def patch(self):
"""Now we check for, then update the patch definition"""
# download the list of titles
url = self.base + "patchsoftwaretitles"
self.logger.debug("About to request PST list %s", url)
ret = requests.get(url, auth=self.auth)
if ret.status_code != 200:
raise ProcessorError(
"Patch list download failed: {} : {}".format(ret.status_code, url)
)
self.logger.debug("Got PST list")
root = ET.fromstring(ret.text)
# loop through 'patchsoftwaretitles' list to find our title
ident = 0
for ps_title in root.findall("patch_software_title"):
if ps_title.findtext("name") == self.pkg.patch:
ident = ps_title.findtext("id")
self.logger.debug("PST ID found")
break
if ident == 0:
raise ProcessorError(
"Patch list did not contain title: {}".format(self.pkg.patch)
)
# get the patch list for our title
url = self.base + "patchsoftwaretitles/id/" + str(ident)
self.logger.debug("About to request PST by ID: %s" % url)
ret = requests.get(url, auth=self.auth)
if ret.status_code != 200:
raise ProcessorError(
"Patch software download failed: {} : {}".format(
str(ident), self.pkg.name
)
)
self.logger.debug("Got our PST")
root = ET.fromstring(ret.text)
# find the patch version that matches our version
done = False
for record in root.findall("versions/version"):
if self.pkg.version in record.findtext("software_version"):
self.logger.debug("Found our version")
if record.findtext("package/name"):
self.logger.debug("Definition already points to package")
return 0
package = record.find("package")
add = ET.SubElement(package, "id")
add.text = self.pkg.idn
add = ET.SubElement(package, "name")
add.text = self.pkg.name
done = True
break
if not done:
# this isn't really an error but we want to know anyway
# and we need to exit so raising an error is the easiest way to
# do that feeding info to Teams
raise ProcessorError(
"Patch definition version not found: {} : {} : {}".format(
str(ident), self.pkg.name, self.pkg.version
)
)
# update the patch def
data = ET.tostring(root)
self.logger.debug("About to put PST: %s" % url)
ret = requests.put(url, auth=self.auth, data=data)
if ret.status_code != 201:
raise ProcessorError(
"Patch definition update failed with code: %s" % ret.status_code
)
self.logger.debug("patch def updated")
# now the patch policy – this will be a journey as well
# first get the list of patch policies for our software title
url = self.base + "patchpolicies/softwaretitleconfig/id/" + str(ident)
self.logger.debug("About to request patch list: %s" % url)
ret = requests.get(url, auth=self.auth)
if ret.status_code != 200:
raise ProcessorError(
"Patch policy list download failed: {} : {}".format(
str(ident), self.pkg.name
)
)
root = ET.fromstring(ret.text)
# loop through policies for the Test one
pol_list = root.findall("patch_policy")
self.logger.debug("Got the PP list and name is: %s" % self.pkg.name)
for pol in pol_list:
# now grab policy
self.logger.debug("examining patch policy %s" % pol.findtext("name"))
if "Test" in pol.findtext("name"):
pol_id = pol.findtext("id")
url = self.base + "patchpolicies/id/" + str(pol_id)
self.logger.debug("About to request PP by ID: %s" % url)
ret = requests.get(url, auth=self.auth)
if ret.status_code != 200:
raise ProcessorError(
"Patch policy download failed: {} : {}".format(
str(pol_id), self.pkg.name
)
)
# now edit the patch policy
root = ET.fromstring(ret.text)
self.logger.debug(
"Got patch policy with version : %s : and we are : %s :"
% (root.findtext("general/target_version"), self.pkg.version)
)
if root.findtext("general/target_version") == self.pkg.version:
# we have already done this version
self.logger.debug("Version %s already done" % self.pkg.version)
return 0
root.find("general/target_version").text = self.pkg.version
root.find("general/release_date").text = ""
root.find("general/enabled").text = "true"
# create a description with date
now = datetime.datetime.now().strftime(" (%Y-%m-%d)")
desc = "Update " + self.pkg.title + now
root.find("user_interaction/self_service_description").text = desc
data = ET.tostring(root)
self.logger.debug("About to change PP: %s" % url)
ret = requests.put(url, auth=self.auth, data=data)
if ret.status_code != 201:
raise ProcessorError(
"Patch policy update failed with code: %s" % ret.status_code
)
pol_id = ET.fromstring(ret.text).findtext("id")
self.logger.debug("patch() returning pol_id %s", pol_id)
return pol_id
raise ProcessorError("Test patch policy missing")
def main(self):
"""Do it!"""
self.setup_logging()
self.logger.debug("Starting Main")
# clear any pre-exising summary result
if "patch_manager_summary_result" in self.env:
del self.env["patch_manager_summary_result"]
self.logger.debug("About to update package")
self.pkg.patch = self.env.get("patch")
self.pkg.title = self.env.get("title")
self.pkg.version = self.policy()
pol_id = self.patch()
if pol_id != 0:
self.env["patch_manager_summary_result"] = {
"summary_text": "The following packages were sent to test:",
"report_fields": ["patch_id", "title", "version"],
"data": {
"patch_id": pol_id,
"title": self.pkg.title,
"version": self.pkg.version,
},
}
print("%s version %s sent to test" % (self.pkg.title, self.pkg.version))
else:
self.logger.debug("Zero policy id %s" % self.pkg.patch)
if __name__ == "__main__":
PROCESSOR = PatchManager()
PROCESSOR.execute_shell()
view raw PatchManager.py hosted with ❤ by GitHub
PatchManager.py

Once again the first 25 lines are housekeeping. This time we have two classes, the first, Package, is merely a convenience to hold all the information on a particular package. The title and patch come from the AutoPkg recipe and the pkg_name, version, and pkg_id come from the JP server.

The second class, PatchManager, does the actual work. Once again we open by setting up the logging and our input and output variables before our first function, policy. This was copied almost verbatim from an earlier version of the system and every time I look at it I think how much nicer it would look if I grabbed the JSON from the Jamf Pro P API instead of the XML but I’ve stuck with “if it ain’t broke don’t fix it”. Generally, if my code is both reading and writing from the Classic API, which only accepts XML, I’ll use the XML and ElementTree module throughout. It’s only when I’m writing something that purely extracts data from the API will I use the JSON. It might look a little untidier and be just that little bit harder to debug but you get used to it.

The next function, patch, does all the work.

First off we would like to grab the patchsoftwaretitle record for our title, but for reasons unknown the patchsoftwaretitle endpoint does not allow you to specify one by name, only ID. Instead we get the entire list and search it for our title before asking for that title by ID.

Now we need to find the version definition in the patch software title. If we don’t find a version definition to match our latest package our patch feed is slow to update, we give up and try again tomorrow, raising an error as we go so that a note gets added to our Autopkg report and a message goes into Teams. If we do find one we update it to point to our package.

Next step is to update out Test patch policy to use the new version. We also set the self service description of the policy to “Update <title> (<date>)” where date is today’s date in ISO 8601 format, e.g. 2020-06-29. That’s crucial as it sets the clock for the delay before we move the package from test into production.

Plumbing

We need to add a few lines to the autopkg.sh script to run it.

# run the patch management
/usr/local/bin/autopkg run --recipe-list=/Users/"$(whoami)"/Documents/autopkg_bits/patch.txt \ 
 --report-plist=/Users/"$(whoami)"/Documents/patch.plist \ 
 -k FAIL_RECIPES_WITHOUT_TRUST_INFO=yes

# messages to MS Teams
/Users/"$(whoami)"/Documents/autopkg_bits/PatchTeams.py \   /Users/"$(whoami)"/Documents/patch.plist

You can see the similarity with the first block. Once again we are using a text list of recipes to feed AutoPkg, once again we are calling a script to send messages to Teams.

#!/usr/bin/env python3
# PatchTeams.py v1.0
# Tony Williams 25/07/2019
#
"""See docstring for PatchTeams class"""
import json
import plistlib
import os.path as path
import datetime
import logging
import logging.handlers
import sys
import requests
__all__ = ["PatchTeams"]
# logging requirements
LOGFILE = "PatchTeams.log"
LOGLEVEL = logging.DEBUG
class PatchTeams:
"""When given the location of an output plist from Autopkg parses it
and sends the details on packages productionized to Jamf Pro to Teams
"""
description = __doc__
def __init__(self):
# extremely dumb command line processing
try:
self.plist = sys.argv[1]
except IndexError:
self.plist = "autopkg.plist"
# URL of Teams webhook
self.url = "https://outlook.office.com/webhook/&quot;
# token
self.url += "76ea46bf-3dda-41f0-831d-b0dc655e4f97@43f93f8a-55a8-4263-bd84"
self.url += "-e03688a2ab2d/IncomingWebhook/0ac15911fcfa42deb"
self.url += "1d07f0672950542/63a48cfb-c3ef-4ee9-be63-fafbe4177f30"
# set up logging
now = datetime.datetime.now().strftime("%d/%m/%Y %H:%M")
frmt = "%(levelname)s {} %(message)s".format(now)
# set up logging
logging.basicConfig(filename=LOGFILE, level=LOGLEVEL, format=frmt)
self.logger = logging.getLogger("")
# set logging formatting
# ch = logging.StreamHandler()
ch = logging.handlers.TimedRotatingFileHandler(
LOGFILE, when="D", interval=7, backupCount=4
)
ch.setFormatter(logging.Formatter(frmt))
self.logger.addHandler(ch)
self.logger.setLevel(LOGLEVEL)
# JSON for the message to Teams
# "sections" will be replaced by our work
self.template = """
{
"@context": "https://schema.org/extensions&quot;,
"@type": "MessageCard",
"themeColor": "0072C6",
"title": "Patch Manager",
"text": "Patches to testing",
"sections": [
]
}
"""
# JSON for a section of a message
# we will have a section for each package uploaded
# in this Autopkg run
# (It looks ugly to keep pylint happy)
self.section = """
{"startGoup": "true", "patch_id": "**AppName**", "text": "version"
}
"""
# JSON template for the error message card.
self.err_template = """
{
"@context": "https://schema.org/extensions&quot;,
"@type": "MessageCard",
"themeColor": "0072C6",
"title": "Autopkg",
"text": "Patch management errors",
"sections": [
]
}
"""
# JSON template for a single error on error card.
self.err_section = """
{
"text": "A long message",
"startGoup": "true",
"title": "**Firefox.pkg**"
}
"""
# JSON template for the error message card.
self.none_template = """
{
"@context": "https://schema.org/extensions&quot;,
"@type": "MessageCard",
"themeColor": "0072C6",
"title": "PatchManager",
"text": "**Empty Run**"
}
"""
def PatchTeams(self):
"""Do the patches changed!"""
self.logger.info("Starting Run")
sections = []
empty = False
jsr = "patch_manager_summary_result"
try:
fp = open(self.plist, "rb")
pl = plistlib.load(fp)
except IOError:
self.logger.error("Failed to load %s", self.plist)
sys.exit()
item = 0
if jsr not in pl["summary_results"]:
self.logger.debug("No Patch results")
empty = True
else:
pkgs = pl["summary_results"][jsr]["data_rows"]
for p in pkgs:
sections.append(json.loads(self.section))
name = p["title"]
version = p["version"]
self.logger.debug("Version: %s Name: %s", version, name)
sections[item]["title"] = "**%s**" % name
sections[item]["text"] = version
item = item + 1
j = json.loads(self.template)
j["sections"] = sections
d = json.dumps(j)
requests.post(self.url, data=d)
# do the error messages
fails = pl["failures"]
if len(fails) == 0: # no failures
if empty:
requests.post(self.url, self.none_template)
sys.exit()
sections = []
item = 0
for f in fails:
sections.append(json.loads(self.err_section))
sections[item]["title"] = "**%s**" % f["recipe"]
sections[item]["text"] = f["message"].replace("\n", " ")
item = item + 1
j = json.loads(self.err_template)
j["sections"] = sections
d = json.dumps(j)
requests.post(self.url, d)
if __name__ == "__main__":
PatchTeams = PatchTeams()
PatchTeams.PatchTeams()
view raw PatchTeams.py hosted with ❤ by GitHub
PatchTeams.py

We will also need to write all those recipes and create recipe overrides for them. Our override directory is getting mighty full.

Next post we will look at the last custom processor and moving our patch into production.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s