Using Python and AppleScript to get notified if a site is down

I manage a handful of websites, like this one. Having built a few on other platforms, such as Drupal, I’m familiar with the dreaded error “The website encountered an unexpected error. Please try again later." On sites that I don’t check on frequently, it can be an embarrassment when people begin emailing you with questions about the site being down.

I wrote the following Python script to deal with the problem:

#!/usr/bin/python

import urllib
from subprocess import Popen, PIPE

RECIPIENT = "[email protected]"
URL_TO_CHECK = "http://www.example.com"
ERR_MSG = "Your website is down."

def sendMessage(message):
	scpt = '''
	tell application "Messages" to send "{0}" to buddy "{1}" of (service 1 whose service type is iMessage)
	'''.format(message,RECIPIENT)
	args = []
	p = Popen(['osascript', '-'] + args, stdin=PIPE, stdout=PIPE, stderr=PIPE)
	stdout, stderr = p.communicate(scpt)

try:
	fh = urllib.urlopen(URL_TO_CHECK)
except IOError:
	sendMessage(ERR_MSG)
else:
	# handle database type errors from Drupal sites
	site_content = fh.read()
	target_str = "The website encountered an unexpected error. Please try again later."
	if site_content.find(target_str) != -1:
		sendMessage(ERR_MSG)
	else:
		print "No error"

I run this as a scheduled job using launchd and as long as I have a Messages-capable device with me, I’ll get notifications of issues with the site.

Dynamic UI lists in Indigo 6

Indigo 6 is a popular home automation controller software package on the Mac. Extensibility is one of its main features and it allows users to add a range of features to suit their needs.

Using Python scripting, users can create plugins that provide extended functionality. These plugins can provide a custom configuration UI to the user. Since the documentation around a particular feature - dynamic lists was lacking, I’ve written up my approach here.

Since I live in Canada, the excellent NOAA plugin doesn’t work for me. However Environment Canada provides an XML-based weather data API that we could package into an Indigo plugin. Since the number of Environment Canada station locations is large, I would like the user to select a province first then select locations within that province. This means that I must use a dynamic list for the locations and reload the location list dynamically when the province changes. The solution turned out to be simple. Perhaps it could be even simpler. This is just what I came up with.

Devices.xml configuration

<?xml version="1.0"?>
<Devices>
    <!-- define devices -->
    <Device type="custom" id="station">
        <Name>Weather station</Name>
        <ConfigUI>
            <!-- choose location -->
            <Field id="province" type="menu">
                <Label>Province:</Label>
                <List class="self" filter="" method="listProvinces"/>
                <CallbackMethod>provinceChanged</CallbackMethod>
            </Field>
            <!-- choose location within province -->
            <Field id="location" type="menu">
                <Label>Location:</Label>
                <List class="self" filter="" method="listStations" dynamicReload="true"/>
            </Field>
        </ConfigUI>
    </Device>
</Devices>

In the device configuration I’ve specified a province field and a location field. The former provides a callback method provinceChanged where I can deal with filtering the locations based on the province selection. The other key here is to make the location field dynamically-reloadable (dynamicReload="true".) By doing this, we get another call to the list generator method listStations when the province is selected.

Province selection callback

In plugin.py, I must provide a callback method provinceChanged to save my selection:

def provinceChanged(self, valuesDict, typeId, devId):
    self.selectedProvince = valuesDict['province']

Here’s where the solution might be simpler. The documentation is ambiguous about the status of valuesDict if the device hasn’t been saved yet. Based on that ambiguity, I decided to save the selected province as an instance variable of my Plugin class.

Providing a filtered location list

My dynamic list generator for the locations takes the selected province instance variable into consideration so that when the list is dynamically reloaded, I get a chance to filter the list by province.

def listStations(self, filter="", valuesDict=None, typeId="", targetId=0):
    locations = []
    stations = []
    self.debugLog(u"Generating stations")
    stations = self.locationDB.stationsForProvice(self.selectedProvince)
    for loc in stations:
        option = loc[0]
        city,province = loc[1].encode('utf-8'),loc[2].encode('utf-8')
        stationName = "{0} ({1})".format(city,province)
        locations.append(stationName)
    return locations

I have a suspicion there’s an easier way. If you know of one, let me know and I’ll share it.

Import and tag with Hazel and DEVONthink Pro Office

Hazel and DEVONthink make a great pair as I’ve written before. Using AppleScript, it’s possible to take the import workflow even further by tagging incoming files automatically. Use case I download a lot of mp3 files containing pronunciation of words in a language I’ve been learning. I keep a record of these words and tag them appropriately using my hierarchical tagging system. I’d like to download the files to a directory on the desktop.

Using AppleScript with MailTags

I’m a fan of using metadata to classify and file things rather than declarative systems of nested folders. Most of the documents and data that I store for personal use are in DEVONthink which has robust support for metadata. On the email side, there’s MailTags which lets you apply metadata to emails. Since MailTags also supports AppleScript, I began to wonder whether it might be possible to script workflows around email processing.

Using AdBlock Plus to block YouTube comments

web
YouTube comments are some of the most offensive on the web. Even serious videos attract trolls bent on inscribing their offensiveness and cruelness on the web. Here’s one method of dealing with YouTube comments. Treat the comments block as an advertisement and block it.^[There are other ways of avoiding YouTube comments. I’ve used ViewPure but it’s hard to find content that way even though they seem to be working on making it more seamless to get from YouTube to ViewPure.

Introducing AnkiStats & AnkiStatsServer

The spaced repetition software system Anki is the de facto standard for foreign language vocabulary learning. Its algorithm requires lots of performance data to schedule flashcards in the most efficient way. Anki displays these statistics in a group of thorough and informative statistical graphs and descriptive text. However, they aren’t easily available for the end-user to export. Thus, the reason behind the companion projects AnkiStats and AnkiStatsServer. The premise is that you can run your own more extensive experiments and statistical tests on the data once you have it in hand.

Waking the computer to allow AppleScript to run

I have a number of AppleScript applications that need to run at odd times. These maintenance tasks often attempt to run while the computer is sleeping. Particularly those that rely on UI scripting do not function during this period. This most flexible way of dealing with this is to manipulate the power management settings directly via the pmset(1) command. The variety of options available using pmset is staggering and beyond the scope of this post.

An easier way to automate synchronization of Anki profiles with AppleScript

After waking up this morning with my mouse locked onto the Anki icon in the dock and trying to figure out how to get Activity Monitor up and running so I could force quite my Automator application that I described yesterday I figured it was back-to-the-drawing board. I’d like to have used the Accessibility Inspector to manipulate the PyQt objects in Anki’s windows, they aren’t exposed in a may that you can script them.

Scheduling synchronization of Anki databases on OS X

While working on a project to automatically collect statistics on my Anki databases (stay tuned…) I worked out a system for scheduling synchronization from my desktop OS X machine. Prerequisites LaunchControl is a GUI application that lets you create and manage user services on OS X Anki is a spaced repetition memorization software system The solution relies on Automator. Normally, I don’t care much for Automator. It has too many limits on what tasks I can accomplish and workflows created with it are often fragile.

Resizing of images for Anki with Hazel and ImageMagick

I use Anki to study foreign language vocabulary. It’s the de facto spaced repetition software for memorization.^[Yes, I’m aware that others exist. I’ve tried many but always have come back to Anki.] When making flashcards for language learnings, I try to use imagery as much as possible. So a card may have a Russian word on one side and just an image on the opposite side. (Since I already know the English word that the image represents, why not try to engage a different part of the brain to help with memorization?