Thursday, October 4, 2018

This piece in The Atlantic by Adam Serwer is a reminder that for Trump’s supporters, cruelty isn’t just a side-effect of Trumpism. It’s a feature. That the current U.S. president chose to mock the victim of a sexual assault by a Supreme Court nominee is hardly surprising. But what little hope I had for Trump supporters, particularly those at the Mississippi rally who joined Trump in his mockery, is gone.

“Trump’s only true skill is the con; his only fundamental belief is that the United States is the birthright of straight, white, Christian men, and his only real, authentic pleasure is in cruelty. It is that cruelty, and the delight it brings them, that binds his most ardent supporters to him, in shared scorn for those they hate and fear: immigrants, black voters, feminists, and treasonous white men who empathize with any of those who would steal their birthright.”

How can the educational system have failed so miserably than to allow the ignorant to wallow in their own ignorance and bigotry? The South is the home of millions of adherents, faithful in word, at least, to a religion of kindness, meekness and tolerance. Yet the Venn circles of evangelicals and Trump supporters is near complete. The hypocrisy is nearly palpable.

Searching the Russian National Corpus

The Russian language has a vast and nuanced vocabulary. One approach to learning the vocabulary is to approach it in frequency order. The Nicholas Brown book seems dated and the frequency ordering methodology is not clear to me. Some words seem to be clustered by the beginning letter, which seems statistically unlikely. However, it’s a convenient list and I’m slowly building a table that cross-correlates the Nicholas Brown list with the methodologically-superior Russian National Corpus. To do that I harvested the data from the Corpus and built a Python application to search the database and report the rank and frequency data from it.

Creating a sqlite3 version of the Russian National Corpus

There is a CSV version of the Corpus, but the data is not useful for ordering in a meaningful way. Instead, I took the rank ordered tabular data from the page Частотный список лемм (Frequency list of lemmas) and simply pasted it into a Numbers spreadsheet. Since Numbers is extremely slow even on a fast-performing machine, it beachballed for nearly a minute during the paste operation. After that, I exported it as CSV. To get the CSV file into a sqlite3 database, I created a new table with the following schema:

After mapping the column names to those in the CSV, the import was simple.

Accessing the sqlite3 version of the corpus using Python

Next I wrote a little Python application to access the data and return the rank or frequency of any Russian word. The only trick is that the Russian letter ë is rendered as e in the database; so any word containing ë must be altered before the search. Regular expressions to the rescue! To use the application, just launch it with -h help flag and you’ll see the calling format.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/usr/bin/python
# encoding=utf8

import sqlite3
import sys
import re
import argparse

# '/Users/alan/torrential/russian/vocabulary/RussianNationalCorpus'

# instantiate argument parser
parser = argparse.ArgumentParser(description='Search the Russian National Corpus')
# arguments
parser.add_argument('word', help='Russian word to search for')
parser.add_argument('db_path', help='Path to the sqlite db')
group = parser.add_mutually_exclusive_group()
group.add_argument('--r', action='store_true',help='Show rank order')
group.add_argument('--f', action='store_false',help='Show frequency in instances/million')
# parse
args = parser.parse_args()

word = args.word
replaced = re.sub('ё','е',word)

conn = sqlite3.connect(args.db_path)

col = "rank" if args.r else "frequency"
sql = "SELECT " + col + " FROM corpus WHERE word LIKE '" + replaced + "'"
curs = conn.cursor()
curs.execute(sql)
print curs.fetchone()[0]

And a little AppleScript

Finally, I wrote an AppleScript wrapper that I can launch with a Quicksilver keystroke trigger. The wrapper takes the word off the clipboard and calls the Python app above, replacing the contents of the clipboard with the rank order of the word. For a little fun, it speaks the rank order number in Russian! Here’s the code for the AppleScript wrapper:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
--
-- Created by: Alan Duncan
-- Created on: 2018-09-22
--
-- Copyright (c) 2018 Ojisan Seiuchi
-- Use to your heart's content; just give me a little credit
--

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

set dbPath to "/Users/alan/torrential/russian/vocabulary/RussianNationalCorpus"

set errorFlag to 0
set w to the clipboard
set cmd to "python /Users/alan/Documents/dev/scripts+tools/getRussianRank.py " & w & " " & dbPath & " --r"

try
set rank to do shell script cmd
set the clipboard to rank
on error errMsg
say "плохо"
set errorFlag to 1
end try
if errorFlag is 0 then
set saying to "Готово " & (rank as string)
say saying
end if

If you want a pre-built sqlite3 version of the Russian National Corpus, here it is.

Sunday, September 16, 2018

Regex 101 is a great online regex tester.


Speaking of regular expressions, for the past year, I’ve used an automated process for building Anki flash cards. One of the steps in the process is to download Russian word pronunciations from Wiktionary. When Wiktionary began publishing transcoded mp3 files rather than just ogg files, they broke the URL scheme that I relied on to download content. The new regex for this scheme is: (?:src=.*:)?src=\"(\/\/.*\.mp3)

Edit 2018-09-17: Nope, still not right. This is the new working version:
data-.+\s?/?><source\s+src=\"(\/\/.*\.mp3)


Gina Loudon a liar and an idiot. She claims in her recent book proclaiming the sanity of Donald Trump that she has a Ph.D. in psychology. In fact, she does not. Her degree is from an online “school.”

Thursday, September 13, 2018

Politico has a piece today about Trump’s outrageous claims in the face of weather disasters. In almost every context, he reveals himself to be an abject fool; but lurking beneath that idiocy is another layer of loathsomeness - the complete lacking in understanding of science. I want a reporter to ask him any of the following questions about hurricanes:

  • “Mr. Trump, can you describe for us your understanding of how hurricanes form?”
  • “What role do Coriolis forces play in the formation of tropical cyclones.”
  • “Given that hurricanes possess massive amounts of energy, what are the sources of that energy?”

An article from the Times on why yelling at children is comparable to physical punishment. Children who are subjected to yelling have lower self-esteem, and more depressive and anxiety symptoms.[1]


  1. The article cites a study that shows a reciprocal amplifying effect of yelling and behavioural problems: "Mothers’ and fathers’ harsh verbal discipline at age 13 predicted an increase in adolescent conduct problems and depressive symptoms between ages 13 and 14. A child effect was also present, with adolescent misconduct at age 13 predicting increases in mothers’ and fathers’ harsh verbal discipline between ages 13 and 14."

How fascism works

A recent piece in The Atlantic by Peter Beinart filled in a cognitive gap in understanding how a large minority of U.S. citizens continue to support an abjectly incompetent, almost certainly criminal, willfully ignorant, and generally hateful man as president. The article Why Trump supporters believe he is not corrupt makes the argument that when Trump defenders concern themselves with the idea of corruption they are not thinking of political corruption so much as corruption of the purity. This is consistent with Jonathan Haight’s research into the determinants of a person’s moral judgments as a function of political affiliation.[1] Conservatives are likelier than liberals to concern themselves with tradition and purity. When Donald Trump uses the word disgusting which he has done scores of times on Twitter, he’s invoking the conservative fear of taint. The Special Prosecutor’s inquiry into possible collusion and other crimes committed during the 2016 elections, in Trump’s view, are not only unlawful, biased, or unfavourable in some other objective way. It is, to Trump, disgusting (“this Rigged and Disgusting Witch Hunt.”)

According to Beinart, the key to understanding the failure of Trump’s supporters to acknowledge his corruption is the result of how they define corruption.

In a forthcoming book titled How Fascism Works, the Yale philosophy professor Jason Stanley makes an intriguing claim. “Corruption, to the fascist politician,” he suggests, “is really about the corruption of purity rather than of the law. Officially, the fascist politician’s denunciations of corruption sound like a denunciation of political corruption. But such talk is intended to evoke corruption in the sense of the usurpation of the traditional order.”

Why were Trump’s supporters so convinced that Clinton was the more corrupt candidate even as reporters uncovered far more damning evidence about Trump’s foundation than they did about Clinton’s? Likely because Clinton’s candidacy threatened traditional gender roles. For many Americans, female ambition—especially in service of a feminist agenda—in and of itself represents a form of corruption. “When female politicians were described as power-seeking,” noted the Yale researchers Victoria Brescoll and Tyler Okimoto in a 2010 study, “participants experienced feelings of moral outrage (i.e., contempt, anger, and/or disgust).”

Fascists, like Trump, pull a sleight of hand. By invoking the risk of corrupting purity, tradition and order, they distract their supporters from the real political corruption.


  1. This has been noted before by Thomas Edsall back in early 2016 writing for The New York Times.

Managing classical music collections in iTunes in 2018

Like many, I use iTunes to manage my music library. Unlike most, I don’t stream music[1]. That means I have a large music collection - one that stretches the limits of iTunes’ organizational system. The overwhelming majority of the music in our collection is classical; and this is the source of the difficulty.

The problem

For background on why classical music is so poorly handled, read Robinson Meyer’s piece in The Atlantic “The Tragedy of iTunes and Classical Music”. Several issues make is difficult to organize classical works:

  • iTunes regards all music as “songs”. Of course, some classical works are songs, but most are not.
  • The mp3 metadata standard as originally released was incomplete and completely ignorant of organizational requirements beyond “song” and “album.”
  • And of course, much is driven by Apple’s focus on profits. For every classical enthusiast, there must be 1000 popular music listeners. There’s simply too little return on any investment in tailoring the software to meet the needs of classical music listeners.

Naming and metadata standards

  1. Use Work/movement designation - This considerably improves the organization of multi-movement works in iTunes.
  2. Use opus and number designations - Both should be abbreviated, op. and no. respectively. That’s lower case abbreviation followed by a period and a space. If the work is in a sequence, then No. is part of the title and should be capitalized as in “Piano Trio No. 1 in C minor" The # symbol is never used. A slash / is never used.
  3. If there is a catalogue number, then it should be used. The catalogue number should be the last element in the name of the work, separated from the remainder by a comma and a single space. Köchel numbers and Deutsch should be abbreviated K. and D. respectively. However Bach-Werke-Verzeichnis catalogue numbers should not use periods.
  4. Add metadata in track comments - Since iTunes does not yet have all of the metadata that I need to organize my music, I add additional metadata in the Comments field. Four elements comprise the metadata that I currently use: a) Musical form (symphony, concerto, sonata, etc.), b) Musical period (e.g. baroque, classical, etc.), c) Ensemble - solo, duet, trio, orchestra, etc. d) Highlighted instruments - a comma-delimited list of instruments that are featured. While iTunes provides a genre designation, I simply use “Classical” and apply my own period classification in my free-text metadata.

Organizational approach

Since classical music isn’t really organized by “song” and an album is much less important than the works it contains, I break down the album structure and reorganize it by work. In addition to using the work/movement designation in the track info, I create playlists for individual works and playlists folders for broader categories of works by the same composer.

The top-level folder is the composer followed by nested folders of category of works (e.g. sonatas, concerti, etc.) then instrument folders under that level. If I have multiple recordings of the same work, the work itself gets its own folder.

What next for iTunes?

The work/movement designation in track info has been transformational; but iTunes still has a long way to go. Please don’t suggest music to me. Don’t show me a bunch of irrelevant album covers on the iTunes Store page; it’s meaningless to me. On mobile devices, find a way of showing the entire track name. Classical track names are often very long. Sometimes on the phone, I can only see “Piano Quartet No. 3 in…” Wait, is that the one in C minor or in G minor? No idea. iTunes still has a long way to go.


  1. Partly because we live in a rural setting with slow and unreliable internet access, partly because I don't trust that artists are getting properly compensated, and partly because I like to organize my music my own way.

This is an interesting essay in The Guardian on the idea of quarantining extremist ideas.

A non-trivial proportion of the population regards the media as having a responsibility to represent all idea with equal validity. So the appearance of extremist ideas in the press, even if they are treated negatively, results in more legitimacy than they are due. The authors in this essay make a case for quarantining these extreme ideas, refusing to cover them. Strategic silence, they call it.

If I were to implement this, anything Trump writes on Twitter would be quarantined because social media is not a platform for deliberate thought and government policy.