Trump meets computational linguistics

Trump orating

“I actually called her, and she never mentioned my name. You know, I - when I sold - oh, did I get a call from one of the Environmental Protection Agency, they couldn’t find it because it comes out in big globs, right, and you say to yourself, ‘How does that help us?’”

Trump is one of the most amusing orators in the history of presidential politics in the the U.S. But I wondered what would happen I took the text of a few of his speeches and fed it into a algorithm that uses Markov chains to shake things up a bit.

A Markov Chain is a process in which states undergo random transitions in which the probability of the next state depends only on the current state. This is a characteristic often called “memorylessness.” In statistical modeling, Markov Chains have serious applications. But I’ve used it here to analyze and distort Trump speeches so that they sound even more rambling than in reality. This results in such keepers as:

"Well, they couldn’t find it because then it comes out in big globs, right, and you had Iraq and you know, China comes over and they say, we can’t believe it or not. I’m doing it. Number two, I’m a free trader.

The algorithm takes a “snapshot” of the preceding two words in a sentence, scans the text for those two words, generating a table of words that follow those couplets. It then chooses a random word based on the probabilities.

“We have wounded warriors, incredible people. And you know, we are doing a big problem, Russia’s a problem, China’s a problem. We’ve got a lot of things. We are rebuilding China. We’re rebuilding many countries. The Euro, China is going to be repealed and replaced. It will die in ’17 anyway.”

Some of the sentences are grammatically correct; many are not. But they have similar cadence and “feel” as a real Trump speech.

““Mr. Trump, where are those hats made?” I said. I had no choice, because I don’t want me to refute his statement. How dare you say that. They don’t know what they’re saying — that nobody believes. So you have a presidential election coming up. We have $2 trillion and we are going to move the plant back to the ceiling, $500 billion is?”

I used a Python implementation of the Markov text generator described above. You can read the full text of a run. The source text comes from three different speeches given by Mr. Trump.

Stop Facebook tracking

network web

Although I understand Facebook’s business model and I (basically) understand how money is made on the internet, I have no compunction about blocking ads, trackers, beacons and all manner of scripts. The current system creates layers upon layers of networks that exist to track one’s activities on the internet and market products and services more specifically. The problem is that unless I take specific action, I don’t get to choose what I reveal to companies that want to track me. Some have argued that using services like Facebook constitutes an implied contract between you - the user, and the internet application provider. It’s a ridiculous argument because I could make a similar argument that my use of their service implies a contract not to track.

Here’s how I block tracking. Some is specific to Mac OS X and Safari and some is general.

Register your advertising preference

The Digital Advertising Alliance allows you register your advertising preference with participating companies and ad networks. After turning off any ad blockers and after setting your browser preferences to accept third-party cookies, visit their page and opt our of all participating companies. When I ran it on my computer, it blocked 123 companies from behavioural tracking.

Use Ghostery

Ghostery blocks trackers

The browser extension Ghostery works with a number of different browers to block scripts that provide information about your browsing to third parties (and fourth, fifth, and sixth…) For example, I visited the, a technology-oriented site ahd Ghostery reported at 11 trackers, beacons, and widgets. Mind you, perhaps the previous step of registering with the Digital Advertising Alliance is sufficient to block many of these scripts. But since participation by the companies is voluntary, I would prefer to block them on my end also. Ghostery is very easy to use and has the ability to whitelist a site that you trust and to pause blocking as some sites have scripts that are essential to their functionality.

Browser privacy settings

Check your privacy settings in your browser. For Safari, it’s Safari > Preferences…. Make sure that you set the Cookies and website data to “Allow from current website only.” And check the feature to block website tracking.

For Chrome, it’s Chrome > Preferences. Search for “cookies” in the settings search field. I set my cookies preferences to “Keep local data only until you quit your browser” and “Block third-party cookies and site data.”

Close Facebook when you’re done

When you’re done with your session on Facebook, logout (FB menu far right) and close its window or tab.

Clear Facebook cookies

After I’ve finished using Facebook, I clear any cookies that it has set on my computer. You can do this manually:

  • Safari: Safari > Preferences… > Cookies and website data > Details… Search for and delete its cookies.

  • Chrome: Chrome > Preferences…. Search the settings for cookies and clear browsing data. I cannot find a function that allows you to delete cookies in a more granular way.

On Mac, I use a script that I borrowed and modified from Dr. Drang that automates the deletion of specific cookies in Safari. I set it up to run using a key combination that I press whenever I close down Facebook. Here’s the script, if you’re interested:

set deCookie to {""}

tell application "Safari" to activate

tell application "System Events"
tell process "Safari"
keystroke "," using command down
delay 1
tell window 1
click button "Privacy" of toolbar 1
delay 3
repeat with d in deCookie
click button "Details…" of group 1 of group 1
keystroke d
delay 1
select row 1 of table 1 of scroll area 1 of sheet 1
click button "Remove" of sheet 1
end try
click button "Done" of sheet 1
end repeat
end tell
keystroke "w" using command down
end tell
end tell

Use an ad blocker

Download Ad Block Plus to block advertisements on web pages. It’s spectacular how many ads that it blocks. I don’t know that it stops tracking completely; but the aesthetic result of blocking ads is spectacular. Again, I don’t regard this as an ethical issue in the least. Services that I use regularly that wish to charge me money are free to do so. If I see value in the service I’ll use it. Monetizing the use of a site by tracking one’s users and selling the information to other parties is hardly a way to build trust.

See also

  1. Interestingly, Business Week itself runs 15 trackers, beacons and other suspicious scripts on its site...

  2. Again, another site that condemns tracking but itself runs 14 trackers...

Leaving something and taking something away.


“Every person passing through this life will unknowingly leave something and take something away. Most of this “something” cannot be seen or heard or numbered or scientifically detected or counted. It’s what we leave in the minds of other people and what they leave in ours. Memory. The census doesn’t count it. Nothing counts without it.” - Robert Fulghum All I Really Need to Know I Learned in Kindergarten

Maybe buildings, golf resorts, “reality TV” shows, steaks, and airplanes count. I don’t know.

The anti-polarizing effect of keeping one's identity small.

Keeping your identity small

Several years ago, Paul Graham, of Y Combinator fame wrote an essay entitled “Keep your Identity Small.” The premise is that discussions of religion and politics almost never result in anything resembling the give-and-take around other subjects. Two people can have an intelligent conversation about the pros and cons of certain brands of rice cookers[1]; but if the discussion turns to religion or politics, it’s essentially over. Graham’s conclusion is that religion and politics both engage a person’s identity. Once a conversation turns to identity, it’s hard to separate issues from people[2]

Perhaps the widely lamented political polarization in the U.S. is a product of over-identification. Sentences that begin with “I am an x.” shut down the fuller understanding of x, y, z, and others that you haven’t even heard of yet.

Take time to read Graham’s essay.

  1. We happen to use a Zojirushi and like it a lot; but I'd never introduce myself as a Zojirushian.

  2. The best-selling book "Getting to Yes" by Fisher and Ury on negotiation techniques touched on this issue with it's first principle of "Separate the people from the problem." Emotions are a source of real problems in negotiation because people respond with anger when their personal interests are at stake. By treating issues as entities separate from the people that involve them, holiding them, inspecting and debating them on their own terms, it becomes easier to have conversations about them.

Properly understanding ISIS

Cultural contradictions of ISIS

An interesting piece from The Atlantic on understanding ISIS on their own clearly-stated terms.

“We have misunderstood the nature of the Islamic State in at least two ways. First, we tend to see jihadism as monolithic, and to apply the logic of al‑Qaeda to an organization that has decisively eclipsed it…Bin Laden viewed his terrorism as a prologue to a caliphate he did not expect to see in his lifetime. The Islamic State, by contrast, requires territory to remain legitimate, and a top-down structure to rule it.”

Bin Laden also had no qualms about promoting his ideological interests while easily living in an increasingly secular world. ISIS

“In fact, much of what the group does looks nonsensical except in light of a sincere, carefully considered commitment to returning civilization to a seventh-century legal environment, and ultimately bringing about the apocalypse…Muslims can reject the Islamic State; nearly all do. But pretending that it isn’t actually a religious, millenarian group, with theology that must be understood to be combatted, has already led the United States to underestimate it and back foolish schemes to counter it. We’ll need to get acquainted with the Islamic State’s intellectual genealogy if we are to react in a way that will not strengthen it, but instead help it self-immolate in its own excessive zeal.”

Perhaps in their ideological zeal, a form of which eclipses every other form of Islamic fundamentalism, their own historical and cultural contradictions have escaped their attention. They reject modernity and wish to restore a seventh-century world order; but apparently see no incongruity in employing modern weapons in hopes achieving their ends. When Muhammad lived[1], weapons largely consisted of sharp things, stones, and shields. Bringing down airliners with IEDs packed with PETN or RDX and triggered by electronic timers is an anachronism.

That’s the problem with anti-modernity campaigns, it’s hard to do without involving yourself in the modern.

  1. The historicity of Muhammad as an actual living figure is still debated. One of the problems with gathering evidence on this subject is that pre-Islamic Arabia is a "black box" with little historical record. It's worthwhile reading about the subject; but it's far from conclusive.

A new hexo deployer for Amazon web services

I recently migrated this and my other blog to Hexo which is a very fast static blogging framework built on node.js. As when I used Octopress, this blog is still hosted from an AWS S3 bucket. However the deployers that I tried with Hexo failed because of dependencies that were incompatible with the OS X version I was running. Not being a node.js expert, and having no time to delve into node.js internals, I wrote a new deployer:


It relies on Transmit to upload the public directory of a Hexo baked blog to the S3 bucket that hosts the site. You can find it on npm and Github.

Scope of attention and consumption

I have a hypothesis. The wider the scope of your attention to social media and the popular press the more material goods you consume.

Having had too many hobbies and pursuits in my own life, I’ve noticed that the more diverse my attention, the more I ended up consuming. Is it that one’s physical environment (how much stuff you buy and surround yourself with) reflects your mental environment? Or is it the other way around? Maybe it’s really both. The less focused you are on meaningful, low consumption pursuits, the more you buy; and the more you buy, the more distracted you are by all the stuff around you. A vicious circle.

Fortunately, there’s a better version - the virtuous circle. The less you own, the less distracted you are and the more likely you are to focus your time and energy on what’s important.

Collapsing DEVONthink groups via AppleScript

I’ve been moving to a tag-based system for organizing content in DEVONthink. All of my content for each database goes into a single group called “reference.” If I want to find something, I search the hierarchical tag structure instead of diving into some arbitrary list of groups.

But I still have groups that I’d like to collapse into the reference group. So I wrote an AppleScript to perform this action. Notably, most of the action is in the processGroup() handler which is recursive because we do not know how deep the group hierarchy goes.

Here’s the script if it’s something you can use:

-- collapse all groups into a single reference group
-- Created by Alan Duncan on 2015-11-23 03-15-56
-- Copyright (c) 2015 Alan K. Duncan.
-- Distributed under the terms of the MIT license

-- some error codes we might encounter
property InvalidRecordIndexError : -1719

-- warning string template
property ConfirmText : "Collapse all groups in database: "

-- name of groups that we don't want to move contents of
property ExcludedRecordNames : {"Inbox", "Tags", "Mobile Sync", "Trash", "reference"}

-- the reference group
global refGroup

-- list of groups to delete
global deleteGroups

tell application "DEVONthink Pro"
set questionText to ConfirmText & name of current database & "?"
set confirm to display dialog questionText buttons {"Yes", "No"} default button 2
set answer to button returned of confirm
if answer is equal to "Yes" then
set deleteGroups to {}
set refGroup to referenceGroup(current database) of me
tell current database
repeat with aRecord in (every record whose type is group)
if name of aRecord is not in ExcludedRecordNames then
-- this is a legitimate group to process
processGroup(aRecord) of me
-- this group has been processed; it can be deleted
set end of deleteGroups to aRecord
end if
end repeat

cleanupGroups() of me
end tell
end if
end tell

-- remove all groups that are marked for deletion
on cleanupGroups()
tell application "DEVONthink Pro"
repeat with deleteGroup in deleteGroups
delete record deleteGroup
end repeat
end tell
end cleanupGroups

-- recursively process groups
on processGroup(aGroup)
tell application "DEVONthink Pro"
set theChildren to children of aGroup
repeat with aRecord in theChildren
-- if this child is a group, then enter recursively
if type of aRecord is group then
processGroup(aRecord) of me
move record aRecord to refGroup
end if
end repeat
end tell
end processGroup

-- return the reference group, creating it if it doesn't exist
on referenceGroup(db)
using terms from application "DEVONthink Pro"
tell db
set referenceGroup to the first record whose name is "reference"
on error error_message number error_number
if error_number is InvalidRecordIndexError then
-- try to create a group "reference"
set refGroup to create record with {name:"reference", type:group} in root
end if
end try
end tell
end using terms from
end referenceGroup

A minimalist mind

To deal with the explosion of information available to us, we’re told to avoid the filter bubble by seeking out a variety of sources. Or we’re told to pursue a low information diet. But we’re also told that to be informed is one of the duties of citizenship. What are we to do? Here are some other options:

  1. Stop caring about what doesn’t affect you. There’s apparently a Syrian refugée crisis in Europe. It’s unfortunate; but I won’t read about it. What good does it do? Nothing. So why bother reading about it? My sphere of interest should coincide with my sphere of influence. I feel bad about their situation; but all I can do it live my own life as simply as I can.

  2. Don’t click link bait. Ever. Do the world a favor and stop clicking link bait. Don’t click any title that begins with a number, e.g. “6 easy ways to make the most of working from home.” Don’t click any link with women in bikinis. These are rabbit holes from which you eventually emerge with self-loathing. And by clicking on link bait, you’re voting for crap on the web. Even some well-known newspapers like the Washington Post are full of link bait. Learn to recognize link bait and stop clicking it.

  3. Know what you want. Before you open a browser window, you exactly what you’re looking for. Find it, save it. Then close the window. Failure to know what you’re looking for on the web just makes you susceptible to what everyone else thinks you should look at. And 99.9% of it is crap.

  4. Stop social networking. Facebook, Twitter and whatever comes next are enormous wastes of mental energy. If after diving into Facebook only to emerge an hour later you feel the guilt and self-loathing of a heroin addict, you should. Facebook and its worthless spawn are the crack cocaine of the web.

  5. Use a crutch. “You can’t change what you don’t measure.” So measure your computer use. Get RescueTime and install it on your computer(s). It will give you reports on how much time you spend doing various tasks on the computer. Unfortunately, it doesn’t do much on the iPhone because Apple has iOS locked down. But on your laptop, desktop and Android devices you can get excellent data on what your spending your time doing at the computer.

  6. Put down your phone. Few things send me into more rage than seeing people walking around mesmerized by their little glowing rectangles. Seriously, what the hell is wrong with people? It’s not that interesting. Get a life.

  7. Train yourself to ignore advertisements. Block the hell out of advertisements. I’m completely unmoved by specious arguments about how unethical it is to block ads and how it will destroy the web as we know it. It’s not my fault that the web is monetized in screwy ways. I didn’t ask for valuable services to be free. Stop making yourself a target for advertiser’s Jedi mind tricks. Install an AdBlocker and aggressively block their psychomanipulations.

It’s much easier to have a quiet mind by making your forays into the web as transactional as possible.

Working with DEVONthink Pro Office and Hazel

My main organizational tool DEVONthink Pro Office, a tool I’ve used for many years. I’ve written previously about it and how I use it to find things and how I synchronize databases across machines.

I’m a relative newcomer to Hazel though. Hazel’s tagline is “automated organization for your Mac.” Hazel works as an agent to keep folders organized on the Mac. It’s an engine that applies per-folder rules to take actions on files and folders. Actions can include tagging files, moving them to other folders, running AppleScripts, deleting them, etc.

Since DEVONthink is the centerpiece of my organizational tools on the Mac, I wondered if Hazel and DEVONthink might be able to work together in a productive way. It’s an experiment that turned out well. I’ll describe two cases where I’m using them together. Read more after the break.

Read More