Pre-processing Russian text for the AwesomeTTS add-on in Anki

The Anki add-on AwesomeTTS has been a vital tool for language learners using the Anki application on the desktop. It allows you to have elements of the card read aloud using text-to-speech capabilities. The new developer of the add-on has added a number of voice options, including the Microsoft Azure voices. The neural voices for Russian are quite good. But they have one major issue, syllabic stress marks that are sometimes seen in text intended for language learners cause the Microsoft Azure voices to grossly mispronounce the word.

For example, the sentence Цензу́ра потака́ет извращённому уму́ больше, чем сло́во из трёх букв само́ по себе. is incorrectly pronounced because it has stress marks and because the letter ë is correctly displayed. Apparently Microsoft Azure doesn’t like it when ë is correctly rendered.

Fortunately, there’s a text pre-processor built into the plugin. Here is how to use this pre-processor to change the text that’s fed to the TTS provider:

In the add-on configuration view, navigate to Text → Advanced and add two rules:

  1. The first rule basically just strips the Unicode accent grave character (U+0301) from any vowels.
  2. The second rule transliterates ë to e . Yes, it offends me to do this because they are two different letters in the Russian alphabet, but it appears that the models were trained on text written in the way that’s often encountered, unfortunately.

With those two rules in place, the pronounciations going to Microsoft Azure should be correct.

Factor analysis of failed language cards in Anki

After developing a rudimentary approach to detecting resistant language learning cards in Anki, I began teasing out individual factors. Once I was able to adjust the number of lapses for the age of the card, I could examine the effect of different factors on the difficulty score that I described previously.

Findings

Some of the interesting findings from this analysis:

  • Prompt-answer direction - 62% of lapses were in the Russian → English (recognition) direction.1
  • Part of speech - Over half (51%) of lapses were among verbs. Since the Russian verbal system is rich and complex, it’s not surprising to find that verb cards often fail.
  • Noun gender - Between a fifth and a quarter (22%) of all lapses were among neuter nouns and among failures due to nouns only, neuter nouns represented 69% of all lapses. This, too, makes intuitive sense because neuter nouns often represent abstract concepts that are difficult to represent mentally. For example, the Russian words for community, representation, and indignation are all neuter nouns.

Interventions

With a better understanding of the factors that contribute to lapses, it is easier to anticipate failures before they accumulate. For example, I will immediately implement a plan to surround new neuter nouns with a larger variety of audio and sample sentence cards. For new verbs, I’ll do the same, ensuring that I include multiple forms of the verb, varying the examples by tense, number, person, aspect and so on.

Future directions

I’d like to extend this approach to a more statistically-rigorous prediction scheme, so that I can more accurately target efforts to prevent the accumulation of lapses.

References


  1. Note that the fractions in the fx all lapses column for the card direction group do not add up to 1.0 because I excluded a minute number of image cards from the analysis. ↩︎

Refactoring Anki language cards

Regardless of how closely you adhere to the 20 rules for formating knowledge, there are cards that seem destined to leechdom. For me part of the problem is that with languages, straight-up vocabulary cards take words out of the rich context in which they exist in the wild. With my maturing collection of Russian decks, I recently started to go through these resistant cards and figure out why they are so difficult.

Parsing Russian Wiktionary content using XPath

As readers of this blog know, I’m an avid user of Anki to learn Russian. I have a number of sources for reference content that go onto my Anki cards. Notably, I use Wiktionary to get word definitions and the word with the proper syllabic stress marked. (This is an aid to pronunciation for Russian language learners.) Since I’m lazy to the core, I came up with a system way of grabbing the stress-marked word from the Wiktionary page using lxml and XPath.

Being grateful for those who push our buttons

We need people to push our buttons, otherwise how are we to know what buttons we have? Jetsunma Tenzin Palmo Ten Percent Happier podcast, February 8, 2021 Jetsunma Tenzin Palmo is a Buddhist nun interviewed on the excellent Ten Percent Happier podcast. It’s always possible to reframe situations where someone “pushes our buttons” to see it as an opportunity to better understand that there are these buttons, these sensitivities that otherwise evade our awareness.

Directly setting an Anki card's interval in the sqlite3 database

It’s always best to let Anki set intervals according to its view of your performance on testing. That said, there are times when directly altering the interval makes sense. For example, to build out a complete representation of the entire Russian National Corpus, I’m forced to enter vocabulary terms that should be obvious to even elementary Russian learners but which aren’t yet in my nearly 24,000 card database. Therefore, I’m entering these cards gradually.

Where the power lies in 2021

From an article recently on the BBC Russian Service: Блокировка уходящего президента США в “Твиттере” и “Фейсбуке” привела к необычной ситуации: теоретически Трамп еще может начать ядерную войну, но не может написать твит. “Blocking the outgoing U.S. President from Twitter and Facebook has led to an unusual situation: theoretically Trump can still start a nuclear war, but cannot write a Tweet." In only a week, he won’t be able to do either.

More on integrating Hazel and DEVONthink

Since DEVONthink is my primary knowledge-management and repository tool on the macOS desktop, I constantly work with mechanisms for efficiently getting data into and out of it. I previously wrote about using Hazel and DEVONthink together. This post extends those ideas about and looks into options for preprocessing documents in Hazel before importing into DEVONthink as a way of sidestepping some of the limitations of Smart Rules in the latter. I’m going to work from a particular use-case to illustrate some of the options.

Undoing the Anki new card custom study limit

Recently I hit an extra digit when setting up a custom new card session and was stuck with hundreds of new cards to review. Desparate to fix this, I started poking around the Anki collection SQLite database, I found the collection data responsible for the extra cards. In the col table, find the newToday key and you’ll find the extra card count expressed as a negative integer. Just change that to zero and you’ll be good.

Copy Zettel as link in DEVONthink

Following up on my recent article on cleaning up Zettelkasten WikiLinks in DEVONthink, here’s another script to solve the problem of linking notes. Backing up to the problem. In the Zettelkasten (or archive) - Zettel (or notes) are stored as list of Markdown files. But what happens when I want to add a link to another note into one that I’m writing? Since DEVONthink recognizes WikiLinks, I can just start typing but then I have to remember the exact date so that I can pick the item out of the contextual list that DEVONthink offers as links.