Splitting text into sentences: Russian edition

Splitting text into sentences is one of those tasks that looks simple but on closer inspection is more difficult than you think. A common approach is to use regular expressions to divide up the text on punction marks. But without adding layers of complexity, that method fails on a sentence such as:

“Trapper John, M.D. was as fine as any Ph.D.”

It’s obviously only one sentence, but try it with regex and the difficulty is obvious.

A solution suggested on Stack Overflow is to use the spaCy natural language processing module along with its ‘sentencizer’ pipeline to do the heavy lifting. The recommended solutions are all based on English language processing; so I was anxious to see if it would work on Russian text. The short answer is “yes.” This post is just to document the solution.

from spacy.lang.ru import Russian

nlp_simple = Russian()
nlp_simple.add_pipe('sentencizer')

doc = nlp_simple(text)
sentences = [str(sent).strip() for sent in doc.sents]

What’s up with Pinboard? And an alternative

Beginning somewhere around April 2022, the bookmarking web application Pinboard began to suffer prolonged outages without really any substantive commentary from the developer. Reports on Hacker News reveal a pattern of frequently-broken functionality. As of this writing, the API is no longer functioning.

One of the great things about the HN community is that you can almost always find an open-source tool to get the job done. That’s how I discovered Espial. It’s a minimalist open-source self-hosted bookmarking tool that looks and works like Pinboard. It also imports the Pinboard export JSON format.

Espial installed readily for me on macOS and seems very usable. My advice is to export your Pinboard bookmarks while you can and spin-up an instance of Espial.