Creating Obsidian tables of content
When viewing longer Markdown notes in Obsidian, tables of content (TOC) help a lot with navigation. There is a handful of community plugins to help with TOC generation, but I have two issues with them:
- It creates a dependency on code whose developer may lose interest and eventually abandon the project. At least one dynamic TOC plugin has suffered this fate.
- All of the TOC plugins have the same visual result. When you navigate to a note, Obsidian places the focus at the top of the note, beneath the frontmatter. That’s fine unless the content starts with a TOC markup block, in which case it’s not the TOC itself that is displayed, but the markup for the TOC plugin itself as depicted in the image below.
For me the solution was to write a script that scans the vault looking for this pair of markers:
<!--ts-->
<!--te-->that define the start and end for the TOC block. When these markers are found, the script inserts a TOC between the markers. The downside of course is that the markers, which are just raw HTML comments in the Markdown document are still visible when you’re in Edit mode. But it’s still better than not seeing the actual TOC at all when you navigate to a note.
Technical Details
Finding Files that Need TOC Updates
The script begins by using the find command to locate recently modified Markdown files in your Obsidian vault. It specifically looks for files modified within the last minute, making it efficient for running as a background process. This approach ensures that TOCs are updated only in notes you’ve recently worked on, rather than scanning your entire vault every time.
find_cmd="find \"$VAULT_DIR\" \
-path \"*/$EXCLUDE_DIRS\" -prune \
-o -type f \
-name \"*.md\" \
-newermt \"1 minute ago\" \
-print"
For each file, the script:
- Checks if valid TOC markers exist outside of code blocks
- Counts the headings in the document
- Compares the current TOC entries against the document’s headings
- Updates the TOC if:
- The TOC is empty but headings exist
- The heading content has changed
- The file was modified since the last run
This strategy minimizes unnecessary updates and ensures TOCs stay in sync with your content as you write and edit.
Handling Code Blocks and Marker Detection
One of the challenges with this approach is distinguishing between actual TOC markers and examples of markers included in code blocks. For instance, if you’re writing documentation about the TOC script itself, you might include the TOC markers in a code example.
The script solves this by carefully tracking when it’s inside or outside of code blocks:
# Check for code block start/end
if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
# Extract the fence type (``` or ~~~)
fence_match="${BASH_REMATCH[1]}"
if [[ "$in_code_block" == "false" ]]; then
in_code_block=true
code_fence="$fence_match"
elif [[ "$line" == *"$code_fence"* ]]; then
in_code_block=false
code_fence=""
fi
}
When analyzing a file, the script maintains state variables to track:
- Whether we’re currently inside a code block
- The type of code fence used (``` or ~~~)
- Whether we’re in YAML frontmatter
This context-aware parsing ensures that TOC markers inside code blocks are completely ignored. This is crucial because it prevents the script from incorrectly modifying code examples or documentation that happens to contain the markers.
Usage
Basic Usage
The script can be run directly from the command line:
./obsidian_toc_monitor.sh /path/to/obsidian/vaultYou can also specify which directories to exclude:
./obsidian_toc_monitor.sh --exclude ".obsidian,Templates,Attachments" /path/to/vaultAutomating TOC Updates
There are several ways to automate the script depending on your operating system:
On macOS:
- Using launchd - Create a launch agent to run the script periodically:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.user.obsidian.tocmonitor</string>
<key>ProgramArguments</key>
<array>
<string>/path/to/obsidian_toc_monitor.sh</string>
<string>/path/to/obsidian/vault</string>
</array>
<key>StartInterval</key>
<integer>60</integer>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>Save this to ~/Library/LaunchAgents/com.user.obsidian.tocmonitor.plist and load it with:
launchctl load ~/Library/LaunchAgents/com.user.obsidian.tocmonitor.plist- Using cron - Add an entry to your crontab:
* * * * * /path/to/obsidian_toc_monitor.sh /path/to/obsidian/vaultOn Linux:
- Using systemd - Create a user service:
[Unit]
Description=Obsidian TOC Monitor
[Service]
Type=simple
ExecStart=/path/to/obsidian_toc_monitor.sh /path/to/obsidian/vault
Restart=always
RestartSec=60
[Install]
WantedBy=default.targetSave this to ~/.config/systemd/user/obsidian-toc.service and enable it:
systemctl --user enable --now obsidian-toc.service- Using cron (same as macOS)
Adding TOC Markers to Your Notes
To prepare a note for TOC generation, simply add these markers where you want the TOC to appear:
# My Note Title
<!--ts-->
<!--te-->
## First Section
Content here...
The script will detect these markers and insert a formatted TOC.
You get the benefits of a dynamic TOC without visual issues.Script code
#!/bin/bash
# Default configuration
LOG_FILE="$HOME/Library/Logs/obsidian_toc_monitor.log"
LAST_RUN_FILE="/tmp/obsidian_toc_monitor_lastrun"
EXCLUDE_DIRS=".obsidian pdf Templates Attachments Meta"
DEBUG="${DEBUG:-false}"
VERBOSE="${VERBOSE:-true}" # Set to true to show terminal output
# Global counters
FILE_COUNT=0
MODIFIED_COUNT=0
# Color codes for better terminal output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
RESET='\033[0m'
# Usage information
show_usage() {
echo "Usage: $(basename "$0") [OPTIONS] VAULT_DIRECTORY"
echo ""
echo "Options:"
echo " -h, --help Show this help message and exit"
echo " -q, --quiet Run in quiet mode (no terminal output)"
echo " -d, --debug Enable debug logging"
echo " -e, --exclude DIRS Comma-separated list of directories to exclude (default: $EXCLUDE_DIRS)"
echo " -l, --log FILE Path to log file (default: $LOG_FILE)"
echo ""
echo "Example:"
echo " $(basename "$0") /path/to/obsidian/vault"
echo " $(basename "$0") --exclude '.obsidian,pdf,Templates' /path/to/vault"
}
# Parse command line arguments
POSITIONAL_ARGS=()
while [[ $# -gt 0 ]]; do
case $1 in
-h|--help)
show_usage
exit 0
;;
-q|--quiet)
VERBOSE=false
shift
;;
-d|--debug)
DEBUG=true
shift
;;
-e|--exclude)
EXCLUDE_DIRS="$2"
shift 2
;;
-l|--log)
LOG_FILE="$2"
shift 2
;;
-*|--*)
echo "Error: Unknown option $1"
show_usage
exit 1
;;
*)
POSITIONAL_ARGS+=("$1")
shift
;;
esac
done
set -- "${POSITIONAL_ARGS[@]}"
# Check if vault directory was provided
if [ $# -lt 1 ]; then
echo "Error: No vault directory specified"
show_usage
exit 1
fi
# Set the vault directory
VAULT_DIR="$1"
# Check if vault directory exists
if [ ! -d "$VAULT_DIR" ]; then
echo "Error: Vault directory does not exist: $VAULT_DIR"
exit 1
fi
# Ensure log directory exists
mkdir -p "$(dirname "$LOG_FILE")"
# Touch the last run file if it doesn't exist
[ ! -f "$LAST_RUN_FILE" ] && touch "$LAST_RUN_FILE"
# Logging functions
log_info() {
echo "[INFO] $1" >> "$LOG_FILE"
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${GREEN}[INFO]${RESET} $1"
fi
}
log_error() {
echo "[ERROR] $1" >> "$LOG_FILE"
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${RED}[ERROR]${RESET} $1"
fi
}
log_debug() {
if [[ "$DEBUG" == "true" ]]; then
echo "[DEBUG] $1" >> "$LOG_FILE"
echo -e "${BLUE}[DEBUG]${RESET} $1"
fi
}
generate_toc() {
local FILE_PATH="$1"
local TEMP_FILE="$2"
# First add the TOC heading
echo "## Table of contents" > "$TEMP_FILE"
echo "" >> "$TEMP_FILE" # Add blank line after the heading
# Find headings and generate TOC
local min_level=6
declare -a heading_lines
local in_code_block=false
local code_fence=""
local in_frontmatter=false
while IFS= read -r line; do
# Handle YAML frontmatter
if [[ "$line" == "---" ]]; then
if [[ "$in_frontmatter" == "false" ]]; then
in_frontmatter=true
else
in_frontmatter=false
fi
continue
fi
# Skip processing if in frontmatter
if [[ "$in_frontmatter" == "true" ]]; then
continue
fi
# Skip processing the TOC heading itself
if [[ "$line" == "## Table of contents" ]]; then
continue
fi
# Check for code block start/end
if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
# Extract the fence type (``` or ~~~)
fence_match="${BASH_REMATCH[1]}"
if [[ "$in_code_block" == "false" ]]; then
in_code_block=true
code_fence="$fence_match"
elif [[ "$line" == *"$code_fence"* ]]; then
in_code_block=false
code_fence=""
fi
continue
fi
# Only process headings if we're not inside a code block
if [[ "$in_code_block" == "false" ]]; then
if [[ "$line" =~ ^[[:space:]]*(#{1,6})[[:space:]]+(.*[^[:space:]]) ]]; then
# The heading marker is captured in BASH_REMATCH[1]
local marker="${BASH_REMATCH[1]}"
local level=${#marker}
# The heading text is captured in BASH_REMATCH[2]
local title="${BASH_REMATCH[2]}"
# Update min level if needed
if [[ $level -lt $min_level ]]; then
min_level=$level
fi
# Store heading info for later processing
heading_lines+=("$level|$title")
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${CYAN}→ Level $level:${RESET} $title"
fi
fi
fi
done < "$FILE_PATH"
# Generate TOC entries
for entry in "${heading_lines[@]}"; do
IFS='|' read -r level title <<< "$entry"
# Clean markdown formatting
clean_title="$title"
clean_title="${clean_title//\*\*/}" # Remove bold
clean_title="${clean_title//\*/}" # Remove italic
clean_title="${clean_title//\`/}" # Remove code
# Calculate relative indent level
rel_level=$((level - min_level))
if [[ $rel_level -eq 0 ]]; then
echo "* [[#$clean_title]]" >> "$TEMP_FILE"
else
indent=$(printf "%$((rel_level * 4))s" "")
echo "${indent}- [[#$clean_title]]" >> "$TEMP_FILE"
fi
done
# Return the number of headings found
echo ${#heading_lines[@]}
}
update_toc() {
local FILE_PATH="$1"
local START_LINE="$2"
local END_LINE="$3"
local was_updated=false
log_debug "Updating TOC for $FILE_PATH (markers at lines $START_LINE and $END_LINE)"
# Create temporary files
local TEMP_TOC=$(mktemp)
local TEMP_FILE=$(mktemp)
# Generate the TOC
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${YELLOW}Parsing headings...${RESET}"
fi
local heading_count=$(generate_toc "$FILE_PATH" "$TEMP_TOC")
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${YELLOW}Generating TOC with $heading_count headings...${RESET}"
fi
# Extract current TOC content for comparison
local current_toc=$(sed -n "${START_LINE},${END_LINE}p" "$FILE_PATH")
# Create updated file with new TOC
{
# Copy lines before TOC
sed -n "1,$((START_LINE-1))p" "$FILE_PATH"
# Copy start marker line
sed -n "${START_LINE}p" "$FILE_PATH"
# Insert new TOC content
cat "$TEMP_TOC"
# Copy end marker line
sed -n "${END_LINE}p" "$FILE_PATH"
# Copy lines after TOC
sed -n "$((END_LINE+1)),\$p" "$FILE_PATH"
} > "$TEMP_FILE"
# Extract new TOC content for comparison
local new_toc=$(sed -n "${START_LINE},$((START_LINE+1+$(wc -l < "$TEMP_TOC")+1))p" "$TEMP_FILE")
# Compare TOCs to see if an update is needed
if ! diff -q <(echo "$current_toc") <(echo "$new_toc") >/dev/null; then
# TOC content has changed - update the file
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${GREEN}✓ Updating TOC with $heading_count entries${RESET}"
fi
# Move updated file into place
mv "$TEMP_FILE" "$FILE_PATH"
# Add a small delay and touch the file to ensure changes are detected
sleep 0.1
touch "$FILE_PATH"
log_info "TOC updated for: $FILE_PATH"
was_updated=true
# Increment global counter
((MODIFIED_COUNT++))
else
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${GREEN}✓ TOC already up to date${RESET}"
fi
log_debug "No TOC changes needed for: $FILE_PATH"
fi
# Clean up temporary files
rm -f "$TEMP_TOC" "$TEMP_FILE" 2>/dev/null
return $([ "$was_updated" = true ] && echo 0 || echo 1)
}
find_valid_toc_markers() {
local FILE_PATH="$1"
local START_LINE_VAR="$2"
local END_LINE_VAR="$3"
# Use grep to find potential marker line numbers
local potential_start_markers=$(grep -n "<!-- *ts *-->" "$FILE_PATH" | cut -d':' -f1)
local potential_end_markers=$(grep -n "<!-- *te *-->" "$FILE_PATH" | cut -d':' -f1)
# If no potential markers found at all, return false
if [[ -z "$potential_start_markers" || -z "$potential_end_markers" ]]; then
return 1
fi
# Parse the file line by line to verify markers are outside code blocks
local line_num=0
local in_code_block=false
local code_fence=""
local in_frontmatter=false
local valid_start_line=""
local valid_end_line=""
while IFS= read -r line; do
((line_num++))
# Handle YAML frontmatter
if [[ "$line" == "---" ]]; then
if [[ "$in_frontmatter" == "false" ]]; then
in_frontmatter=true
else
in_frontmatter=false
fi
continue
fi
# Skip processing if in frontmatter
if [[ "$in_frontmatter" == "true" ]]; then
continue
fi
# Check for code block start/end
if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
# Extract the fence type (``` or ~~~)
fence_match="${BASH_REMATCH[1]}"
if [[ "$in_code_block" == "false" ]]; then
in_code_block=true
code_fence="$fence_match"
log_debug "Line $line_num: Code block starts"
elif [[ "$line" == *"$code_fence"* ]]; then
in_code_block=false
code_fence=""
log_debug "Line $line_num: Code block ends"
fi
continue
fi
# Only check for TOC markers if we're not inside a code block
if [[ "$in_code_block" == "false" ]]; then
# Check if current line is in our potential marker list
if echo "$potential_start_markers" | grep -q "^$line_num$"; then
valid_start_line=$line_num
log_debug "Line $line_num: Valid start marker found"
fi
if echo "$potential_end_markers" | grep -q "^$line_num$"; then
valid_end_line=$line_num
log_debug "Line $line_num: Valid end marker found"
fi
fi
done < "$FILE_PATH"
# Only succeed if we found valid markers outside code blocks
if [[ -z "$valid_start_line" || -z "$valid_end_line" ]]; then
return 1
fi
# Set the output variables
eval "$START_LINE_VAR=$valid_start_line"
eval "$END_LINE_VAR=$valid_end_line"
return 0
}
process_file() {
local FILE_PATH="$1"
local update_needed=false
# Skip if file doesn't exist or isn't a regular file
if [[ ! -f "$FILE_PATH" ]]; then
return 1
fi
# Skip non-markdown files
if [[ ! "$FILE_PATH" =~ \.(md|markdown)$ ]]; then
return 1
fi
# Increment global counter
((FILE_COUNT++))
log_debug "Checking file: $FILE_PATH"
if [[ "$VERBOSE" == "true" ]]; then
filename=$(basename "$FILE_PATH")
echo -e "${YELLOW}Checking:${RESET} $filename"
fi
# Find valid TOC markers (outside code blocks)
local start_line=""
local end_line=""
if ! find_valid_toc_markers "$FILE_PATH" start_line end_line; then
log_debug "No valid TOC markers found in file - $FILE_PATH"
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${YELLOW}→ No valid TOC markers found${RESET}"
fi
return 1
fi
log_debug "Found valid TOC markers at lines $start_line and $end_line"
# Count the number of headings in the file, excluding those in code blocks
heading_count=0
in_code=false
code_fence=""
in_front=false
declare -a heading_lines
while IFS= read -r line; do
# Handle YAML frontmatter
if [[ "$line" == "---" ]]; then
if [[ "$in_front" == "false" ]]; then
in_front=true
else
in_front=false
fi
continue
fi
if [[ "$in_front" == "true" ]]; then
continue
fi
# Check for code block start/end
if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
fence="${BASH_REMATCH[1]}"
if [[ "$in_code" == "false" ]]; then
in_code=true
code_fence="$fence"
elif [[ "$line" == *"$code_fence"* ]]; then
in_code=false
code_fence=""
fi
continue
fi
# Only count headings outside code blocks
if [[ "$in_code" == "false" && "$line" =~ ^[[:space:]]*(#{1,6})[[:space:]]+(.*[^[:space:]]) ]]; then
# Skip TOC heading
if [[ "$line" =~ "Table of contents" ]]; then
continue
fi
# Extract heading level and text
level=${#BASH_REMATCH[1]}
title="${BASH_REMATCH[2]}"
# Store heading info for later comparison
heading_lines+=("$title")
((heading_count++))
fi
done < "$FILE_PATH"
log_debug "Found $heading_count headings in file (excluding code blocks)"
# Extract TOC content (between markers)
local toc_content=$(sed -n "$((start_line+1)),$((end_line-1))p" "$FILE_PATH")
# Extract TOC entry texts
declare -a toc_texts
while IFS= read -r line; do
if [[ "$line" =~ \[\[#(.*)\]\] ]]; then
toc_text="${BASH_REMATCH[1]}"
toc_texts+=("$toc_text")
log_debug "TOC entry: $toc_text"
fi
done < <(echo "$toc_content")
# Count TOC entries
toc_entries=${#toc_texts[@]}
log_debug "TOC has $toc_entries entries"
# Check if any heading text has changed
headings_changed=false
if [[ ${#toc_texts[@]} -eq ${#heading_lines[@]} ]]; then
for ((i=0; i<${#heading_lines[@]}; i++)); do
# Clean heading text for comparison
clean_heading="${heading_lines[$i]}"
clean_heading="${clean_heading//\*\*/}"
clean_heading="${clean_heading//\*/}"
clean_heading="${clean_heading//\`/}"
if [[ "${toc_texts[$i]}" != "$clean_heading" ]]; then
log_debug "Heading text changed: '${toc_texts[$i]}' → '$clean_heading'"
headings_changed=true
break
fi
done
else
log_debug "Different number of headings (TOC: ${#toc_texts[@]}, Doc: ${#heading_lines[@]})"
headings_changed=true
fi
# Get the last modified time of the file
file_mtime=$(stat -f "%m" "$FILE_PATH" 2>/dev/null || stat -c "%Y" "$FILE_PATH")
last_run_time=$(stat -f "%m" "$LAST_RUN_FILE" 2>/dev/null || stat -c "%Y" "$LAST_RUN_FILE")
# Decide if update is needed
if [[ "$toc_entries" -eq 0 || "$headings_changed" == "true" || "$file_mtime" -lt "$last_run_time" ]]; then
reason=""
if [[ "$toc_entries" -eq 0 ]]; then
reason="TOC has no entries"
elif [[ "$headings_changed" == "true" ]]; then
reason="Heading content has changed"
else
reason="File was modified by user"
fi
log_info "Processing file: $FILE_PATH ($reason)"
update_toc "$FILE_PATH" "$start_line" "$end_line"
return $?
else
log_debug "TOC is up to date (entries: $toc_entries, headings: $heading_count)"
if [[ "$VERBOSE" == "true" ]]; then
echo -e " ${GREEN}✓ TOC already up to date${RESET}"
fi
return 1
fi
}
# Main script starts here
log_info "Starting TOC monitor run on vault: $VAULT_DIR"
# Convert exclude dirs from comma-separated to space-separated if needed
EXCLUDE_DIRS="${EXCLUDE_DIRS//,/ }"
# Build the find command with exclude directories
find_cmd="find \"$VAULT_DIR\""
for dir in $EXCLUDE_DIRS; do
find_cmd="$find_cmd -path \"*/$dir\" -prune -o"
done
find_cmd="$find_cmd -type f -name \"*.md\" -newermt \"1 minute ago\" -print"
log_debug "Find command: $find_cmd"
if [[ "$VERBOSE" == "true" ]]; then
echo -e "\n${MAGENTA}====================================================${RESET}"
echo -e "${MAGENTA} OBSIDIAN TOC MONITOR - STARTING RUN $(date '+%H:%M:%S')${RESET}"
echo -e "${MAGENTA}====================================================${RESET}\n"
echo -e "${CYAN}Checking for recently modified Markdown files in:${RESET}"
echo -e "${CYAN}$VAULT_DIR${RESET}\n"
if [[ -n "$EXCLUDE_DIRS" ]]; then
echo -e "${CYAN}Excluding directories:${RESET} $EXCLUDE_DIRS\n"
fi
fi
# Get the list of files to process
FILE_LIST=$(eval "$find_cmd")
# Process each file
if [[ -n "$FILE_LIST" ]]; then
while IFS= read -r file; do
process_file "$file"
done <<< "$FILE_LIST"
else
log_info "No recently modified files found"
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${YELLOW}No recently modified files found${RESET}"
fi
fi
# After processing all files, update the last run timestamp
touch "$LAST_RUN_FILE"
# Show summary
if [[ "$VERBOSE" == "true" ]]; then
echo -e "\n${MAGENTA}====================================================${RESET}"
if [[ $MODIFIED_COUNT -gt 0 ]]; then
echo -e "${GREEN} ✓ Updated $MODIFIED_COUNT TOCs out of $FILE_COUNT files checked${RESET}"
else
echo -e "${GREEN} ✓ No updates needed. Checked $FILE_COUNT files.${RESET}"
fi
echo -e "${MAGENTA}====================================================${RESET}\n"
fi
log_info "TOC monitor run completed. Updated $MODIFIED_COUNT files."
exit 0
