Creating Obsidian tables of content

April 14, 2025

Maintaining Tables of Contents in Obsidian with a Bash Script

When viewing longer Markdown notes in Obsidian, tables of content (TOC) help a lot with navigation. There is a handful of community plugins to help with TOC generation, but I have two issues with them:

It creates a dependency on code whose developer may lose interest and eventually abandon the project. At least one dynamic TOC plugin has suffered this fate.
All of the TOC plugins have the same visual result. When you navigate to a note, Obsidian places the focus at the top of the note, beneath the frontmatter. That’s fine unless the content starts with a TOC markup block, in which case it’s not the TOC itself that is displayed, but the markup for the TOC plugin itself as depicted in the image below.

For me the solution was to write a script that scans the vault looking for this pair of markers:

<!--ts-->
<!--te-->

that define the start and end for the TOC block. When these markers are found, the script inserts a TOC between the markers. The downside of course is that the markers, which are just raw HTML comments in the Markdown document are still visible when you’re in Edit mode. But it’s still better than not seeing the actual TOC at all when you navigate to a note.

Technical Details

Finding Files that Need TOC Updates

The script begins by using the find command to locate recently modified Markdown files in your Obsidian vault. It specifically looks for files modified within the last minute, making it efficient for running as a background process. This approach ensures that TOCs are updated only in notes you’ve recently worked on, rather than scanning your entire vault every time.

find_cmd="find \"$VAULT_DIR\" \
  -path \"*/$EXCLUDE_DIRS\" -prune \
  -o -type f \
  -name \"*.md\" \
  -newermt \"1 minute ago\" \
  -print"

For each file, the script:

Checks if valid TOC markers exist outside of code blocks
Counts the headings in the document
Compares the current TOC entries against the document’s headings
Updates the TOC if:
- The TOC is empty but headings exist
- The heading content has changed
- The file was modified since the last run

This strategy minimizes unnecessary updates and ensures TOCs stay in sync with your content as you write and edit.

Handling Code Blocks and Marker Detection

One of the challenges with this approach is distinguishing between actual TOC markers and examples of markers included in code blocks. For instance, if you’re writing documentation about the TOC script itself, you might include the TOC markers in a code example.

The script solves this by carefully tracking when it’s inside or outside of code blocks:

# Check for code block start/end
if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
    # Extract the fence type (``` or ~~~)
    fence_match="${BASH_REMATCH[1]}"
    
    if [[ "$in_code_block" == "false" ]]; then
        in_code_block=true
        code_fence="$fence_match"
    elif [[ "$line" == *"$code_fence"* ]]; then
        in_code_block=false
        code_fence=""
    fi
}

When analyzing a file, the script maintains state variables to track:

Whether we’re currently inside a code block
The type of code fence used (``` or ~~~)
Whether we’re in YAML frontmatter

This context-aware parsing ensures that TOC markers inside code blocks are completely ignored. This is crucial because it prevents the script from incorrectly modifying code examples or documentation that happens to contain the markers.

Usage

Basic Usage

The script can be run directly from the command line:

./obsidian_toc_monitor.sh /path/to/obsidian/vault

You can also specify which directories to exclude:

./obsidian_toc_monitor.sh --exclude ".obsidian,Templates,Attachments" /path/to/vault

Automating TOC Updates

There are several ways to automate the script depending on your operating system:

On macOS:

Using launchd - Create a launch agent to run the script periodically:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.user.obsidian.tocmonitor</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/obsidian_toc_monitor.sh</string>
        <string>/path/to/obsidian/vault</string>
    </array>
    <key>StartInterval</key>
    <integer>60</integer>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

Save this to ~/Library/LaunchAgents/com.user.obsidian.tocmonitor.plist and load it with:

launchctl load ~/Library/LaunchAgents/com.user.obsidian.tocmonitor.plist

Using cron - Add an entry to your crontab:

* * * * * /path/to/obsidian_toc_monitor.sh /path/to/obsidian/vault

On Linux:

Using systemd - Create a user service:

[Unit]
Description=Obsidian TOC Monitor

[Service]
Type=simple
ExecStart=/path/to/obsidian_toc_monitor.sh /path/to/obsidian/vault
Restart=always
RestartSec=60

[Install]
WantedBy=default.target

Save this to ~/.config/systemd/user/obsidian-toc.service and enable it:

systemctl --user enable --now obsidian-toc.service

Using cron (same as macOS)

Adding TOC Markers to Your Notes

To prepare a note for TOC generation, simply add these markers where you want the TOC to appear:

# My Note Title

<!--ts-->
<!--te-->

## First Section
Content here...


The script will detect these markers and insert a formatted TOC.

You get the benefits of a dynamic TOC without visual issues.

Script code

#!/bin/bash

# Default configuration
LOG_FILE="$HOME/Library/Logs/obsidian_toc_monitor.log"
LAST_RUN_FILE="/tmp/obsidian_toc_monitor_lastrun"
EXCLUDE_DIRS=".obsidian pdf Templates Attachments Meta"
DEBUG="${DEBUG:-false}"
VERBOSE="${VERBOSE:-true}"  # Set to true to show terminal output

# Global counters
FILE_COUNT=0
MODIFIED_COUNT=0

# Color codes for better terminal output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
RESET='\033[0m'

# Usage information
show_usage() {
    echo "Usage: $(basename "$0") [OPTIONS] VAULT_DIRECTORY"
    echo ""
    echo "Options:"
    echo "  -h, --help            Show this help message and exit"
    echo "  -q, --quiet           Run in quiet mode (no terminal output)"
    echo "  -d, --debug           Enable debug logging"
    echo "  -e, --exclude DIRS    Comma-separated list of directories to exclude (default: $EXCLUDE_DIRS)"
    echo "  -l, --log FILE        Path to log file (default: $LOG_FILE)"
    echo ""
    echo "Example:"
    echo "  $(basename "$0") /path/to/obsidian/vault"
    echo "  $(basename "$0") --exclude '.obsidian,pdf,Templates' /path/to/vault"
}

# Parse command line arguments
POSITIONAL_ARGS=()
while [[ $# -gt 0 ]]; do
    case $1 in
        -h|--help)
            show_usage
            exit 0
            ;;
        -q|--quiet)
            VERBOSE=false
            shift
            ;;
        -d|--debug)
            DEBUG=true
            shift
            ;;
        -e|--exclude)
            EXCLUDE_DIRS="$2"
            shift 2
            ;;
        -l|--log)
            LOG_FILE="$2"
            shift 2
            ;;
        -*|--*)
            echo "Error: Unknown option $1"
            show_usage
            exit 1
            ;;
        *)
            POSITIONAL_ARGS+=("$1")
            shift
            ;;
    esac
done
set -- "${POSITIONAL_ARGS[@]}"

# Check if vault directory was provided
if [ $# -lt 1 ]; then
    echo "Error: No vault directory specified"
    show_usage
    exit 1
fi

# Set the vault directory
VAULT_DIR="$1"

# Check if vault directory exists
if [ ! -d "$VAULT_DIR" ]; then
    echo "Error: Vault directory does not exist: $VAULT_DIR"
    exit 1
fi

# Ensure log directory exists
mkdir -p "$(dirname "$LOG_FILE")"

# Touch the last run file if it doesn't exist
[ ! -f "$LAST_RUN_FILE" ] && touch "$LAST_RUN_FILE"

# Logging functions
log_info() {
    echo "[INFO] $1" >> "$LOG_FILE"
    if [[ "$VERBOSE" == "true" ]]; then
        echo -e "${GREEN}[INFO]${RESET} $1"
    fi
}

log_error() {
    echo "[ERROR] $1" >> "$LOG_FILE"
    if [[ "$VERBOSE" == "true" ]]; then
        echo -e "${RED}[ERROR]${RESET} $1"
    fi
}

log_debug() {
    if [[ "$DEBUG" == "true" ]]; then
        echo "[DEBUG] $1" >> "$LOG_FILE"
        echo -e "${BLUE}[DEBUG]${RESET} $1"
    fi
}

generate_toc() {
    local FILE_PATH="$1"
    local TEMP_FILE="$2"
    
    # First add the TOC heading
    echo "## Table of contents" > "$TEMP_FILE"
    echo "" >> "$TEMP_FILE"  # Add blank line after the heading
    
    # Find headings and generate TOC
    local min_level=6
    declare -a heading_lines
    local in_code_block=false
    local code_fence=""
    local in_frontmatter=false
    
    while IFS= read -r line; do
        # Handle YAML frontmatter
        if [[ "$line" == "---" ]]; then
            if [[ "$in_frontmatter" == "false" ]]; then
                in_frontmatter=true
            else
                in_frontmatter=false
            fi
            continue
        fi
        
        # Skip processing if in frontmatter
        if [[ "$in_frontmatter" == "true" ]]; then
            continue
        fi
        
        # Skip processing the TOC heading itself
        if [[ "$line" == "## Table of contents" ]]; then
            continue
        fi
        
        # Check for code block start/end
        if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
            # Extract the fence type (``` or ~~~)
            fence_match="${BASH_REMATCH[1]}"
            
            if [[ "$in_code_block" == "false" ]]; then
                in_code_block=true
                code_fence="$fence_match"
            elif [[ "$line" == *"$code_fence"* ]]; then
                in_code_block=false
                code_fence=""
            fi
            continue
        fi
        
        # Only process headings if we're not inside a code block
        if [[ "$in_code_block" == "false" ]]; then
            if [[ "$line" =~ ^[[:space:]]*(#{1,6})[[:space:]]+(.*[^[:space:]]) ]]; then
                # The heading marker is captured in BASH_REMATCH[1]
                local marker="${BASH_REMATCH[1]}"
                local level=${#marker}
                # The heading text is captured in BASH_REMATCH[2]
                local title="${BASH_REMATCH[2]}"
                
                # Update min level if needed
                if [[ $level -lt $min_level ]]; then
                    min_level=$level
                fi
                
                # Store heading info for later processing
                heading_lines+=("$level|$title")
                
                if [[ "$VERBOSE" == "true" ]]; then
                    echo -e "    ${CYAN}→ Level $level:${RESET} $title"
                fi
            fi
        fi
    done < "$FILE_PATH"
    
    # Generate TOC entries
    for entry in "${heading_lines[@]}"; do
        IFS='|' read -r level title <<< "$entry"
        
        # Clean markdown formatting
        clean_title="$title"
        clean_title="${clean_title//\*\*/}"  # Remove bold
        clean_title="${clean_title//\*/}"    # Remove italic
        clean_title="${clean_title//\`/}"    # Remove code
        
        # Calculate relative indent level
        rel_level=$((level - min_level))
        
        if [[ $rel_level -eq 0 ]]; then
            echo "* [[#$clean_title]]" >> "$TEMP_FILE"
        else
            indent=$(printf "%$((rel_level * 4))s" "")
            echo "${indent}- [[#$clean_title]]" >> "$TEMP_FILE"
        fi
    done
    
    # Return the number of headings found
    echo ${#heading_lines[@]}
}

update_toc() {
    local FILE_PATH="$1"
    local START_LINE="$2"
    local END_LINE="$3"
    local was_updated=false
    
    log_debug "Updating TOC for $FILE_PATH (markers at lines $START_LINE and $END_LINE)"
    
    # Create temporary files
    local TEMP_TOC=$(mktemp)
    local TEMP_FILE=$(mktemp)
    
    # Generate the TOC
    if [[ "$VERBOSE" == "true" ]]; then
        echo -e "  ${YELLOW}Parsing headings...${RESET}"
    fi
    
    local heading_count=$(generate_toc "$FILE_PATH" "$TEMP_TOC")
    
    if [[ "$VERBOSE" == "true" ]]; then
        echo -e "  ${YELLOW}Generating TOC with $heading_count headings...${RESET}"
    fi
    
    # Extract current TOC content for comparison
    local current_toc=$(sed -n "${START_LINE},${END_LINE}p" "$FILE_PATH")
    
    # Create updated file with new TOC
    {
        # Copy lines before TOC
        sed -n "1,$((START_LINE-1))p" "$FILE_PATH"
        
        # Copy start marker line
        sed -n "${START_LINE}p" "$FILE_PATH"
        
        # Insert new TOC content
        cat "$TEMP_TOC"
        
        # Copy end marker line
        sed -n "${END_LINE}p" "$FILE_PATH"
        
        # Copy lines after TOC
        sed -n "$((END_LINE+1)),\$p" "$FILE_PATH"
    } > "$TEMP_FILE"
    
    # Extract new TOC content for comparison
    local new_toc=$(sed -n "${START_LINE},$((START_LINE+1+$(wc -l < "$TEMP_TOC")+1))p" "$TEMP_FILE")
    
    # Compare TOCs to see if an update is needed
    if ! diff -q <(echo "$current_toc") <(echo "$new_toc") >/dev/null; then
        # TOC content has changed - update the file
        if [[ "$VERBOSE" == "true" ]]; then
            echo -e "  ${GREEN}✓ Updating TOC with $heading_count entries${RESET}"
        fi
        
        # Move updated file into place
        mv "$TEMP_FILE" "$FILE_PATH"
        
        # Add a small delay and touch the file to ensure changes are detected
        sleep 0.1
        touch "$FILE_PATH"
        
        log_info "TOC updated for: $FILE_PATH"
        was_updated=true
        
        # Increment global counter 
        ((MODIFIED_COUNT++))
    else
        if [[ "$VERBOSE" == "true" ]]; then
            echo -e "  ${GREEN}✓ TOC already up to date${RESET}"
        fi
        log_debug "No TOC changes needed for: $FILE_PATH"
    fi
    
    # Clean up temporary files
    rm -f "$TEMP_TOC" "$TEMP_FILE" 2>/dev/null
    
    return $([ "$was_updated" = true ] && echo 0 || echo 1)
}

find_valid_toc_markers() {
    local FILE_PATH="$1"
    local START_LINE_VAR="$2"
    local END_LINE_VAR="$3"
    
    # Use grep to find potential marker line numbers
    local potential_start_markers=$(grep -n "<!-- *ts *-->" "$FILE_PATH" | cut -d':' -f1)
    local potential_end_markers=$(grep -n "<!-- *te *-->" "$FILE_PATH" | cut -d':' -f1)
    
    # If no potential markers found at all, return false
    if [[ -z "$potential_start_markers" || -z "$potential_end_markers" ]]; then
        return 1
    fi
    
    # Parse the file line by line to verify markers are outside code blocks
    local line_num=0
    local in_code_block=false
    local code_fence=""
    local in_frontmatter=false
    local valid_start_line=""
    local valid_end_line=""
    
    while IFS= read -r line; do
        ((line_num++))
        
        # Handle YAML frontmatter
        if [[ "$line" == "---" ]]; then
            if [[ "$in_frontmatter" == "false" ]]; then
                in_frontmatter=true
            else
                in_frontmatter=false
            fi
            continue
        fi
        
        # Skip processing if in frontmatter
        if [[ "$in_frontmatter" == "true" ]]; then
            continue
        fi
        
        # Check for code block start/end
        if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
            # Extract the fence type (``` or ~~~)
            fence_match="${BASH_REMATCH[1]}"
            
            if [[ "$in_code_block" == "false" ]]; then
                in_code_block=true
                code_fence="$fence_match"
                log_debug "Line $line_num: Code block starts"
            elif [[ "$line" == *"$code_fence"* ]]; then
                in_code_block=false
                code_fence=""
                log_debug "Line $line_num: Code block ends"
            fi
            continue
        fi
        
        # Only check for TOC markers if we're not inside a code block
        if [[ "$in_code_block" == "false" ]]; then
            # Check if current line is in our potential marker list
            if echo "$potential_start_markers" | grep -q "^$line_num$"; then
                valid_start_line=$line_num
                log_debug "Line $line_num: Valid start marker found"
            fi
            
            if echo "$potential_end_markers" | grep -q "^$line_num$"; then
                valid_end_line=$line_num
                log_debug "Line $line_num: Valid end marker found"
            fi
        fi
    done < "$FILE_PATH"
    
    # Only succeed if we found valid markers outside code blocks
    if [[ -z "$valid_start_line" || -z "$valid_end_line" ]]; then
        return 1
    fi
    
    # Set the output variables
    eval "$START_LINE_VAR=$valid_start_line"
    eval "$END_LINE_VAR=$valid_end_line"
    return 0
}

process_file() {
    local FILE_PATH="$1"
    local update_needed=false
    
    # Skip if file doesn't exist or isn't a regular file
    if [[ ! -f "$FILE_PATH" ]]; then
        return 1
    fi
    
    # Skip non-markdown files
    if [[ ! "$FILE_PATH" =~ \.(md|markdown)$ ]]; then
        return 1
    fi
    
    # Increment global counter
    ((FILE_COUNT++))
    
    log_debug "Checking file: $FILE_PATH"
    if [[ "$VERBOSE" == "true" ]]; then
        filename=$(basename "$FILE_PATH")
        echo -e "${YELLOW}Checking:${RESET} $filename"
    fi
    
    # Find valid TOC markers (outside code blocks)
    local start_line=""
    local end_line=""
    if ! find_valid_toc_markers "$FILE_PATH" start_line end_line; then
        log_debug "No valid TOC markers found in file - $FILE_PATH"
        if [[ "$VERBOSE" == "true" ]]; then
            echo -e "  ${YELLOW}→ No valid TOC markers found${RESET}"
        fi
        return 1
    fi
    
    log_debug "Found valid TOC markers at lines $start_line and $end_line"
    
    # Count the number of headings in the file, excluding those in code blocks
    heading_count=0
    in_code=false
    code_fence=""
    in_front=false
    declare -a heading_lines
    
    while IFS= read -r line; do
        # Handle YAML frontmatter
        if [[ "$line" == "---" ]]; then
            if [[ "$in_front" == "false" ]]; then
                in_front=true
            else
                in_front=false
            fi
            continue
        fi
        
        if [[ "$in_front" == "true" ]]; then
            continue
        fi
        
        # Check for code block start/end
        if [[ "$line" =~ ^[[:space:]]*(\`\`\`|\~\~\~) ]]; then
            fence="${BASH_REMATCH[1]}"
            if [[ "$in_code" == "false" ]]; then
                in_code=true
                code_fence="$fence"
            elif [[ "$line" == *"$code_fence"* ]]; then
                in_code=false
                code_fence=""
            fi
            continue
        fi
        
        # Only count headings outside code blocks
        if [[ "$in_code" == "false" && "$line" =~ ^[[:space:]]*(#{1,6})[[:space:]]+(.*[^[:space:]]) ]]; then
            # Skip TOC heading
            if [[ "$line" =~ "Table of contents" ]]; then
                continue
            fi
            
            # Extract heading level and text
            level=${#BASH_REMATCH[1]}
            title="${BASH_REMATCH[2]}"
            
            # Store heading info for later comparison
            heading_lines+=("$title")
            ((heading_count++))
        fi
    done < "$FILE_PATH"
    
    log_debug "Found $heading_count headings in file (excluding code blocks)"
    
    # Extract TOC content (between markers)
    local toc_content=$(sed -n "$((start_line+1)),$((end_line-1))p" "$FILE_PATH")
    
    # Extract TOC entry texts
    declare -a toc_texts
    while IFS= read -r line; do
        if [[ "$line" =~ \[\[#(.*)\]\] ]]; then
            toc_text="${BASH_REMATCH[1]}"
            toc_texts+=("$toc_text")
            log_debug "TOC entry: $toc_text"
        fi
    done < <(echo "$toc_content")
    
    # Count TOC entries
    toc_entries=${#toc_texts[@]}
    log_debug "TOC has $toc_entries entries"
    
    # Check if any heading text has changed
    headings_changed=false
    if [[ ${#toc_texts[@]} -eq ${#heading_lines[@]} ]]; then
        for ((i=0; i<${#heading_lines[@]}; i++)); do
            # Clean heading text for comparison
            clean_heading="${heading_lines[$i]}"
            clean_heading="${clean_heading//\*\*/}"
            clean_heading="${clean_heading//\*/}"
            clean_heading="${clean_heading//\`/}"
            
            if [[ "${toc_texts[$i]}" != "$clean_heading" ]]; then
                log_debug "Heading text changed: '${toc_texts[$i]}' → '$clean_heading'"
                headings_changed=true
                break
            fi
        done
    else
        log_debug "Different number of headings (TOC: ${#toc_texts[@]}, Doc: ${#heading_lines[@]})"
        headings_changed=true
    fi
    
    # Get the last modified time of the file
    file_mtime=$(stat -f "%m" "$FILE_PATH" 2>/dev/null || stat -c "%Y" "$FILE_PATH")
    last_run_time=$(stat -f "%m" "$LAST_RUN_FILE" 2>/dev/null || stat -c "%Y" "$LAST_RUN_FILE")
    
    # Decide if update is needed
    if [[ "$toc_entries" -eq 0 || "$headings_changed" == "true" || "$file_mtime" -lt "$last_run_time" ]]; then
        reason=""
        if [[ "$toc_entries" -eq 0 ]]; then
            reason="TOC has no entries"
        elif [[ "$headings_changed" == "true" ]]; then
            reason="Heading content has changed"
        else
            reason="File was modified by user"
        fi
        
        log_info "Processing file: $FILE_PATH ($reason)"
        update_toc "$FILE_PATH" "$start_line" "$end_line"
        return $?
    else
        log_debug "TOC is up to date (entries: $toc_entries, headings: $heading_count)"
        if [[ "$VERBOSE" == "true" ]]; then
            echo -e "  ${GREEN}✓ TOC already up to date${RESET}"
        fi
        return 1
    fi
}

# Main script starts here
log_info "Starting TOC monitor run on vault: $VAULT_DIR"

# Convert exclude dirs from comma-separated to space-separated if needed
EXCLUDE_DIRS="${EXCLUDE_DIRS//,/ }"

# Build the find command with exclude directories
find_cmd="find \"$VAULT_DIR\""
for dir in $EXCLUDE_DIRS; do
    find_cmd="$find_cmd -path \"*/$dir\" -prune -o"
done
find_cmd="$find_cmd -type f -name \"*.md\" -newermt \"1 minute ago\" -print"

log_debug "Find command: $find_cmd"

if [[ "$VERBOSE" == "true" ]]; then
    echo -e "\n${MAGENTA}====================================================${RESET}"
    echo -e "${MAGENTA}  OBSIDIAN TOC MONITOR - STARTING RUN $(date '+%H:%M:%S')${RESET}"
    echo -e "${MAGENTA}====================================================${RESET}\n"
    echo -e "${CYAN}Checking for recently modified Markdown files in:${RESET}"
    echo -e "${CYAN}$VAULT_DIR${RESET}\n"
    
    if [[ -n "$EXCLUDE_DIRS" ]]; then
        echo -e "${CYAN}Excluding directories:${RESET} $EXCLUDE_DIRS\n"
    fi
fi

# Get the list of files to process
FILE_LIST=$(eval "$find_cmd")

# Process each file
if [[ -n "$FILE_LIST" ]]; then
    while IFS= read -r file; do
        process_file "$file"
    done <<< "$FILE_LIST"
else
    log_info "No recently modified files found"
    if [[ "$VERBOSE" == "true" ]]; then
        echo -e "${YELLOW}No recently modified files found${RESET}"
    fi
fi

# After processing all files, update the last run timestamp
touch "$LAST_RUN_FILE"

# Show summary
if [[ "$VERBOSE" == "true" ]]; then
    echo -e "\n${MAGENTA}====================================================${RESET}"
    if [[ $MODIFIED_COUNT -gt 0 ]]; then
        echo -e "${GREEN}  ✓ Updated $MODIFIED_COUNT TOCs out of $FILE_COUNT files checked${RESET}"
    else
        echo -e "${GREEN}  ✓ No updates needed. Checked $FILE_COUNT files.${RESET}"
    fi
    echo -e "${MAGENTA}====================================================${RESET}\n"
fi

log_info "TOC monitor run completed. Updated $MODIFIED_COUNT files."
exit 0