Research

The idea of the research is to compare different kinds of sequences.

For that we use compression algorithms.

We take our control sequence, build some compression structure (e.g. a Huffman treee), and then measure the compression rate.

Then we take another sequence and use the same compression structure, compress this one.

The difference of the original and the resulting compression rates is our metric.

This research will be applied to programming languages, texts in natural languages, music scores, etc.

Implementation

compsim is a small Python package that realises this idea with a pluggable model layer — the same metric works over any compression structure:

• Huffman (huffman) — the structure named above: a codebook built from the control sequence, with Laplace smoothing + an escape symbol so out-of-vocabulary symbols in another sequence stay encodable.
• Order-N Markov (markov) — a context model scored by cross-entropy (fractional bits); the most theoretically grounded variant, closest to true entropy.
• Generic (generic) — an NCD-style baseline backed by stdlib zlib/bz2/lzma, used as a sanity check.

The distance between a control sequence C and another sequence X is the extra bits-per-symbol needed to encode X with C's structure:

distance(C, X) = rate(X | structure_of_C) − rate(C | structure_of_C)

which is ~0 when X is statistically like C and grows as they diverge (related to KL divergence / cross-entropy). See src/compsim/metric.py.

First domain: music scores

Scores (MIDI / MusicXML / ABC / **kern) are parsed with music21 and tokenised into pitch, interval (transposition-invariant, the default), or (pitch, duration) sequences — see src/compsim/tokenizers/music.py.

Setup

python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"

Run

# End-to-end music experiment: distance matrix -> heatmap + dendrogram + JSON in out/
.venv/bin/python scripts/run_music.py --data data/music --out out --model markov

# Also try:  --model huffman | generic   and   --scheme pitch | interval | pitch_dur

The bundled toy corpus under data/music/ (stepwise "scale" tunes vs. leaping "arpeggio" tunes) cleanly separates into two clusters under every model.

Using music21's built-in corpus (no download)

music21 ships with ~3,000 real works (Bach chorales, Palestrina, Monteverdi, Mozart, Beethoven, trecento, folk tunes, …). Run the experiment straight on them:

.venv/bin/python scripts/run_music21.py --composers bach:8 monteverdi:8 trecento:8

Each name:count pulls up to count works by that composer (omit :count for all). On bach/monteverdi/trecento the Monteverdi madrigals form a clean, distinct cluster while Bach and trecento overlap more — the metric picks up real stylistic structure. See load_music21_corpus in src/compsim/corpus.py.

Tests

.venv/bin/python -m pytest -q