Changelog
All notable changes to this project are documented here. The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
The
contrib/Python packages have their own decoupled version and changelog: see the contrib changelog.
For curated, upgrade-focused notes (highlights and per-version migration steps), see the Release notes.
Unreleased
Added
-
readdirplusis implemented, folding the per-entrylookupinto the directory read: a client that stats what it lists —ls -l, every media scanner — spends one round trip on the directory instead of one more per entry. Expect a few tens of percent off a repeat traversal rather than a multiple; a cold one is dominated by synthesis, where the round trip is a few percent. Directories and the synthetic entries are answered inline, and the file entries fan out across the worker pool in rounds, because concurrentlookups already spread across that pool and a serially resolved page would be slower for a threaded scanner than what it replaces.FUSE_READDIRPLUS_AUTOis requested too, so the kernel keeps using plainreaddirfor a listing nobody stats, where the larger entries would only cost reply pages.musefs_readdirplus_totalreports whether the kernel is sending the op at all (#667). -
--trust-backing-mtimeskips the backing re-stat thatgetattrperforms on a metadata-cache hit, serving the cached size and mtime instead. Off by default, and scoped togetattralone. The re-stat exists to catch an on-disk change that leftcontent_versionuntouched (#279), which is the right default and stays the default; it has no escape hatch for backings where astatis not roughly a microsecond. On NFS, SMB, or a spun-down array it is a network round trip or a head seek, and a warm cache does not help: a scanner walking ten thousand tracks pays ten thousand synchronous stats on every pass, and--attr-ttl-mscannot debounce them because each track is stated once per traversal and a traversal outlives any TTL worth setting.openand the read paths validate unconditionally either way, so a replaced backing file is still caught before a byte is served, and the cold traversal that populates the cache stats regardless — the hit-path stat is the cost of every pass after the first, not of the first.musefs_trust_backing_mtimereports the flag state, which is what tells a quietmusefs_backing_stats_totalfrom a disabled counter (#668). -
Chaptered
.m4bfiles are supported. Amoovmay now hold chapter tracks (text,sbtl) alongside its single audio (soun) track, and every track'sstco/co64chunk offsets are relocated when themoovis regenerated, not just the first track's. A Nero chapter list (moov/udta/chpl) is copied verbatim into the rebuiltudta. Previously every chaptered file — which is to say most of an audiobook library, and chapters are why the.m4bextension exists — was counted asunparseableat scan time (#672). -
Musefs::drain_prefetchwaits for the Phase-2 prefetch pool to finish every job it accepted, or a timeout to elapse. Serving never needs it — prefetch is fire-and-forget there — but sampling the prefetch counters without it misses reads still in flight, and a caller that owns the backing filesystem itself (the latency-injecting mount the read benches use) can otherwise tear it down under a worker mid-read and park that thread in uninterruptible sleep (#671). -
musefs_readahead_prefetch_reads_totalandmusefs_readahead_prefetch_bytes_total(#671) report the positioned backing reads issued by the Phase-2 prefetch workers. The serve-pathmusefs_backing_pread_*counters never saw those threads, so a prefetcher re-reading the stream several times over was invisible to every counter the daemon exposes and showed up only as backing I/O nothing accounted for. That is exactly the shape of the amplification bug fixed in this release, which is why the counters exist. -
musefs_dir_handle_rejections_total(#626), a monotonic counter ofopendircalls that could not be given a cached directory snapshot. The existingmusefs_dir_handlesgauge cannot stand in for it: saturation is bursty, and a walk that produced 7,525 rejections never showed a gauge sample above 593. It is also the signal that the stateless path from #616 is in use and directories are being rebuilt on everyreaddir. -
musefs_serve_warns_suppressed_total(#653), a monotonic counter of serve-path failure warnings the rate limiter downgraded todebug. The count previously escaped only as a parenthetical inside the next admitted warning, which is both unscrapeable and carried by the admitted lines alone. The failure mode matches #626's: suppression is bursty by construction — 10 admitted per 30-second window, the rest dropped — so a scrape landing between bursts sees nothing, and "quiet" and "failing faster than it can log" look identical. Since the limiter moved intomusefs-core(#650) the counter covers the synthesis warns too, not just the FUSE errno path.
Changed
-
Directory handles on the same directory share one listing instead of copying it each.
opendirtook a private snapshot per handle, so the table's memory was the directory's width times the handle count: on a template that collapses a library into one directory, a client opening the 1,024-handle cap on it — which needs no privilege — pinned tens of gigabytes. A listing is now keyed by directory and virtual-tree generation, and handles that agree on both share it. All but the first also skip the tree walk that builds one, which an over-capreaddirconsults before rebuilding as well.musefs_dir_listingsreports the distinct-listing count behindmusefs_dir_handles(#675). -
An MP4 file skipped for its track layout now reports the handler types found (
unsupported MP4 track layout: expected one audio (soun) track, optionally with text/sbtl chapter tracks; found [soun, vide]) instead of a bare "not a supported MP4/M4A file", so the skip explains itself (#672). -
benches/storage_tunables_bench.shgains aprefetchmode that A/Bs two or more musefs binaries (MUSEFS_PREFETCH_BINS="label=path ...") over one NFS+netem corpus, and its real-corpus filter now picks up.opusfiles. Its NFS modes also disable NFS LOCALIO for the run: on Linux 6.12+ a loopback mount negotiates local I/O and bypasses the RPC transport, sotc netemonlohad no effect on the data path and every "NFS" row measured local disk at GB/s (#671). -
Behavior change. A scan that hits a DB constraint violation on one file now runs to completion instead of stopping there (#662). Three observable consequences. It exits 2 —
scancompleted, at least one file failed — where it previously exited 1 as a hard error, so a pipeline keying on the exit code sees a different value for the same library. The store ends up holding every file in the library except the rejected one, rather than only the batches that committed before the abort, so a rescan after the fix no longer has an unknown amount of the walk left to redo. And the rejected file is reported rather than fatal: it is named in the log with the constraint text, counted in the newrejectedbucket of thefailed N: …summary, and missing from the mount while everything else is served. Anything that treated a constraint violation as a signal to stop the scan no longer gets one; the exit code and that summary are the signals to key on. The engineering is in Fixed below. See Scanning and Exit codes. -
The
tags.valuecap rises from 256 KiB to 16 MiB − 1, andtrack_art.descriptionfrom 1 KiB to 8 KiB (schemaMIGRATION_V3). The new tag cap is FLAC's 24-bit metadata-block ceiling — the largest tag synthesis could ever serve — so the store no longer refuses a tag the format itself can carry. Existing stores upgrade in place, and automatically, on the nextmusefs scanormusefs mount: both open the store read-write and run the migration, which only widens the constraints and so carries every existing row across. No rescan of audio is needed and nothing has to be regenerated. -
A backing file whose metadata exceeds a store limit now fails that file instead of being stored with the offending part quietly dropped. Oversize embedded art and binary tags were previously omitted from an otherwise-stored track with only a
warnto show for it, which is easy to lose in a scan of ten thousand files and leaves a mount silently missing data. Such a file is now logged with its path, what was too big, its size and the limit, countedfailed, and skipped; the rest of the directory scans normally. -
The virtual tree interns each node name into one shared
Arc<str>rather than storing it separately inNode.name,Node.rendered_name, thechildrenkey and bothrendered_childrenkeys (#617). Measured over 200,000 tracks / 222,001 nodes: ~1713 to ~1284 bytes per track (~84 MiB, -25%), with tree build 10-15% faster from the removed allocations. A case-insensitive mount saves more, sincefolded_childrenheld a sixth copy. The tuning guide gained a "Memory footprint" section. The full rendered paths were still stored twice at this point; the next entry shares those as well. -
Each entry's rendered path is stored once and shared between the inode allocator's map key and
TrackRenderState.path, rather than allocated independently by each (#629). Measured over 200,000 tracks with 41-byte paths: 1150 to 1078 bytes per track (-6%); with 137-byte paths the saving is -14%. What it removes is exactly one whole path per track, so the gain tracks path length and template depth rather than track count — a flat--templategains little. Sharing is conditional: a disambiguated leaf keeps its own allocation, since keying it on the path its bare-named sibling already interned would collapse two nodes onto one inode. -
Full tree rebuilds and the head of every scan read projected columns instead of materializing a whole
Trackper row (#621) — roughly 40 MB of transient allocation on a 200,000-track store, on a path already holding a pool connection. -
The serve-path warn rate limiter is process-wide instead of FUSE-local (#650). It bounds failure warns to a burst of 10 per 30-second window, but it lived in
musefs-fuseand was reachable only from the errno-reply path, so the warns synthesis itself emits bypassed it: a dropped Vorbis tag key (once per header-cache miss, and that cache is byte-budgeted, so an evicting library re-warns for the same track indefinitely), art over the byte cap, and a failed art-blob read — the last fires per art window, so one bad blob produced many lines for one file. The limiter now lives inmusefs-corenext totelemetry.rsand both crates share one budget, which is the right unit: the operator's concern is total serve-path log volume, not per-crate volume. Only the budget is shared, not the attribution: the emit side is themusefs_core::serve_warn!macro, so each record still takes its target from the call site's own module and per-crateRUST_LOGfiltering (RUST_LOG=warn,musefs_fuse=debug) reaches exactly what it did before. -
readdir's unknown-fhfallback runs on the worker pool instead of inline on the fuser dispatch thread (#623), matching the offload every other blocking operation already used. This matters more now that over-capopendirmakes that fallback the normal path for large directories. -
Scan failures are now broken down by reason and their per-file warnings capped (#651). A scan that ends
failed 37also logsfailed 37: unparseable=30, io=5, oversize=2(andwalk errors N: …for directories the walk could not read), so the number that drives the exit-2 partial-failure signal explains itself instead of having to be reconstructed from N individual lines. The per-file messages themselves are capped at ten per reason per scan, the rest dropping todebug— an unreadable subtree or a share that went away mid-scan no longer emits one warning per file. The existing per-extension skip breakdown moves fromwarntoinfo(so it now needs-v/RUST_LOG=info): cover art and.cuesidecars are the normal contents of a music library, and a warning on every healthy scan only teaches operators to tune warnings out. Theskippedcount itself is unchanged and still printed in the per-target summary.
Fixed
-
A track that renders to
.musefs-metricsor.metadata_never_indexat the mount root no longer collides with the synthetic entry the FUSE layer injects there (#681).readdirappended the synthetic name to the root listing whilelookupintercepted it before consulting the tree, so the name appeared twice with different inodes,lsreported the synthetic inode for both rows, and the user's own subtree was unreachable anywhere under the mount. Both names are now reserved in the virtual-tree namespace (musefs_core::RESERVED_ROOT_NAMES): a rendered root component that lands on one is pushed to its(2)rank, exactly as a colliding rendered name already is, so the synthetic entry keeps the base name and the user's data keeps a reachable one. The reservation is applied once, when the tree is built, and does not depend on--expose-metricsor on the platform — the same library therefore mounts to the same paths and inodes whichever way the flag is set and wherever it runs. -
A directory that had been ranked away from its base name by a collision is no longer re-created once per track. Which directory a path component belongs to is now decided by the rendered name rather than the stored one, so a library where a file renders to the same name as a directory — say a track that renders to
Livenext to an album directoryLive— serves that directory's tracks from oneLive (2)instead of scattering them acrossLive (2),Live (3),Live (4), one directory per track. By the same correction, a directory whose rendered name is literally another's rank keeps its own contents instead of absorbing them:Live(ranked toLive (2)) and a realLive (2)are two directories again. Surfaced while reserving the injected root names, which reach the same collision path (#681). -
Moving a backing file no longer wedges its track for the life of the mount (#679). A scan that retargets a row to a relocated file rewrites the path and the freshness stamp and correctly leaves
content_versionalone — the served bytes did not change — but thegetattrsize cache, the layout cache and every open file handle accepted their cached entry oncontent_versionalone while holding the pre-move path and stamp. Each then validated the live file against the old stamp and failed, permanently: the file listed as-????????? ?and every read returnedEIOuntil a remount. Both caches now compare the row's backing-source identity as well as its content identity, and a poll whose changelog names any track advances the refresh generation, so open handles re-resolve too. A move is not the only way in: any in-place re-stamp that leaves the content unchanged reached the same wedge. -
One unparseable
METADATA_BLOCK_PICTUREno longer discards every other embedded picture in the same Ogg file, and the drop is logged instead of being swallowed by the scan path. Base64 decoding also tolerates ASCII whitespace, so a value wrapped in the older 76-column MIME style decodes rather than failing at the first line break (#673). -
A panicking worker-pool task permanently leaked a SQLite read connection and up to three file descriptors (#669).
threadpoolretires a worker that unwinds and spawns a replacement, and the replacement gets a freshThreadId— the keyDbPool::PerThreadstores connections under, and never evicts. The dead worker's connection therefore stayed in the map for the life of the mount whilemusefs_pool_workerskept reading healthy.read,lookup,getattrandopenalready ran their synthesis inside acatch_unwind, butopendir, thereaddirstateless fallback and bothpoll_refreshtasks did not. Every pool submission now goes through one outer panic boundary, so no task can unwind out of a worker, and the two directory paths additionally guard their listing build so a panic there is answered withEIOinstead of dropping the reply and hanging the syscall. Per-worker connection counts are now genuinely bounded by--workers. -
Phase-2 read-ahead prefetch (
--read-ahead-prefetch) amplified reads instead of merely adding overhead (#671).ReadAhead::insert_windowtrimmed the ring to the first window lying fully behind the read frontier and fell back to index 0 when it found none. Windows are sorted by start, so index 0 is the lowest offset — the window the reader is currently inside. The frontier (next_expected) is the end of the last served read, not the end of the window that served it, so a 512 KiB window feeding 128 KiB reads is never "fully behind" until the reader has consumed all of it, and under ring pressure the fallback dropped exactly that window. Driving the realReadAheadwith the realplan_prefetch/prefetch_depthlogic over 400 sequential 128 KiB reads: 398 of 400 reads missed and refilled synchronously, 3159 MiB of foreground backing reads for 50 MiB asked. The window keeps doubling across those refills becauseoff == next_expectedstill holds, so it is pure amplification rather than seek thrash. The eviction order now prefers a window already fully consumed, then the furthest-future window the reader is not inside, andread_intoadvances the frontier before inserting so the trim sees where the reader actually is.Fixing that exposed a second defect it had been masking. A prefetch dispatch stops at the first window boundary at or past the horizon, so
prefetched_uptolegitimately ends up beyond the horizon;plan_prefetchread that overshoot as a backward seek and re-dispatched the whole horizon from the reader's position on the very next read, once the adaptive window outgrew the FUSE read size. With the eviction fix alone, a real 866 MiB FLAC read through a real kernel mount cost 8.5 GiB of backing reads (9.9x).plan_prefetchnow tolerates an overshoot of up to one window before treating the watermark as a seek, and the same read costs 1.01x the file.With both fixed, the Phase-2 story changes on high-latency backends. On a real loopback NFS mount at 200 ms RTT (
tc netem, LOCALIO disabled) prefetch is now a ~30% single-stream win, 6.3 → 8.2 MB/s, and a ~5% win on four concurrent streams; before the fix it was a regression on both, which is what the earlier "~10% overhead" finding was actually measuring. It stays opt-in: on local disk and overmusefs-latencyfsit still reads the stream a second time speculatively (≈2x the backing bytes) for wall time within noise of amplification alone, so the win is one backend and one run. Measurements and method are in Benchmarks. -
A DB constraint violation raised while ingesting one file no longer aborts the whole scan (#662). This changes observable behavior — the exit code and what the store holds afterwards — and Changed above states that part; what follows is why and how. Cap violations were pre-checked in
check_storable(#644), but that covers only the caps the scanner knows to look for; every other constraint the schema enforces — theCHECKs and theUNIQUE/primary-key constraints — was discovered by SQLite inside the ingest transaction, where the per-file context is gone, and propagated out as fatal. The reported case (#659) killed a scan 41% into an 891k-file library, about an hour in, leaving whatever the earlier batches had committed and no record of where the walk stopped. Pre-checking each newly discovered constraint does not converge, so the error is now classified at the ingest boundary instead: a constraint violation (SQLITE_CONSTRAINT, any extended code) is attributable to the rows one file wrote, and becomes onefailedfile in a newrejectedbucket, named in the log with the constraint text and reported in the end-of-scan breakdown. Errors that say the run itself cannot proceed —SQLITE_CORRUPT,SQLITE_FULL,SQLITE_IOERR,SQLITE_READONLY,SQLITE_NOTADB, and any code this build does not recognise — still abort with the message they always did;SQLITE_BUSYis neither and stays with the writer's locking policy. Because the production path commits throughBulkWriter, whose transaction holds a whole batch, each file is now ingested inside aSAVEPOINT(BulkWriter::item): a statement-levelABORTundoes only the statement that hit the constraint, so without one the batch would commit a half-ingested track — atracksrow whose tags never landed. The savepoint rolls the rejected file back whole and leaves the rest of the batch committable. -
A scan no longer aborts on a backing file that carries the same tag key as both a text value and a binary payload (#659).
tags' primary key is(track_id, key, ordinal); it does not discriminate onvalue_blob, so a track's text rows and binary rows occupy one ordinal space per key.ingestnumbered them independently — text rows from a per-key counter, binary rows from a single running index across the track — so a key present in both classes produced two rows at ordinal 0 and the ingest transaction failed withUNIQUE constraint failed: tags.track_id, tags.key, tags.ordinal. Unlike a cap violation (#644) this was not routed to a per-file failure, so it killed the whole scan — the reported case died 41% into a 891k-file library after an hour. Generalising that containment to any constraint violation is tracked separately in #662. The two classes now draw from one shared per-key counter, text first, which also makes binary ordinals per-key rather than track-wide. Reachable shapes: a FLACCUESHEETVorbis comment beside a CUESHEET metadata block, an MP3TXXXframe whose description names a binary frame the same tag carries (PRIV,GEOB,MCDI, a non-MusicBrainzUFID), and an MP4 freeform atom written with both a text and a binarydatabox. -
Scan log records and the progress bar no longer clobber each other on an interactive terminal (#648).
ScanReporterrenders anindicatifbar on stderr and thelogfacade writes to the same stderr, with nothing coordinating the two: a record was emitted at whatever column the last bar frame left the cursor on, and the next 120 ms tick issued a clear-line that ate part of it. Both the warning and the bar came out mangled. This was reachable on essentially the first interactive scan of any real library, because the end-of-walk skip tally (#341) warns aboutcover.jpg/.cue/.log/.nfosidecars, and every unparseable file warns from inside the pipeline. The CLI now owns a single process-wide stderr draw target: the scan bar draws through it, and the binary'senv_loggeris installed wrapped in a sink that emits each record while that target is suspended — bar cleared, record written, bar redrawn below it. The per-target summary line (scanned N: …, on stdout) is suspended the same way; it used to be glued onto a bar frame whenever no log record happened to precede it. Off a terminal the draw target is hidden and suspending is a no-op, so this change left the--quietand piped milestone paths byte-for-byte alone, as it did the verbosity policy (-v/-vv/-vvv,RUST_LOGtaking precedence), which stays in the binary. (The milestone line is renamed by the progress-bar convergence fix below, in this same release.) -
The scan progress indicator now reaches 100% when files fail (#655). The bar's length is the walked file count, but its position only advanced on a committed file, so any failure left it permanently short — a run with 12 unparseable files out of 42 finished at
30/42 (71%)and was then cleared, which reads as an aborted scan rather than a completed one about to reportfailed 12. On the piped path the final100%milestone was simply never printed, so a log-scraping consumer waited for a line that could not arrive. A dispatched file that fails or races now advances the same progress sequence as a committed one; the two together always account for the walked total, and a debug assertion pins that. The piped milestone line is renamedingested N/M (P%)→processed N/M (P%), because it now counts every file the pipeline finished with rather than only the successes — scripts matching the old prefix need updating. -
A tag larger than the store's cap no longer aborts the entire scan (#644). The scanner had no length check on text tags, so an over-cap value reached the DB
CHECKinside a batch commit and failed the whole run withCHECK constraint failed: length(CAST(value AS BLOB)) <= 262144— naming neither the offending file nor what the number meant. Every cap the scanner can trip is now checked in one place before anything is written, and a violation fails only that file, with a message naming it. The same applies to thetags.key,art.mimeandtrack_art.descriptioncaps, which had the identical unattributed-abort failure mode. Should a store write still fail fatally, the error now names the file it died on. -
A FLAC whose tags outgrow what a
VORBIS_COMMENTblock can hold is rejected at scan time rather than stored and then servedEIOon every read. This is reachable by merging a leading ID3v2 tag's fields into a FLAC's own comments (#602): ID3v2's tag size is synchsafe 28-bit (256 MiB) while a FLAC metadata block is 24-bit. -
A leading ID3v2 tag on an MP3 (or a FLAC) is stepped over by its declared size even when its major version is not one musefs can parse the frames of, instead of the whole file being rejected. The ID3v2 header has the same shape in every version, so its size is enough to step over the tag, and the spec's rule for a version a reader does not understand is to ignore it. Such a tag's frames are still not read.
-
Over-cap
opendirdegrades to a stateless directory handle instead of replyingENFILE(#616). The 1024-handle cap was assumed to sit well above any real client, butbfs— an ordinary parallelfind, and the defaultfindin several distributions — exceeded it immediately: 7,525 rejections over a 200,000-track mount, 17,910 files never enumerated. The rejection surfaced to the operator as "Too many open files in system", which points at the kernel rather than the mount, and an indexer that logs and continues would simply present an incomplete library. Serving over-cap directories through the existing stateless fallback keeps listings complete and preserves the memory bound the cap was added for, at the cost of an O(N) rebuild perreaddir. -
Unmount helpers are resolved against
/usr/bin,/binand/usr/local/binbefore falling back to a bare-namePATHlookup (#620). The mounting guide steers operators toward running as root for kernel passthrough inStructureOnlymode, and nothing sanitizesPATHthere, so a writablePATHentry meant attacker-chosen code executed as root onSIGTERM. -
A failed batch commit during a scan winds the pipeline down before the error propagates (#618).
ByteBudgetgained aclose()that wakes waiters, so a worker parked inacquireon a condvar onlyflushever signalled is no longer stranded. This was benign for the CLI, where process exit reaps everything, butscan_directory_with/revalidate_withare public API and an embedder that caught the error accumulated leaked threads and their in-flight art bytes. The two state-mutexunwrap()s adopted the daemon'slock_recoverpolicy at the same time, retiring the open note inlock.rs. -
find_page_startbounds the number of candidate pages it CRC-validates per call (#619). The header pre-filter almost never admits a falseOggSon real audio, but a file whose audio region is deliberately packed withOggS\x00\x00cleared it at every offset, allowing up to ~65,000 CRC validations — each with its own positioned reads — for a single seeking read. Hardening rather than a live vulnerability: the attacker is whoever can place a file in the scanned library. -
accessis implemented and repliesok(#624), so fuser's default no longer logs[Not Implemented]per mount. The mount carriesROand, withallow_other,DefaultPermissions, so the kernel already enforces the presented mode bits.
Internal
- The
ogg_pagefuzz target round-trips the page machinery the serve path actually depends on —verify_page_crcandpatch_page_header_algebraic— instead of only decoding a header (#625). Coverage against the committed seed rose from 51 edges / 69 features to 129 / 267. - The read-ahead budget invariant restored by #536 now has a concurrent
regression test that the ASan and TSan CI legs actually reach
(#628). The sanitizer legs previously ran a test that builds
ReadAheadPool::new(0), leaving the pool disabled throughout. No live bug was found; this closes the coverage gap. musefs-core/tests/tree_footprint.rsgates the virtual tree's per-track resident cost (#629), turning #617's throwaway probe into a committed ceiling. It samplesVmRSSeither side ofVirtualTree::build_withwith the rendered paths materialized beforehand, so the delta is the tree's own marginal cost. The ceiling is deliberately loose — a gate that catches only a large regression is worth more than one that reddens on a loaded runner — and a floor assertion keeps a broken measurement from passing vacuously.deny.toml's allow and ignore lists can no longer rot (#622). A staleRUSTSEC-2025-0167ignore and an unmatchedISClicense allowance both warned and exited 0, so neither surfaced on a PR — which is how an entry ends up silently pre-exempting the next real advisory for the same crate. Both were dropped, andadvisory-not-detected/license-not-encounteredare now errors in thedenyjob. The promotion is scoped to the root graph, which is what the lists are authored against; the fuzz-lockfile scan inaudit.ymlallows the advisory diagnostic explicitly, since off that graph it is noise.- Issue, pull-request and
CODEOWNERStemplates (#627). The PR checklist covers the steps that are easy to forget and silently break something: the pre-commit hook,cargo +nightly fuzz buildafter a format-layer API change, regenerating the Python schema mirror after amusefs-dbchange, and a changelog entry.
1.3.0 - 2026-08-19
Added
- FLAC files carrying one or more ID3v2 tags in front of the
fLaCmarker are now scanned instead of being skipped with "no parseable audio metadata" (#602). The tag run is stepped over to reach the FLAC stream, and its text frames andAPICpictures are ingested as a fallback beneath the file's ownVORBIS_COMMENT/PICTUREblocks — so a FLAC whose tags live only in the ID3 header lands in the store with its tags. A trailing 128-byte ID3v1 tag is trimmed from the audio length (checked only for files with a leading ID3v2 tag, so a stock FLAC pays no extra read). Neither tag survives into the synthesized file, which is a stock FLAC starting atfLaC.
Changed
musefs-corenow builds its persistent virtual-tree collections onimbl7 instead of the archivedim15. Clears RUSTSEC-2026-0248 / RUSTSEC-2023-0126 (im) and RUSTSEC-2026-0251 / RUSTSEC-2026-0255 (sized-chunks).VirtualTree::childrennow yields an opaqueimpl ExactSizeIterator<Item = (&str, u64)>instead of borrowing the backingOrdMap, taking the persistent-collection crate out ofmusefs-core's public API so swapping it stays an internal detail. In-tree the only caller isreaddir, which iterates; name lookups have always hadVirtualTree::lookup.
Fixed
- Bumped
crossbeam-epoch0.9.18 -> 0.9.20 (RUSTSEC-2026-0204, invalid pointer dereference in thefmt::Pointerimpls) andnum-bigint0.4.7 -> 0.4.8 (0.4.7 was yanked). Both are dev-dependency-only paths (criterion -> rayon, mp4 -> num-rational).
1.2.0 - 2026-06-18
Changed
- Bare
scanis now additive: it skips rows already in the DB instead of re-seeding them from disk. Usescan --forcewhen you want the old full-reimport behavior. revalidateis now its own subcommand and no longer prunes by default. It refreshes changed rows' structural data while preserving curated tags/art; userevalidate --pruneto drop rows whose backing file is gone and garbage-collect orphaned art.
Deprecated
scan --revalidateis a deprecated, warned alias forrevalidateand will be removed next release. It does not prune; userevalidate --prunewhen you need deletion.
Fixed
- Revalidating a changed file no longer clobbers curated tags, art, or binary tags in the DB.
1.1.0 - 2026-06-17
Added
- Runtime telemetry (
.musefs-metrics): an opt-in--expose-metricsflag (envMUSEFS_EXPOSE_METRICS) surfaces a synthetic.musefs-metricsfile at the mount root rendering Prometheus-format counters — getattr/read/open activity, backing read-ahead behavior, and (when built with jemalloc) allocator stats. Off by default; the file is absent unless enabled. See the README Metrics section (#394). - Scan progress indicator:
scanandscan --revalidaterender a live progress bar (indicatif) with an elapsed-time summary on an interactive terminal, falling back to periodicingested N/M (P%)log lines when output is non-interactive. A new--quiet/-qflag suppresses it (#406). --skip-on-missingtemplate flag: an opt-in--skip-on-missing(envMUSEFS_SKIP_ON_MISSING) drops a track from the mount when a top-level template field stays unresolved, instead of substituting--default-fallback. Per-field--fallbackchains and[...]optional sections are unaffected (a field resolved via its fallback counts as present). The motivating case is--template '$!{beets_path}' --skip-on-missing, which hides tracks beets left without abeets_pathrather than collapsing them into anUnknownbucket (#408).--read-ahead-prefetchflag: opt-in background prefetch threads layered on top of read amplification, default off — benchmarks found amplification alone delivers the entire read-ahead win, while the threads add ~10% overhead with no measured benefit. Enable only when profiling a backend where a single large read does not self-pipeline (#255).- riscv64 release platform: prebuilt
riscv64gc-unknown-linux-{gnu,musl}binaries andlinux/riscv64Docker images now ship with each tagged release. Container bases bumped to current stable: glibc Debian bookworm → trixie (bookworm has no riscv64 image), musl Alpine 3.20 → 3.23 (3.20 is end-of-life). statfsreply: the mount now reports a non-zero synthetic capacity with ample free space instead of fuser's all-zero default, sodfno longer shows a 0-byte filesystem and capacity-checking importers (Lidarr et al.) don't balk (#368).- Per-extension skip breakdown: at end of scan, a summary line breaks the
skippedcount down by lowercased extension (e.g.skipped 42: jpg=20, cue=10, log=8, <none>=4), logged atwarnso it shows by default, so a large skip count is diagnosable — expected sidecars versus genuinely unexpected files. Log-only; theScanStatsstruct and CLI summary are unchanged (#341). musefs vacuumcommand: compact the SQLite store, reclaiming free pages left by prunes, orphan-art GC, and the schema migration. RunsVACUUM+ a WAL checkpoint and reports the space reclaimed; run it while unmounted (#566).
Fixed
- Art/serve rowid-reuse consistency: the read fast path's WAL-snapshot +
content_versionguard, previously gated only on binary-tag layouts, now covers all DB-rowid segments (artArtImage/OggArtSlicetoo) viaRegionLayout::streams_db_rowid, and the stateless no-fh read fallback now applies the same snapshot/recheck and re-validates its freshly opened backing fd against the resolved stamp. A concurrent external retag +gc_orphan_art+ reinsert can no longer splice a wrong image or stale tag bytes mid-read (the audio-bytes invariant was never affected) (#502, #503). - Per-field
--fallbackcase-insensitivity: fallback keys are now ASCII lowercased to match template field names, so--fallback AlbumArtist=…(any uppercase) is honored instead of silently never matching (#504). - Tag value byte cap: both the schema
CHECK(rebuilt in theMIGRATION_V2upgrade) and the read-timetags.valueguard now count bytes, not UTF-8 characters, so the 256 KiB materialized-memory bound is exact rather than up to ~4x looser for multibyte text. The upgrade drops any pre-existing over-cap rows (already unreadable under the byte-counting reader guard) (#505). - Embedded NUL in ID3 metadata: synthesized ID3 frames now reject a DB-sourced tag key, tag value, art mime, or art description containing an embedded NUL instead of emitting a frame a downstream parser would misread (#506).
- Orphan-art GC NULL safety:
gc_orphan_artusesNOT EXISTSrather thanNOT IN (subquery), so a NULLart_idcould not silently turn the GC into a no-op (#507). - Mount usability:
mountnow warns when the mountpoint is non-empty (its contents are shadowed for the mount's lifetime), and a permission-denied mount (e.g. an AppArmor-restricted prefix) prints actionable guidance instead of a bare "Permission denied" (#508, #509). - Silent mp4 oversize drops: oversized embedded
covrcover art and binary freeform (----) values in.m4a/.m4bfiles are skipped in the format layer before materialization (to avoid building a large image out of a largemoov), which previously dropped them with nothing in the logs. The scan now emits awarnline for each, matching the logging the other formats already had (#343, follow-up to #284). - xattr log noise:
getxattr/listxattr/setxattr/removexattrnow replyENOTSUPexplicitly (read-only filesystem, no extended attributes) instead of falling through to fuser's default, which logged a[Not Implemented]warn on every xattr probe (ls -l, indexers, backup tools). The caller-visible result is unchanged (#364). - MP4 path-to-
ilstleniency: the walk tomoov/udta/meta/ilstnow uses the same lenient box scan as the metadata extractors, so a single malformed or truncated sibling box anywhere on the path no longer suppresses an otherwise well-formedilstand silently drops every tag and cover. The audio/structure path stays strict (#542). - QuickTime bare
metaatoms: themetaparser only consumes the 4-byte FullBox version/flags prefix when it is actually present (a zero word), so a QuickTime-style baremeta— which has no such prefix — is read instead of landing mid-header and dropping all tags and art (#543). scanexit code on ingest failure:scan/scan --revalidatenow exit2when any file fails to parse/ingest (failed > 0), instead of always exiting0. A pipeline such asmusefs scan … && musefs mount …can now detect a partial or total ingest failure; a clean scan still exits0and a hard error still exits1(#554).- Release smoke audio-bytes check:
scripts/smoke-binary.sh(the per-arch release gate) now compares the served file's encoded audio stream against the untouched backing file, asserting the cardinal byte-identical-audio invariant rather than only checking thefLaCmagic — so a target-specific positioned-read or offset regression in a cross-compiled binary is caught (#547).
1.0.0 - 2026-06-12
First stable release.
Added
- Lidarr integration: a new
contrib/lidarr/package that drives symlink-based placeholder imports and syncs Lidarr metadata into the musefs SQLite store. - FUSE mount-access controls: new
--allow-other,--owner, and--groupflags mount withallow_other+default_permissionsso accounts other than the mounting user can reach the view and the presented owner/group/mode bits are enforced;--owner/--groupimply--allow-other. A non-rootallow_othermount is pre-flight checked against/etc/fuse.confuser_allow_otherand fails early with guidance if it is missing. See the README Ownership and permissions section (#293, #294). - Hardened deployment assets: the container image runs as a dedicated
unprivileged user with a build-arg-configurable UID/GID, and the
musefs-scan.servicesystemd unit ships a strong sandbox (the FUSE-mountingmusefs.servicedeliberately cannot be sandboxed). See the systemd hardening notes (#317, #318, #319). - crates.io distribution: the
musefsbinary is published to crates.io as of this release and installable withcargo install musefs. A new thinmusefswrapper crate owns the binary (musefs-cliis now a library crate), and a tag-triggered release workflow publishes all crates in dependency order. - Fuzzing & property tests: coverage-guided
cargo-fuzztargets for every format parser (FLAC, MP3, MP4, Ogg, WAV), the byte-level primitives (Ogg page parsing, base64 windowing, VorbisComment), and the serve path — the latter drives the full synthesis pipeline over hostile DB rows and binary tags via a fuzzing-gatedDb::with_raw_conn. Plusproptestinvariants — panic-freedom, the byte-identical audio guarantee, and tag round-trip — an end-to-end read-fidelity property, and amutageninterop test asserting an independent reader sees the tags we synthesize.
Changed
mount --dbnow requires an existing store. Mounting against a missing database path is rejected before any FUSE setup instead of silently creating and migrating an empty store, so a mistyped--dbfails loudly rather than mounting an empty view.scan --dbstill creates the store if absent (#309).
Fixed
- Scanner no longer drops files and embedded art silently: embedded cover
art over
MAX_ART_BYTES(and binary tags overMAX_BINARY_TAG_BYTES) were filtered out at ingest with no log line, so a track whose art exceeded the cap appeared to simply have none — indistinguishable from a scan bug. The drop is now logged (RUST_LOG=warn). Likewise, a supported-extension file that fails to parse or errors mid-probe was countedfailedwith the underlying error discarded; the reason is now logged. Note: oversized art in.m4a/.m4bfiles is dropped earlier, inside the format layer, and is not yet logged (#284, #343). - Lidarr custom-script env var casing: Lidarr stores custom-script
environment variables in a .NET
StringDictionary, which lowercases every key, so a Linux script actually receiveslidarr_sourcepath/lidarr_eventtyperather than the PascalCase names Lidarr's docs list. The integration read the PascalCase names, so with a real Lidarr every import failed and every event parsed as unsupported. Lidarr env vars are now resolved case-insensitively. Found by the issue #141 real-instance smoke run. - VorbisComment parse OOM (DoS): a crafted comment block declaring a huge
entry count made
Vec::with_capacityattempt a multi-gigabyte allocation; the pre-allocation is now bounded by the readable byte count. Found by the newvorbiscommentfuzz target. - MP4 box-bounds integer overflow: an untrusted 64-bit extended box size made
the box-bounds check (
pos + total) overflowusize— a panic in debug and a silent wrap in release that accepted a bogus box length. The addition is now checked. Found by themp4fuzz target. - ID3v2 parsing unbounded allocation (DoS): the
id3crate eagerly allocates a frame's declared size (ID3v2.3 frame sizes are plain 32-bit, up to 4 GiB), so a crafted tag could exhaust memory at scan time — via an MP3 or a WAV embeddedid3chunk. Parsing is now gated on validated ID3v2 frame bounds and an ID3v2 tag at offset 0 (theid3reader scans forward). Found by themp3andwavfuzz targets. - Scan counters now match their documented contract:
musefs scanreports every non-audio file (any unsupported or missing extension —.jpg,.cue,.log,.nfo, cover art, etc.) asskipped, and supported-extension files that fail to parse (e.g. a corrupt.flac) asfailed. Previously malformed files were miscounted asskippedand unsupported files were not counted at all, so expectskippedto be larger than before on a real library (#301). - Symlink scans no longer double-count: with
--follow-symlinks, a file reached via both its real path and a symlink is ingested and counted once instead of inflatingscanned; multiple hardlinks to the same inode are likewise collapsed to a single track (#302). - Stable inodes on case-insensitive mounts: the inode allocator is now keyed on the case-folded path in case-insensitive mode, so an unrelated deletion that flips a merged directory's display casing no longer reassigns a survivor's inode (#305).
- Lidarr autoscan now honors the scan timeout: an import/release-triggered
autoscan applies the shared 120s scan timeout, matching the beets and Picard
integrations, so a wedged
musefs scanfails with a controlled timeout instead of blocking the custom-script process indefinitely (#312).
0.2.0 - 2026-05-27
First public release.
Added
- Formats: synthesis for M4A/M4B (MP4), Ogg (Opus, Vorbis, FLAC-in-Ogg), and WAV, alongside the existing FLAC and MP3 — metadata generated on the fly from the SQLite store and spliced in front of byte-identical backing audio.
- Arbitrary tag support: a single canonical tag vocabulary maps common fields
to each format's native slot (ID3 frame / MP4 atom / Vorbis field); any other
tag round-trips through the format's extension slot (ID3
TXXX, MP4----freeform, raw Vorbis field). User-defined key casing is preserved. - beets plugin (
contrib/beets/): syncs beets' canonical tags and cover art into the store keyed by each file's real path, with no remount and no audio rewrite. - Performance, concurrency & caching pass: worker-pool offload of blocking
reads, lock-free virtual-tree swap, per-handle I/O, a bounded LRU header-layout
cache, debounced single-flighted refresh with stable inodes, kernel/mount
tuning flags, bounded-memory MP4 resolves, and opt-in
--keep-cachewith auto-invalidation.
Notes
- Read-only mount; tag edits happen out-of-band against the SQLite store and are
picked up automatically (
PRAGMA data_versionpolling). See the README Supported formats section and the per-format docs for round-trip limitations.
0.1.0
- Initial MVP (FLAC and MP3 synthesis, virtual tree with beets-style templates,
synthesis/structure-onlymount modes, auto-refresh,scan/scan --revalidate). Never published publicly; superseded by 0.2.0.