Performance

CHDSharp is designed for both batch verification throughput (parallel) and low-latency random access (single-threaded, cached). This page documents the knobs and what to expect.

Typical throughput

Measured on a modern desktop with the bundled corpus and typical game images (single-threaded I/O, Release build):

Actual numbers depend on the mix of codecs (LZMA is the slowest, raw/zstd the fastest), hunk size, and storage speed.

Tuning parallelism

Chd.TaskCount controls the worker count for CheckFile(deepCheck: true) (default 8, valid range 1–64). It is a process-global setting — set it before calling:

Chd.TaskCount = Environment.ProcessorCount;   // or a value you benchmarked
var result = Chd.CheckFile(stream, "game.chd", deepCheck: true);

Memory is bounded even with many workers:

• pooled buffers (ArrayPool) for compressed input, decompressed output, and caches;
• a semaphore capping in-flight repeat-block cache to 512 MiB (blocksToKeep = (512 MiB) / hunkbytes).

Caching model

Repeat-block cache (verification)

ChdBlockRead detects hunks referenced multiple times (deduplicated SELF entries) and:

1. computes a usage weight per block (LZMA/Huffman/CD-LZMA blocks are the most expensive to re-decompress, so they get the highest priority),
2. keeps the top-N blocks’ decompressed copies within the 512 MiB budget (KeepMostRepeatedBlocks),
3. serves repeated hunks from the cache instead of re-decompressing.

Last-hunk cache (random access)

ChdFile.Read() caches the most recently read hunk, so byte ranges that stay within one hunk cost a single decompression.

Multi-hunk LRU cache (libchdr #36)

ChdFile.ReadHunk() retains decompressed hunks in a configurable LRU cache so random reads that revisit hunks avoid re-decompression. Default size is 1 (equivalent to the single-hunk slot); set CacheSize or call ConfigureCache(n) to keep the last n distinct hunks (n <= 1 disables it). Memory is capped at CacheSize * HunkBytes.

chd.ConfigureCache(16);        // keep 16 decompressed hunks
chd.ReadHunk(100, buf);        // decompressed
chd.ReadHunk(100, buf);        // served from cache

Use a larger CacheSize when a workload performs random/scattered reads that repeatedly touch the same subset of hunks.

Precache (whole file in RAM)

ChdFile.Precache() reads the entire compressed file into memory once. Every subsequent hunk read is served from RAM — no stream seeks, no disk I/O. Ideal for:

• random-access workloads over slow/remote storage,
• many small reads across a large image,
• repeated passes over the same file.

var err = chd.Precache();
if (err != ChdError.Chderrnone) { /* >2 GiB file → out of memory; IO failure */ }

// from here on, ReadHunk/Read serve from RAM

Precache() is idempotent, restores the stream position, and returns Chderroutofmemory for files larger than 2 GiB or when allocation fails.

Reusable codec state

ChdCodecState keeps per-codec scratch alive across hunks:

• LZMA dictionary window (sized to the hunk),
• zstd decompressor instance,
• FLAC decoder + audio buffers,
• Huffman lookup tables (1 MiB each for AVHuff contexts).

This avoids reallocating the most expensive buffers on every hunk — the main reason sequential Read stays in the hundreds of MB/s.

Memory profile

Practical tips

• Batch-verify in parallel with CheckFile, not ReadAllBytes per file.
• Extract with ExtractToDirectory (streams hunk-by-hunk) rather than ReadAllBytes for large images.
• Random access patterns benefit from Precache when the underlying stream is slow; on local NVMe the difference is usually small.
• Hunk size matters: larger hunks amortize per-hunk overhead but waste space on small reads. This is a property of the CHD file itself (set by chdman), not the library.
• For sequential streaming, prefer EnumerateHunks() (single buffer, no per-call allocation) over repeated Read calls.