blocks: add resumable reobfuscation for existing block files #33324

l0rinc commented at 11:51 PM on September 5, 2025: contributor

Problem: Nodes that synced before v28 may store block and undo files effectively unobfuscated with a zero XOR key. There is no built-in way to obfuscate those existing files, rotate their key, or deliberately replace it with a zero key without resyncing.

Fix: Add -reobfuscate-blocks[=<value>] to rewrite existing blk*.dat and rev*.dat files before chainstate loading. With no value or =1, the option generates a random 8-byte key. A 16-character hexadecimal value selects an exact key, including all zeros to remove obfuscation, while malformed values fail during startup. If the selected key is already active, block and undo files are left unchanged.

Files are rewritten in fixed-size chunks, staged with a .reobfuscated suffix, committed before the originals are removed, and renamed to their original names after all rewrites finish. The target key is staged as xor.dat.reobfuscated, which is also used to resume an interrupted migration. File timestamps are preserved, and progress is reported in the log and through Kernel notifications marked as resumable.

Benchmark: The migration is single-threaded and processes one file at a time.

CPU	Storage	Block count	Size	Files	Time (min)	Blocks/min
Apple M4 Max laptop	SSD	~955k	~854 GB	11,232	10.18	93,804
Intel Core i9	SSD	~909k	~725 GB	10,238	23.1	39,351
Raspberry Pi 5	SSD	~914k	~728 GB	10,276	72.78	12,558
Intel Core i7	HDD	~909k	~720 GB	10,156	208.7	4,356
Raspberry Pi 4B	HDD	~915k	~730 GB	10,304	1467	624

Similar work:

DrahtBot commented at 11:51 PM on September 5, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33324.

Reviews

See the guideline and AI policy for information on the review process.

Type	Reviewers
Concept ACK	stickies-v
Stale ACK	sedited

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#35531 (txindex: hash keys and pack positions to reduce disk usage by andrewtoth)
#35139 (test: Add thread-safe fast-failing test macros by maflcko)
#34132 (coins: drop error catcher, centralize fatal read handling by l0rinc)
#33854 (fix assumevalid is ignored during reindex by Eunovo)
#33637 (refactor: optimize block index comparisons (1.4-6.8x faster) by l0rinc)
#30342 (kernel, logging: Pass Logger instances to kernel objects by ryanofsky)
#24230 (indexes: Stop using node internal types and locking cs_main, improve sync logic by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

l0rinc force-pushed on Sep 6, 2025

l0rinc force-pushed on Sep 7, 2025

l0rinc force-pushed on Sep 8, 2025

sedited commented at 8:21 AM on September 8, 2025: contributor

Concept ACK

stickies-v commented at 10:06 PM on September 8, 2025: contributor

Concept ACK, this seems like useful functionality to expose.

Should we split ObfuscateBlocks out of init? I have split it into many local lambdas, but we may want to find better home for those methods...

I don't like using startup options for one-time operations (I feel the same about e.g. -reindex). Without having thought it through too much yet, maybe we can bundle this e.g. as part of bitcoin-util or a separate bitcoin-xor-blocks utility?

Should we repurpose the existing -blocksxor arg instead?

With this PR, IIUC we'd have -blocksxor, reobfuscate-blocks, and the existence of the xor.dat file that all have some redundancy and thus potential for conflict (e,g. blocksxor=0, reobfuscate-blocks=1, and a non-zero xor.dat file). Reducing that complexity seems like it would be useful.

DrahtBot added the label Needs rebase on Sep 9, 2025

l0rinc force-pushed on Sep 10, 2025

DrahtBot removed the label Needs rebase on Sep 10, 2025

in src/init.cpp:1366 in aa587f3740 outdated

1361 | +    std::vector<std::byte> buf;
1362 | +    buf.resize(node::MAX_BLOCKFILE_SIZE);
1363 | +
1364 | +    // Migrate undo and block files atomically
1365 | +    for (const auto& [name, files] : {std::make_pair("undo", collect_undo_files()),
1366 | +                                      std::make_pair("block", collect_block_files())}) {

ajtowns commented at 5:10 AM on September 11, 2025:

I was surprised when it hit 100% of the undo files then restarted on the block files and was much slower -- might be better to do the slow files first, or ideally to do block and undo files intermixed so you just have a single 0% to 100% run.

l0rinc commented at 3:56 AM on September 12, 2025:

I had that version before, but didn't like that the small and big files made the percentages look unevenly spaced. But I have reverted that version and shuffled the files, this should make the progress feel more uniform - thank you for the observation!

in src/init.cpp:1351 in aa587f3740 outdated

1346 | +        old_blocks.read(buf);
1347 | +
1348 | +        AutoFile new_blocks{fsbridge::fopen(file + suffix, "wb")};
1349 | +        new_blocks.write_buffer(buf);
1350 | +
1351 | +        if (old_blocks.fclose() || !new_blocks.Commit() || new_blocks.fclose()) return false;

ajtowns commented at 5:22 AM on September 11, 2025:

Would be good to try to reset the timestamp of new_blocks to match that of old_blocks here.

        // attempt to preserve timestamp
        fs::last_write_time(file + suffix, fs::last_write_time(file));

l0rinc commented at 3:56 AM on September 12, 2025:

Done, thanks!

in src/init.cpp:1346 in aa587f3740 outdated

1341 | +    }};
1342 | +
1343 | +    auto migrate_single_blockfile{[&](const fs::path& file, const Obfuscation& delta_obfuscation, std::vector<std::byte>& buf) -> bool {
1344 | +        AutoFile old_blocks{fsbridge::fopen(file, "rb"), delta_obfuscation}; // deobfuscate & reobfuscate with a single combined key
1345 | +        buf.resize(fs::file_size(file)); // reuse buffer
1346 | +        old_blocks.read(buf);

ajtowns commented at 5:23 AM on September 11, 2025:

Rather than reading the entire blockfile into memory at once, consider chunking it:

        size_t left = fs::file_size(file);
        while (left > 0) {
            size_t chunk = std::min<size_t>(left, 2 * MAX_BLOCK_SERIALIZED_SIZE);
            buf.resize(chunk);
            old_blocks.read(buf);
            new_blocks.write_buffer(buf);
            left -= chunk;
        }

l0rinc commented at 5:49 AM on September 11, 2025:

We could do that with the recently introduced buffered readers - but that's considerably slower. Is it a problem to read all of it in memory when we don't even have dbcache yet? The total memory usage is just 160 MB during migration, we should be fine until 1 GB at least, right?

ajtowns commented at 6:20 AM on September 11, 2025:

I don't think buffered readers is the right thing (that's for when you want to process small amounts of data while still reading it from the file in large chunks), and trying the above didn't seem particularly slow to me.

I guess it could be simplified a bit to:

buf.resize(2 * MAX_BLOCK_SERIALIZED_SIZE);
while (true) {
    size_t size = old_blocks.detail_fread(buf);
    if (size == 0) break;
    new_blocks.write_buffer(std::span(buf, 0, size));
}

l0rinc commented at 3:58 AM on September 12, 2025:

What's problem would chunking solve in your opinion? I don't mind doing it, but the current version is slightly simpler and slightly faster, so I need at least some justification for giving up both :)

ajtowns commented at 12:36 PM on September 12, 2025:

Loading a large file entirely into memory when it's not necessary is just bad practice. What if we changed to .blk files of 1GB each? What if we're running on a node that's memory constrained and configured dbcache down to 4MB?

If we're worried about speed, then doing it in parallel helps on my system since obfuscation ends up CPU bound when single-threaded -- with the current code, it takes 238s (4min); with 8 threads it's 65s; with 16 threads it's 47s. Using 16MB chunks (BLOCKFILE_CHUNK_SIZE), 8 threads is also ~128MB of memory, but a user on a severely memory constrained system could reduce the thread count if they wanted. Here's roughly what I'm thinking: https://github.com/ajtowns/bitcoin/commits/202509-reobfus/

l0rinc commented at 1:48 PM on September 12, 2025:

Loading a large file

This is run on request, before anything else loads, it's not that large, only 160 MB memory is needed.

For reference, applying the mentioned dbcache=4 (which isn't used here yet) still makes the node use > 1 GB memory: <img width="1202" height="631" alt="image" src="https://github.com/user-attachments/assets/1867ba38-ba35-4ce5-86de-5bfe8add145b" />

Edit: doing an actual massif memory measurement with dbcache=4 and -blocksonly reveals that the actual memory usage is lower than that (but still higher than the 160MB needed for a single blockfile):

Command:            ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=200000 -dbcache=4 -blocksonly -printtoconsole=1                                               
Massif arguments:   --time-unit=ms --massif-out-file=/mnt/my_storage/logs/massif-e66f04d0131b8c2db13ddd649e9eb20910eb6d1d-200000-4.out                                                    
ms_print arguments: /mnt/my_storage/logs/massif-e66f04d0131b8c2db13ddd649e9eb20910eb6d1d-200000-4.out                                                                                     
--------------------------------------------------------------------------------                                                                                                          
                                                                                                                                                                                          
                                                                                                                                                                                          
    MB                                                                                                                                                                                    
383.1^#                                                                                                                                                                                   
     |#                                                                                                                                                                                   
     |#                                                                                                                                                                                   
     |#                                                                                                                                                                                   
     |#                                                                                                                                                                                   
     |#                                                                                                                                                                                   
     |#                                   :  : :::      :  :    ::::  @                                                                                                                   
     |#   :@: :::@::::::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#::::@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::                                                                                                            
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::
     |#:: :@::: :@::: ::::::::::@:::@::::::::::::::::::@:::::::@::::::@:::::::
   0 +----------------------------------------------------------------------->h
     0                                                                   2.506

Here's roughly what I'm thinking: ajtowns/bitcoin@202509-reobfus (commits)

Multithreading is indeed a very good argument for chunking, thanks a lot for the patch, I'll apply it soon and add you as coauthor!

l0rinc commented at 7:02 AM on September 20, 2025:

Thanks again for the review, I have pushed a change to fix the CI and took a few suggestion from your branch (chunking, code simplifications), but kept the original file iteration with progress indicator for now. The parallelization complicates the situation considerably, I will see if I can find a simpler way or if single-threaded execution is also acceptable. Edit: grouped the block and undo files for more uniform iteration instead of shuffling

l0rinc commented at 3:33 AM on September 26, 2025:

I have pushed a new version (rebased, extended test), let me know what you think.

I have implemented a very simple multithreaded version but I couldn't convince it to achieve any speedup whatsoever - I guess xor operations are a lot cheaper than disk reads/writes. The total CPU usage was at 20% even with 50 threads.

I have pushed my threaded solution to https://github.com/l0rinc/bitcoin/pull/40/files#diff-b1e19192258d83199d8adaa5ac31f067af98f63554bfdd679bd8e8073815e69dR1361-R1379, but I kept the single-threaded version here.

in src/init.cpp:1295 in aa587f3740 outdated

1290 | +        return files;
1291 | +    }};
1292 | +    auto collect_block_files{[&]() -> std::set<fs::path> {
1293 | +        std::set<fs::path> files;
1294 | +        while (true) {
1295 | +            if (auto f{m_blockman.GetBlockPosFilename(FlatFilePos(files.size(), 0))}; fs::exists(f)) {

ajtowns commented at 5:43 AM on September 11, 2025:

This doesn't seem like it would work correctly with pruning, when blk0000.dat has been deleted?

l0rinc commented at 3:58 AM on September 12, 2025:

Good call, changed it back to regex matching

ajtowns commented at 5:46 AM on September 11, 2025: contributor

Having it be a startup option like -reindex seems fine to me.

l0rinc force-pushed on Sep 12, 2025

ajtowns commented at 7:32 AM on September 15, 2025: contributor

tidy wants emplace_back over push_back

luke-jr commented at 1:29 AM on September 18, 2025: member

I agree a separate utility for this seems better - this requires very little of the existing codebase, in theory.

Also suggest making the files with the same names, but in a new directory, and then atomically rename the directory when complete, rather than every single file.

l0rinc commented at 1:49 AM on September 18, 2025: contributor

I agree a separate utility for this seems better

Can you quote what you're agreeing with specifically, not sure who suggested that. Besides, @andrewtoth already has a tool for that, it was mentioned in the PR description.

but in a new directory, and then atomically rename

I will think about it, could make sense, but in that case unrelated files should also be copied over (maybe duplicated to be safe) - and listing the directory content wouldn't make the progress obvious. What's wrong with the current approach?

l0rinc force-pushed on Sep 20, 2025

l0rinc force-pushed on Sep 22, 2025

l0rinc force-pushed on Sep 26, 2025

l0rinc force-pushed on Oct 3, 2025

l0rinc commented at 1:01 AM on October 3, 2025: contributor

Added kernel notifications (thanks @ryanofsky) and improved crash resistance at the very last step (final rename back to old names) - try it out with ./build/bin/bitcoin-qt -reobfuscate-blocks -stopatheight=1. <img width=500 src="https://github.com/user-attachments/assets/95b72e06-63fe-4057-989d-24ab9e93bfac" />

l0rinc marked this as ready for review on Oct 3, 2025

l0rinc renamed this:
~~RFC: blocks: add `-reobfuscate-blocks` arg to xor existing blk/rev on startup~~
blocks: add `-reobfuscate-blocks` arg to xor existing blk/rev on startup
on Oct 3, 2025

l0rinc force-pushed on Dec 10, 2025

l0rinc renamed this:
~~blocks: add `-reobfuscate-blocks` arg to xor existing blk/rev on startup~~
blocks: add `-reobfuscate-blocks` argument to enable (de)obfuscating existing blocks
on Dec 11, 2025

l0rinc force-pushed on Jan 25, 2026

DrahtBot added the label CI failed on Jan 25, 2026

DrahtBot commented at 7:07 PM on January 25, 2026: contributor

🚧 At least one of the CI tasks failed. Task lint: https://github.com/bitcoin/bitcoin/actions/runs/21336964680/job/61410315485 LLM reason (✨ experimental): Lint check failed: direct use of std::filesystem flagged by the std_filesystem lint rule.

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

l0rinc force-pushed on Jan 25, 2026

DrahtBot removed the label CI failed on Jan 25, 2026

DrahtBot added the label Needs rebase on Feb 8, 2026

l0rinc force-pushed on Feb 8, 2026

DrahtBot removed the label Needs rebase on Feb 8, 2026

DrahtBot added the label Needs rebase on May 26, 2026

l0rinc force-pushed on Jun 8, 2026

DrahtBot removed the label Needs rebase on Jun 8, 2026

in src/node/blockstorage.cpp:1260 in 51a7b252e7

1257 | +    const Obfuscation result{obfuscation};
1258 | +    if (result) {
1259 | +        LogInfo("Using obfuscation key for blocksdir *.dat files (%s): '%s'\n", fs::PathToString(opts.blocks_dir), HexStr(obfuscation));
1260 | +    } else {
1261 | +        LogWarning("The obfuscation of the blocksdir *.dat files isn't active, restart with `-reobfuscate-blocks` option to start the obfuscation process. "
1262 | +                   "Note that this operation can take more than an hour on slow systems.");

sedited commented at 3:27 PM on June 11, 2026:

Not sure if a warning is really warranted here. Feel like this pull request would have gotten more review over the past year if this were a pressing operation.

l0rinc commented at 1:45 PM on June 18, 2026:

fair, done

in src/init.cpp:1394 in 6ab1eb624b

1389 | +            if (!node::ObfuscateBlocks(*g_shutdown, chainman_opts.notifications, block_obfuscation_suffix, blocks_dir, xor_dat, xor_new, requested_key)) {
1390 | +                if (ShutdownRequested(node)) return {ChainstateLoadStatus::INTERRUPTED, {}};
1391 | +                return {ChainstateLoadStatus::FAILURE_FATAL, _("Block obfuscation failed")};
1392 | +            }
1393 | +        }
1394 | +    }

sedited commented at 9:15 PM on June 11, 2026:

This function is already very busy. This logic also doesn't really need much state. The notifications are available from the node object and the blocks dir could just be re-read from the args. How about moving it to right before the cache size calculations in AppInitMain. It could also be moved into its little function.

l0rinc commented at 1:45 PM on June 18, 2026:

Was hoping someone will suggest that :)

in src/init.cpp:1380 in 43eb6ec1b1

1375 | +
1376 | +        std::optional<std::array<std::byte, Obfuscation::KEY_SIZE>> requested_key{};
1377 | +        bool reobfuscate_requested{false};
1378 | +        if (const auto arg{args.GetArg("-reobfuscate-blocks")}) {
1379 | +            if (arg->size() == 2 * Obfuscation::KEY_SIZE) {
1380 | +                if (!IsHex(*arg)) {

sedited commented at 9:25 PM on June 11, 2026:

It feels a bit off that getting the length wrong is not fatal. Maybe we should put these in same statement and have separate branches for the "0" and empty case?

l0rinc commented at 1:41 PM on June 18, 2026:

Even I got confused trying to explain why I did it as such - thanks, split, covered with test, added you as coauthor.

in src/node/blockstorage.cpp:1347 in 43eb6ec1b1 outdated

1342 | +        auto read_buffer{std::span{chunk}.first(chunk_size)};
1343 | +        for (size_t n{0}; (n = old_blocks.detail_fread(read_buffer)); ) {
1344 | +            new_blocks.write_buffer(std::span{chunk}.first(n));
1345 | +        }
1346 | +        if (!old_blocks.feof() || old_blocks.fclose() || !new_blocks.Commit() || new_blocks.fclose()) return false;
1347 | +        fs::last_write_time(staged_path, fs::last_write_time(entry.path)); // preserve timestamp

sedited commented at 9:27 PM on June 11, 2026:

I don't quite understand what the purpose of this is. Where does the timestamp become relevant?

l0rinc commented at 1:32 PM on June 18, 2026:

Was explicitly requested to minimize unintended changes #33324 (review)

in src/node/blockstorage.cpp:1360 in 6ab1eb624b

1356 | @@ -1356,6 +1357,7 @@ bool ObfuscateBlocks(
1357 |          auto files{CollectBlockAndUndoFiles(blocks_dir)};
1358 |          std::ranges::sort(files, {}, &BlockFileEntry::file_num);
1359 |          LogInfo("[obfuscate] Reobfuscating %s block and undo files", files.size());
1360 | +        notifications.progress(_("Reobfuscating blocks…"), 0, false);

sedited commented at 9:38 PM on June 11, 2026:

Could this be the first kernel notification where resume is possible?

l0rinc commented at 1:45 PM on June 18, 2026:

Good point, adjusted, added you as coauthor, thanks.

sedited commented at 9:50 PM on June 11, 2026: contributor

The migration logic looks nice.

I concur that this would be a good candidate for a separate c++ binary that also gets shipped in releases. However I'm not sure if this particular pull request is the practice case for ironing out how to launch such binaries from the GUI, how they would handle directory locks, and how they would communicate failure. The logic here is fairly straight forward. If somebody wants to take a stab at pioneering such a utility, that could easily be done later.

l0rinc force-pushed on Jun 18, 2026

l0rinc commented at 2:45 PM on June 18, 2026: contributor

Rebased and folded in the latest review feedback. I also re-ran the reobfuscation check before pushing and updated the PR description with the refreshed numbers for M4.

Main changes since the previous push:

Moved the startup reobfuscation handling out of InitAndLoadChainstate into ReobfuscateBlocksIfRequested(), called before cache sizing in AppInitMain.
Made -reobfuscate-blocks parsing stricter: only 0, empty/1, or exact 16-character hex keys are accepted; wrong-length keys now fail early and are covered by the functional test.
Downgraded the unobfuscated-blocks startup notice from warning to info.
Marked GUI reobfuscation progress as resumable and reused a single translated progress title.
Addressed the tidy/test nits (emplace_back, named progress args, assert_not_equal).

this would be a good candidate for a separate c++ binary that also gets shipped in releases @andrewtoth already has a separate tool for this, see #33324 (comment) @ajtowns, @stickies-v, @luke-jr, @ryanofsky care to take another look?

l0rinc force-pushed on Jun 18, 2026

DrahtBot added the label CI failed on Jun 18, 2026

DrahtBot commented at 3:59 PM on June 18, 2026: contributor

🚧 At least one of the CI tasks failed. Task test ancestor commits: https://github.com/bitcoin/bitcoin/actions/runs/27767629923/job/82158598757 LLM reason (✨ experimental): CI failed due to a C++ build error: init.cpp calls node::ObfuscateBlocks(...) with 7 arguments, but the function expects 6.

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

DrahtBot removed the label CI failed on Jun 18, 2026

sedited approved

sedited commented at 7:10 PM on June 18, 2026: contributor

ACK 181753328a8e27c7fac5300a44885843c3b6855f

@andrewtoth already has a separate tool for this, see #33324 (comment)

I saw and I looked at the pull request at the time, which is why I specifically mentioned a c++ tool. I think we should start using the bitcoin command more, and this would be a good candidate for it. As much as I like rust, I'm against adding released utilities written in yet another different language. With the code already here, it would probably be fairly straight forward to implement.

DrahtBot requested review from stickies-v on Jun 18, 2026

l0rinc requested review from ajtowns on Jul 2, 2026

DrahtBot added the label Needs rebase on Jul 12, 2026

l0rinc force-pushed on Jul 12, 2026

DrahtBot removed the label Needs rebase on Jul 12, 2026

l0rinc renamed this:
~~blocks: add `-reobfuscate-blocks` argument to enable (de)obfuscating existing blocks~~
blocks: add resumable reobfuscation for existing block files
on Jul 22, 2026

l0rinc force-pushed on Jul 22, 2026

l0rinc commented at 10:32 PM on July 22, 2026: contributor

Rebased and pushed again to fix a minor crash-safety bug and a -blocksxor=0 handling inconsistency - while simplifying the implementation a bit.

in src/node/blockstorage.cpp:1379 in 22790069f6

1376 | +                notifications.progress(title, percent, /*resume_possible=*/true);
1377 | +                last_percent = percent;
1378 | +            }
1379 | +        }
1380 | +        if (files.empty()) notifications.progress(title, /*progress_percent=*/100, /*resume_possible=*/true);
1381 | +        Assert(Remove(xor_dat));

sedited commented at 9:38 AM on July 23, 2026:

It does feel a bit weird to me to assert on what is not an invariant of the program, but rather an exceptional condition. Why not just throw or return here (and for the rename assertion). I think it would be preferable to just use the throwing fs::remove and fs::rename.

l0rinc commented at 7:37 AM on July 25, 2026:

Thanks, rebased and pushed: filesystem failures now propagate instead of aborting on assertions. I also did some cleanup, split the main feature commit into migration, startup wiring, and functional coverage, added Obfuscation::Key and made same-key reruns skip rewriting block and undo files. Reran the benchmark on my M4 Max - finished 955k blocks in ~10 minutes.

index: remove always-false obfuscation parameter

All `BaseIndex::DB` callers leave database obfuscation disabled, and the only explicit argument is `false`.
Remove the parameter and rely on the `DBParams` default.

9bc00c96eb

refactor: add `Obfuscation` key helpers

Name the fixed-size key type and add helpers for parsing keys and deriving the XOR delta between keys.
This keeps the block reobfuscation commit focused on file migration.

7dfa7c62f7

node: extract XOR key file helpers

Centralize `xor.dat` reading and creation so block reobfuscation can reuse the serialization and close-error handling.

33cb3e5b6d

node: extract block and undo file discovery

Extract `blk` and `rev` discovery from `CleanupBlockRevFiles` so block reobfuscation can share a directory scan that handles gaps left by pruning.

f2a0111367

node: log when block file XOR is inactive

Suggest `-reobfuscate-blocks` when the key is zero and block-file XOR is enabled.
Suppress the suggestion for `-blocksxor=0`, which explicitly leaves the files unobfuscated.

Co-authored-by: sedited <seb.kung@gmail.com>

b75127f90b

blocks: reobfuscate block and undo files

Rewrite existing block and undo files under a requested or random XOR key.
Stage each replacement and the target key before removing active files so interrupted migrations can resume safely.
Skip file rewrites when the target key is already active.

Co-authored-by: Anthony Towns <aj@erisian.com.au>

be49925393

init: add block file reobfuscation option

Add `-reobfuscate-blocks` and run the requested or pending migration before chainstate loading.
Accept a random or exact key, reject malformed values, and reject migration with `-blocksxor=0`.

Co-authored-by: sedited <seb.kung@gmail.com>

81743a520c

test: cover block file reobfuscation

Cover random and exact keys, same-key reruns, interrupted resume, invalid values, staged-key mismatches, and `-blocksxor=0`.

4df642a48b

gui: report block reobfuscation progress

Report block reobfuscation through `kernel::Notifications` so GUI users can follow the long-running startup operation.
Set `resume_possible` because a restart continues from `xor.dat.reobfuscated`.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: sedited <seb.kung@gmail.com>

d2bd095c19

l0rinc force-pushed on Jul 25, 2026