Block-header-only, faster startup client #7

issue gavinandresen opened this issue on December 20, 2010
  1. gavinandresen commented at 1:53 PM on December 20, 2010: contributor

    Bitcoin newbies have to endure an hour or two (or more) while bitcoin downloads and indexes all transactions and blocks.

    Satoshi has mostly implemented code that downloads just the block headers; as long as you're not generating blocks, you don't need all the old transactions.

  2. gavinandresen commented at 5:44 PM on December 20, 2010: contributor

    See the blockheaders feature branch here for initial work on this. Notes from Satoshi:

    CBlockIndex contains all the information of the block header, so to operate with headers only, I just maintain the CBlockIndex structure as usual. The nFile/nBlockPos are null, since the full block is not recorded on disk.

    The code to gracefully switch between client-mode on/off without deleting blk*.dat in between is not implemented yet. It would mostly be a matter of having non-client LoadBlockIndex ignore block index entries with null block pos. That would make it re-download those as full blocks. Switching back to client-mode is no problem, it doesn't mind if the full blocks are there.

    If the initial block download becomes too long, we'll want client mode as an option so new users can get running quickly. With graceful switch-off of client mode, they can later turn off client mode and have it download the full blocks if they want to start generating.

  3. laanwj referenced this in commit 50d08dc1e1 on Sep 18, 2011
  4. sipa referenced this in commit d73bad33e8 on Jun 2, 2013
  5. bardiharborow commented at 11:50 AM on March 29, 2014: contributor

    @gavinandresen, we need to add another service category (NODE_SPV??) for SPV clients, so that they can operate without getting queried for blocks by other peers.

  6. laanwj commented at 11:54 AM on March 29, 2014: member

    Why, how is this any different from what SPV clients do? See #3884 for another step towards this.

  7. bardiharborow commented at 12:24 AM on March 30, 2014: contributor

    @laanwj, my understanding is that most SPV clients at the moment either get headers from a centralised source (Electrum) or use their own p2p network. I don't know, some may use the main Bitcoin network, but wouldn't they be queried for blocks by other clients and have to ignore those requests which could confuse the other peer (maybe?). I think it would be neater if other peers knew that you could be asked for headers, but not for blocks/txs. Anyway, sorry to bring up an old issue. Just thought someone should start working through the massive backlog, so I started from the back.

  8. christophebiocca commented at 12:29 AM on March 30, 2014: none

    @bardiharborow SPV nodes talk to full nodes, not to each other (although that might be possible as well?). There's a service bit specifically for offering full blocks instead of just headers: https://en.bitcoin.it/wiki/Protocol_specification#version

  9. sipa commented at 3:06 PM on March 30, 2014: member

    Electrum does SPV validation, but is in the strictly technical sense not an SPV node, as it does not participate in the P2P network.

    Actual SPV nodes (like Bitcoin Wallet for Android or Multibit) do connect to the P2P network (only to other full nodes, service flag NODE_NETWORK), and fetch blocks and headers from it. SPV nodes so far don't connect to each other, both because they provide very little service (most don't even keep all old headers around) and because they are only online sporadically.

  10. bardiharborow commented at 9:36 PM on March 30, 2014: contributor

    I've just realised that nodes can chose not to advertise their IP, which I assume is what they do. Does anyone know how hard this would be to implement? The only technical discussion seems to be 3 years old. Wouldn't we also need a way of being alerted to transactions and downloading merkle branches? (This may already exist but the wiki maybe out of date and I'm not completely on top of the code.)

  11. sipa commented at 9:38 PM on March 30, 2014: member

    All this exists. SPV clients are used, and the protocol provides everything necessary. See BIP37 for example.

    The only thing we don't have is the ability to run the reference code base in such an SPV client mode, where it doesn't do full block validation. There are several steps towards that, but the first is certainly headers-first synchronization.

  12. laanwj commented at 2:18 AM on March 31, 2014: member

    @bardiharborow This has all been done years ago. Please move questions like these to the stack exchange or forums, or #bitcoin on IRC. Github is not the place for basic discussion like this.

  13. justmoon referenced this in commit 3de48d4a52 on Apr 7, 2014
  14. kac- referenced this in commit 3eb6774e8a on Apr 22, 2014
  15. laanwj added the label P2P on May 9, 2014
  16. kac- referenced this in commit 35f0a4e2e9 on Oct 8, 2014
  17. gavinandresen commented at 6:28 PM on October 28, 2014: contributor

    Headers-first download merged. And there are plenty of SPV wallets. Closing this.

  18. gavinandresen closed this on Oct 28, 2014

  19. rdponticelli referenced this in commit 920c243932 on Nov 26, 2014
  20. rubensayshi referenced this in commit 2dd28153fd on Mar 16, 2015
  21. dexX7 referenced this in commit 23e8332381 on Apr 8, 2015
  22. TheBlueMatt referenced this in commit 704d415d05 on Oct 20, 2015
  23. brishtiteveja referenced this in commit 3ebf38ee00 on Jan 25, 2016
  24. braydonf referenced this in commit c8ee50d671 on May 12, 2016
  25. tempgit9 referenced this in commit 9562d65297 on Jan 16, 2017
  26. AkioNak referenced this in commit 4cc1ea7290 on Jul 24, 2017
  27. MarcoFalke referenced this in commit 351fadd68a on Dec 19, 2017
  28. MarcoFalke referenced this in commit 88411e98e5 on Dec 19, 2017
  29. classesjack referenced this in commit 3f731c8379 on Jan 2, 2018
  30. hkjn referenced this in commit 3aab98f8dc on Jan 13, 2018
  31. hkjn referenced this in commit 7e80175d2b on Feb 23, 2018
  32. KrzysiekJ referenced this in commit 2759850592 on Mar 28, 2018
  33. effectsToCause referenced this in commit 2fea219d9c on Jun 22, 2018
  34. dginovker referenced this in commit d2186b3db6 on Jul 29, 2018
  35. MarcoFalke referenced this in commit 4d550ffab6 on Jul 30, 2018
  36. hashtobewild referenced this in commit 8ef8754605 on Aug 26, 2018
  37. dooglus referenced this in commit 94bef303b0 on Oct 20, 2018
  38. DigiGreenCoin referenced this in commit 010b7f223f on Oct 29, 2019
  39. MarcoFalke referenced this in commit cf43f3f0a8 on Dec 5, 2019
  40. Warchant referenced this in commit 7958da26e8 on Dec 31, 2019
  41. Bushstar referenced this in commit 01915f3cc6 on May 5, 2020
  42. MarcoFalke referenced this in commit 4946400470 on Jun 26, 2020
  43. laanwj referenced this in commit 31bdd86631 on Jul 15, 2020
  44. KolbyML referenced this in commit 1a313221ad on Aug 1, 2020
  45. KolbyML referenced this in commit fb35e0e26f on Sep 4, 2020
  46. laanwj referenced this in commit 924a4ff7eb on Oct 29, 2020
  47. jonasschnelli referenced this in commit c33662a0ea on Dec 2, 2020
  48. KolbyML referenced this in commit 3d9f0283ff on Dec 5, 2020
  49. MarcoFalke referenced this in commit 590e49ccf2 on Apr 4, 2021
  50. MarcoFalke referenced this in commit bce09da122 on Apr 28, 2021
  51. fanquake referenced this in commit fa00bb2c5c on Apr 29, 2021
  52. MarcoFalke referenced this in commit eb9a1fe037 on May 7, 2021
  53. laanwj referenced this in commit ee9befe8b4 on May 12, 2021
  54. MarcoFalke referenced this in commit c857148636 on May 15, 2021
  55. satindergrewal referenced this in commit 46ce232dc0 on Jun 22, 2021
  56. MarcoFalke locked this on Sep 8, 2021
Linked (view graph)
#340 bug introduced in 0.3.23, crash when no network interfaces up.#447 RPC 'sendfrom' with invalid address crashes bitcoind#672 bitcoin-qt crashes in db on startup#737 Deadlock during startup (Appears to be AddAddress / DNSSeed related)#982 Startup blocks on mutex at "Loading Addresses"#999 v0.6.0rc5 segfault on initial blockchain download when out of diskspace#1065 addrMan crash on CentOS#1098 segfault in qrcode message dialog#1156 Addrman crash#1472 detachdb - coredump backtrace#1505 Boost exceptions in ReadConfigFile() not caught#2397 ReserveKeyFromKeyPool(): read failed#2416 Bitcoind aborts on startup: ./db/skiplist.h:152 Assertion `n >= 0' failed.#2472 System error: database corrupted: void boost::recursive_mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed#3997 crash in CMerkleTx::GetDepthInMainChainINTERNAL()#4410 bitcoin-qt core dumping#4502 GetMyExternalIP hangs shutdown when waiting for response#4995 null pointer of pindexPrev crash.#5330 cs_main lock not held when reindexing blocks on disk?#5698 Code vulnerable to segfault after a network fork#5795 Bizarre test_bitcoin crash, but passes in debugger#5910 Gitian 64-bit bitcoin-qt segfaults on Ubuntu 14.04#6332 Segmentation fault on bitcoind or bitcoin-qt startup (64bit Linux)#6411 connection limit calculation for select() is overly ambitious with -txindex#6540 scheduler_tests (currently disabled) occasionally deadlocks#6775 Bitcoind v0.11.0 segfaults on startup trying to read wallet.#8749 Could not link against boost_filesystem#8984 GUI hangs during sync#9348 Node crashed with assertion failure in network code#9687 bitcoin-qt crashed in FinalizeNode()#9759 Setting -maxsigcachesize to zero causes segfault#9881 v0.14.0rc2 crash on shutdown#9883 [Qt] Segfault on startup#9997 Running out of disk space during GUI init results in hang and non-drawing windows#10209 Stalled shutdown#10210 Frozen (for 15 minutes) on shutdown [dnsseed thread]#10293 SEGV in segwit fundrawtransaction#10611 Gitian build (current master) crashes on OSX#10861 Segfault on shutdown after pressing 'q' to postpone utxo db upgrade#11355 LevelDB read failure: Corruption: block checksum mismatch#11763 deadlock on shutdown#12038 deadlock on database corruption#12229 Shutdown deadlock in SyncWithValidationInterfaceQueue#12279 shutdown took 8 minutes in v0.15.0.1#12690 LevelDB "corrupted compressed block contents" errors#12915 Segmentation fault in util: ScheduleBatchPriority#13316 make check fails for configure option --with-incompatible-bdb and Berkeley DB 5.3#13806 qa: Fix bench/block_assemble assert failure#14330 [tests] segfault during`DEBUG=1` run of existing_data_no_obfuscate locally#14349 mac: make check failure on macOS 10.14#14359 bitcoin-qt hangs in futex() / __cxa_guard_acquire()#14754 bitcoin-qt hangs on startup on Ubuntu 18.10/Debian Buster#14842 segfault in bench_bitcoin if configured using --with-libs=no#14923 segmentation fault on gui startup#14967 TSAN issue in cuckoocache_erase_parallel_ok test after 14935#15227 bitcoind terminated with uncaught exception if configure --debug-enabled#15300 rpc: segfault if combinepsbt called with empty inputs#15310 gui: crash if encrypt / change passphrase window is open and wallet is unloaded#15379 bitcoind crashes after exception in scheduler thread#15674 Segmentation fault on Ubuntu Linux with trickle#15819 macOS: crash while loading wallet with berkeley-db@4#16307 scheduler: crash after releasing wallet#16815 wallet: CPU use proportional to wallet transaction count when idle#16875 gui: Avoid calling processEvents#16904 Bus error (core dumped) after "Using obfuscation key for" log line#17540 ci build fails in ./qt/test/test_bitcoin-qt on aarch64 native tsan#17620 MemorySanitizer complains about unitialized variable in RPC method names#17627 Suppress false positive warning about uninitialized entropy buffers#17674 tests: Add initialization order fiasco detection in Travis#17729 Fuzzing Bug undefined-behavior pubkey.cpp:210:38#18046 Some fuzzing crashes when running under ASan and/or UBSan#18090 gui: Crash if shutdown during processNewTransaction#18362 gui: segfault unloading and immediately reloading wallet with gui#18372 sanitizer: heap-use-after-free in checkinputs_test#18706 crash briefly after launching binaries compiled using --without-gui and --with-incompatible-bdb#18762 fuzz: AddressSanitizer: SEGV on unknown address in /strprintf#18763 fuzz: AddressSanitizer: container-overflow in /script fuzzer#18913 ci: fuzz/process_message fails#18924 QT Crash when using open wallet dialog#18948 qt: Call setParent() in the parent's context#19024 UB: double lock of mutex in init.cpp#19049 test: Potential deadlock in wallet_tests/CreateWalletFromFile#19211 feature_reindex.py triggers data race#19353 Fix mistakenly swapped "previous" and "current" lock orders#19417 Intermittent CI failure: ThreadSanitizer failure (data race) when running wallet_multiwallet.py --usecli#19448 qa: Data race detected when running `mempool_expiry.py`#19678 UBSan warning when fuzzing with -fsanitize=integer#19715 ThreadSanitizer: data race in feature_reindex.py#19789 fuzz: ASAN complaint on macOS with -fsanitize=fuzzer,address,undefined#19955 test: Potential deadlock in wallet_tests#20163 TSAN race after merge of #20090#20242 fuzz: Properly initialize PrecomputedTransactionData#20334 ci: fuzz/decode_tx fails#20542 Unexpected test fail: wallet/wallet.cpp:3113:51: runtime error: division by zero#20618 Data race in interface_zmq.py#21466 Avoid use of "socket" syscall when formatting IP addresses in CNetAddr::ToString#21479 fuzz: UndefinedBehaviorSanitizer warnings in netbase.cpp#21584 Fix assumeutxo crash due to invalid base_blockhash#21586 test: Add missing suppression for signed-integer-overflow:txmempool.cpp#21605 wallet: Segmentation fault during sync#21632 MemorySanitizer: use-of-uninitialized-value in sqlite3BtreeOpen#21773 fuzz: Ensure prevout is consensus-valid#21802 refactor: Avoid UB in util/asmap (advance a dereferenceable iterator outside its valid range)#21948 test: Fix off-by-one in mockscheduler test RPC#22233 fuzz: UndefinedBehaviorSanitizer warnings in consensus/tx_verify.cpp#22537 UndefinedBehaviorSanitizer: implicit-integer-sign-change consensus/tx_verify#22548 CheckQueue_Correct_Random data race#22588 Cirrus CI showed error in netaddress.cpp file when checking PR#22592 LeakSanitizer detects memory leak if CDBWrapper ctor fails#23160 test_bitcoin-qt error: "Unable to init server: Could not connect: Connection refused"#23248 Bitcoin Core crashes under clang-13 libc++ with "Bad system call (core dumped)"#23321 bitcoind segfault's while loading tr descriptor on regtest v22.0#24290 segfault when compiled with depends DEBUG=1 and libc++#24340 util: Add missing unlinkat to syscall sandbox#24426 test: Fix intermittent Tsan issue#24819 data race in SimpleRoundTripTest logging for macOS#25077 Fix chain tip data race and corrupt rest response#25146 Starting with an unsupported wallet configured leads to a segfault (master only?)#25323 miniscript fuzzer failing after qa-assets update#25365 ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) in BaseIndex#25632 SUMMARY: ThreadSanitizer: SEGV src/index/base.cpp:388:9 in BaseIndex::SetBestBlockIndex(CBlockIndex const*)::$_1::operator()() const#25692 psbt.h:896:51: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned long'#25824 UndefinedBehaviorSanitizer: stack-overflow in miniscript (descriptor_parse)#25857 IBD core dumped during recursive call to CCoinsViewCache::FetchCoin (while connecting best chain tip)#26068 Segmentation fault in the scheduler thread when an index fails to commit to the db#26274 bitcoind dumps core when deriveaddresses is called with index 2147483647 (2^31-1)#26477 validation: fix broken maxtipage comparison#27222 test: use-of-uninitialized-value in sqlite3Strlen30#27354 wallet: Data race in GetOrCreateLegacyScriptPubKeyMan vs IsMine#27355 index: ThreadSanitizer: data race on vptr #27360 ci: use LLVM/clang-16 in native_asan job#27492 ci: failure in Docker build step#27582 TSAN: lock-order-inversion (potential deadlock) in ZapSelectTx test#27922 ci: fix llvm-symbolizer in MSAN jobs#27924 fuzz: banman, Assertion `banmap == banmap_read' failed#28510 migratewallet crashes on an freshly created datadir ( wallet/wallet.h:959: int wallet::CWallet::GetLastBlockHeight() const: Assertion `m_last_block_processed_height >= 0' failed.)#28570 fuzz: msan fuzz CI currently broken on x86_64#28665 'std::out_of_range' crash in I2P fuzz test#28917 fuzz, parse_iso8601: attempt to dereference an end-of-stream istreambuf_iterator#28918 fuzz, coinselection: Assertion 'result_bnb->GetChange(coin_params.m_cost_of_change, CAmount{0}) == 0' failed#28966 test: Add missing CBlockPolicyEstimator::processBlockTx suppression#28969 fuzz: Avoid signed-integer-overflow in wallet_notifications fuzz target#29153 new crash in v26.0#29178 ubsan: misaligned-pointer-use in crc32c/src/crc32c_arm64.cc#29767 ThreadSanitizer: data race src/flatfile.cpp:47:13 in FlatFileSeq::Open(FlatFilePos const&, bool)#29908 test: SegFault in `ismine_tests` on SunOS / illumos#30057 ci: msan use-of-uninitialized-value when -jobs=1 (or higher)#30284 Mini miner `AncestorFeerateComparator` Signed Integer Overflow#30402 Illegal Instruction in `CoinStatsIndex::CustomAppend`#30760 ci: fuzz_msan failed with ==4201==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55f0c9bdeffb in SetArgs#31203 fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction#31234 fuzz: connman target: terminate called after throwing an instance of 'std::bad_alloc'#31618 multiprocess: `ipc_tests` fail on *BSD#31770 fuzz: oss-fuzz coverage build is failing#31921 intermittent ipc_tests (Timeout)#32089 Failure to run Fuzz tests when running with corpus#32111 wallet: migratewallet crashes "Assertion `m_wallet_flags == 0' failed"#32112 wallet: migratewallet crashes "Assertion `legacy_spkm' failed"#32173 validation: `CheckBlockIndex` crashes during block reconsideration#32558 intermittent issue in qt addressBookTests: (table_view->model()->rowCount()): [3 != 2] Loc: [./qt/test/addressbooktests.cpp(183)]#32780 lsan: add more Qt suppressions#32957 SegFault in QSortFilterProxyModelPrivate::build_source_to_proxy_mapping#33063 util: Revert "common: Close non-std fds before exec in RunCommandJSON"#33097 fuzz: `txgraph`: Assertion `cmp == 0' failed#33150 intermittent TSAN failure in lockmanager_tests::blockmanager_readblock_hash_mismatch#33245 integer sanitizer warning, when running with -natpmp=1 enabled#33277 intermittent Segmentation fault in mptest IPC path (callFnAsyncParams)#33548 `test_bitcoin-qt`: segfault under LTO (CMAKE_INTERPROCEDURAL_OPTIMIZATION=ON)#33949 ubsan: add another suppression for InsecureRandomContext#33964 Startup crash on macOS with GCC 15.2: `std::source_location::file_name()` is `nullptr`#34014 mptest hangs, when run in a loop#34109 refactor: Use uint64_t over size_t for serialize corruption check in fees.dat#34110 scriptpubkeyman fuzz target creates Base58 too large; Fails on 32-bit (armhf, ...)#34645 oss-fuzz: coins_view_overlay: ASSERT: m_dirty_count == 0#34655 fuzz: keep `coins_view` fuzzers within caller contracts#34711 src/ipc/libmultiprocess/src/mp/proxy.cpp:416:44: runtime error: member access within null pointer of type 'mp::Waiter'#34732 i386 segv in #0 0xf7a49ce0 in kj::operator*(kj::_::Stringifier, kj::Exception const&) (/lib/i386-linux-gnu/libkj-1.1.0.so+0x38ce0)#34734 test: Fix shutdown vptr race in BlockFilterIndexSync bench#34756 ipc: AddressSanitizer: SEGV nptl/pthread_mutex_lock.c:80:23 in __pthread_mutex_lock#34777 ipc: AddressSanitizer: heap-use-after-free in `capnp::CallContext<ipc::capnp::messages::BlockTemplate::GetBlockParams, ipc::capnp::messages::BlockTemplate::GetBlockResults>::getParams()`#34782 mptest: mp::CancelMonitor: AddressSanitizer: stack-use-after-return

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-03 00:16 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me