From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Wed, 10 Dec 2025 11:48:51 -0800 Received: from mail-oa1-f60.google.com ([209.85.160.60]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1vTQB7-00043l-Es for bitcoindev@gnusha.org; Wed, 10 Dec 2025 11:48:50 -0800 Received: by mail-oa1-f60.google.com with SMTP id 586e51a60fabf-3f509212de4sf124993fac.3 for ; Wed, 10 Dec 2025 11:48:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1765396123; cv=pass; d=google.com; s=arc-20240605; b=SJGlXzRevv1reAJp9mlGXx43VFjSwiBivRPXHdfyRTvWZk01/qK9kbMh4/taZBhsZ6 vrFo+1eDKsjy31KnhSNGTRUStBWOPrFEUGLF61TlZQfMtFqCWkofbOxUOCspnGh4pvXX Srfi7mVlIXt6qeJgufdvcuny0K+KxYYy9Q1hIkpQkpVu/jd5Zz/mNCURUfLqf6yH5Tv8 LDnDnfnu1VhdcLD+74PJpHii6uuHRuU/nsssWCkQlRCZHu66Fuy8E8dKt+AK7Yie5Si5 3NNHeph8x+1BIjljDPjEfU6qAs2XUD4dGq0UPPCui8D0bytgH1LXzqaxW6pVCF4oFEDm gQxA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:sender:dkim-signature; bh=mBiqHRA1yTjyklvxzlvyK2modzciaWOmaQzX5OH9n7M=; fh=ROmm5GFjmhVeQaA0n0I8GUKhaRPOhd2RvZNywpHzQs0=; b=ibblcUFIQK6cjYPXHGRyGQ/ftY+jf2hzf3hz0fYKeCdOsd869THTh+pYM7A8+ZJptL LuJyK1xKYzHp3nlUvVeYDFGX/5duA+A0kH0wVmKHdQPq1Uk4yTPca4B9Tn5OHzz+B5Nx hUYEI5fx6Tt9YakNljtADjUA68uvX4P4hwVjVbWfryNaXU3hJbVRlInczkYmDcHHMr8T NtzpXAMOUNRDesGc57c04HBAwEi/ApgoYIQ3etZhveuiX4kY+QtHY13uz9gYOWzaaLlT Q1cFbTa92rrRpR1dCrks50B96aBRx4VFRtUhRyM9lYwWjvX9k9M3PSHDd5bKxqx4ydr/ /qhQ==; darn=gnusha.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@q32-com.20230601.gappssmtp.com header.s=20230601 header.b=LixsTMBf; spf=pass (google.com: domain of earonesty@gmail.com designates 2a00:1450:4864:20::630 as permitted sender) smtp.mailfrom=earonesty@gmail.com; dara=pass header.i=@googlegroups.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1765396123; x=1766000923; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:sender:from:to:cc:subject:date:message-id :reply-to; bh=mBiqHRA1yTjyklvxzlvyK2modzciaWOmaQzX5OH9n7M=; b=L6OTz/ZBWaA8DJKf2OFXcLnZlP8H549cBIqf82BeU98LWpvguvDlo0p5UA6ZhAHCsr ZawbSgHvqMQpKdWhbeLy8h8gEfPhCoBqurXsyC+pPa/XeigBo7DAtPpOy3r+soFyxuXP KmFOxLKHz/g0sUcPMBH+5z3jtdlyRtSZz/VPcdqC0uT9wOkx4Ggl84u8sYi0fJejdoEj +xu0+0Uqwsdmc9anSVkMq9pzBKhD6TWPObcAOypSPl6/SMEvlvkCPzhf+Hh7k/K5B4k6 gXU6yBQZBRxwqf6TRjkLXtnIkld/CU0QbLNQtnRPsZeTx8ll9xH0zN1I6jFfnXkJE0s/ tr8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765396123; x=1766000923; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:x-gm-gg:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=mBiqHRA1yTjyklvxzlvyK2modzciaWOmaQzX5OH9n7M=; b=B2bPmb2a5OGhJV+jfGREeHhCrPho4w6ysLg/hLVb4zZR+gucz3tkCURaSTIQGfDVhX YY8f2NGAtOQ7JaZfXWMPLThXVdMM2UD3uSSIxBaNcyGGDqtjyM6U86MSaYOzAz29tGJu t6vX2dbgKYlDoVKaAHkq6BGhIixIIoqFzr4/BojAmGgu67vJazRTqjzBrDY8FvwAsBy4 oOGNBfKYhSrlWUn4S3dB3c0XzON3HIcpuvnz3NklYqw/m84ajNGhwSuMPIIVi0Cy0p0D iXmNIkvVdjOs1c6Heke+7enoNyvsISsh4Ah87G11k5h+GTQe9AxV10yhsetpDJ+n+x8A eCDA== Sender: bitcoindev@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCW/StST7JnxQCL5ulkMr4hkaEbh1kfQUDD5ftXtZ6H6iQviGErLyKaKMWltvbZ02mAEEkhl7DJmqKSU@gnusha.org X-Gm-Message-State: AOJu0YwU9/TCdboJXc+QyVllqhpUUIu9goTHRrcXTxAJCJbYMV+wK6eQ FqvFU8cjSh54NBDuirWF4mYcl40scvSpJPaD9+gc34cc5D8KOEIQafQD X-Google-Smtp-Source: AGHT+IEuH1Brb6Wnn0oQCAZwatjCwHhthjisPKpxw7no4bNND8kajYOFxOWc53lGbVgvgieaY78m6Q== X-Received: by 2002:a05:6820:179a:b0:659:9a49:8e7a with SMTP id 006d021491bc7-65b2ad1a19emr2003173eaf.74.1765396122972; Wed, 10 Dec 2025 11:48:42 -0800 (PST) X-BeenThere: bitcoindev@googlegroups.com; h="AWVwgWZz/uWctyumBq+BknkpSKTfoBBWeyjdJC+RuNrRTuN8gg==" Received: by 2002:a05:6870:8128:b0:3e2:d619:f0df with SMTP id 586e51a60fabf-3f5d73a509els46428fac.0.-pod-prod-04-us; Wed, 10 Dec 2025 11:48:38 -0800 (PST) X-Received: by 2002:a05:6808:3084:b0:450:c6a0:3f39 with SMTP id 5614622812f47-45586528da9mr2358366b6e.58.1765396118570; Wed, 10 Dec 2025 11:48:38 -0800 (PST) Received: by 2002:a05:600c:c059:20b0:477:99f7:45de with SMTP id 5b1f17b1804b1-47933ebc714ms5e9; Wed, 10 Dec 2025 11:33:22 -0800 (PST) X-Received: by 2002:a05:600c:a0a:b0:479:1a0a:ebbe with SMTP id 5b1f17b1804b1-47a8375ab54mr36332555e9.14.1765395199436; Wed, 10 Dec 2025 11:33:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1765395199; cv=none; d=google.com; s=arc-20240605; b=bMNCJ3gl5m75QGDfWLSeNfMnrM+lglhVOA0Z7LZPNdlpIHMwBBkuS3ZI4qcP6Vdp73 TemmtA0njlIteb0ekp/RPcLlN0ZifiBTAXE0DLMrCZYzOtAuD38S6IzxoIlpijJunxvO pMgrRn9Y+6vitG5Y/YlPhSg+ce+sglMygZaly6egCv3hkkDMTmtxqwsYk2pfoUfACZKN +9xAJVJM7x0H+8iJg3e8MGugPlWtMDUuLQZRe7WbQIke2uoby6F7xQRKP6v+ZFZvcEZN EGxEPqvXG9AjOPQZJpwaVT407o3SygGV64QIjypLi4z5hsk9PwuRGW4yhhiZQRLyGeme xyEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=509zxwSeg1yQn20Wg3WJjeudmpMU9NhybGBlMvD82oI=; fh=VUyRMGDsLDyKXHBc8DWjokFBiSMTvXavinKdBJZhUls=; b=lj96eIx+98I2wxqbQp7lgOpGSY8P84i6IKF599bBbRQJsbg+igyUQ7IQeOSTEOTtwW VcuwTJnPFUWZdGkj+6qavdFkD+M9xi/Jv02cls3NEzLCo0E1IN29Wyq+D2Fdqa12yaJc j3Z+5Iofm+adCbZ0PNOzCTj2ChvMfHT6bhU38G55jrnUojYyqhs3L3iho4UjfEdD7C+e JK5yFnOYF1y5yPlL6mwGteMm3xxlCFwdkKD1xJe7Mn6yVu1kiwWQd9o2fzKpZIJO6q+J vGdlBWdLcotsHfa8mMeiqCpIXfqEwDwpWxWQhGh0uTMhJGWhiysiqaCm4mJriLpOtCKq 7PBw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@q32-com.20230601.gappssmtp.com header.s=20230601 header.b=LixsTMBf; spf=pass (google.com: domain of earonesty@gmail.com designates 2a00:1450:4864:20::630 as permitted sender) smtp.mailfrom=earonesty@gmail.com; dara=pass header.i=@googlegroups.com Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com. [2a00:1450:4864:20::630]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-47a7d96c000si433325e9.2.2025.12.10.11.33.19 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 10 Dec 2025 11:33:19 -0800 (PST) Received-SPF: pass (google.com: domain of earonesty@gmail.com designates 2a00:1450:4864:20::630 as permitted sender) client-ip=2a00:1450:4864:20::630; Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-b735487129fso25295266b.0 for ; Wed, 10 Dec 2025 11:33:19 -0800 (PST) X-Gm-Gg: ASbGncsORQ6sv4p5NFLHwLpKn+E8M5jrNhlgk8kY+hUab2Laz4GGYf3M24a4/W4TY4D nle8lnSP0dhbMIVUlQx9W0OB69O+HKWTgUoEu4XQFUBPn5xQAL63y32LktXmt4O51aiJMZ8o26i FCt0G8x2fqlrOG3Md+/2K8enjSvmc26bE/vjr/NzvyAq8RwcX00N57omd3btJx8J+mzlQxfDozp Z6Q3d0qExIuHH6TL4Xo9Fj8iauBKlz9aMmI+JhrioU12HgFFw30PEEpPAA19ZdRAxeajW/vbGPH 6OPahtKXtwPFfF3WObA9rdLVDfr1zW6Wtw== X-Received: by 2002:a17:907:60d0:b0:b73:8759:62f6 with SMTP id a640c23a62f3a-b7ce84bff38mr374603066b.60.1765395198485; Wed, 10 Dec 2025 11:33:18 -0800 (PST) MIME-Version: 1.0 References: <049858f4-28ba-4ffb-8966-f8bef09035b7n@googlegroups.com> In-Reply-To: <049858f4-28ba-4ffb-8966-f8bef09035b7n@googlegroups.com> From: Erik Aronesty Date: Wed, 10 Dec 2025 11:33:08 -0800 X-Gm-Features: AQt7F2qkXSYQj50LWSKeoMynGlM5QZGu-aJTnlpffuF2a8ApEZcyXCqXQ1aR6J4 Message-ID: Subject: Re: [bitcoindev] Re: Reducing RAM requirements with dynamic dust To: Eric Voskuil Cc: Bitcoin Development Mailing List Content-Type: multipart/alternative; boundary="000000000000b1a87106459e19f2" X-Original-Sender: erik@q32.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@q32-com.20230601.gappssmtp.com header.s=20230601 header.b=LixsTMBf; spf=pass (google.com: domain of earonesty@gmail.com designates 2a00:1450:4864:20::630 as permitted sender) smtp.mailfrom=earonesty@gmail.com; dara=pass header.i=@googlegroups.com Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -0.7 (/) --000000000000b1a87106459e19f2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Core: Keep UTXO lookups in RAM as much as possible. This gives predictable validation under adversarial load and trades memory for protection from pathological disk latency. Libbitcoin: Represent history in append-only, memory-mapped files. Let the operating system decide what stays hot. Accept more disk I/O in return for simpler concurrency and no explicit UTXO map. To create 200 million outputs as an attack, the cost is roughly 48 BTC based on eight vb per output at low feerates. Given the operational problems that a very large UTXO set creates, it may be worthwhile to develop a clear storage abstraction layer. The goal would be to ensure that validation correctness does not depend on the transitional behavior of a specific storage engine. Once the validator has a well defined storage contract, alternate backends such as a libbitcoin style layout could exist as storage formats while still benefiting from the reliability of the rest of the Bitcoin implementation. (I would be interested in something like this.) On Wed, Dec 10, 2025 at 10:34=E2=80=AFAM Eric Voskuil wr= ote: > > Given the increasing RAM requirements, due to the increasing UTXO set > > I have been assured multiple times recently that this problem does not > exist in Bitcoin Core. I'm not sure about that, but I can say for sure th= at > this is not a problem in Libbitcoin and is therefore not problem with the > Bitcoin protocol. > > Best, > Eric > > On Wednesday, December 10, 2025 at 1:12:31=E2=80=AFPM UTC-5 uuowwpevskfco= rdh wrote: > >> Given the increasing RAM requirements, due to the increasing UTXO set, I >> suggest seeing the UTXO set size as a controlled variable. A feedback >> mechanism sets a dynamic dust level, below from which UTXOs are >> removed/discarded and thus freeing RAM. >> >> Below is an overview essay better expressed by grok, which can also be >> seen in here: >> https://hackmd.io/P-2lzGb8TiC86IOE3OGiYA?view >> >> # Enhancing Bitcoin's Scalability: A PID-Inspired Approach to Managing >> UTXO Set Growth >> >> ## Abstract >> >> Bitcoin=E2=80=99s UTXO set is currently an unbounded accumulator that ri= sks >> long-term centralization as node RAM requirements grow without limit. >> Existing fee incentives have proven insufficient against sustained >> low-value output creation (e.g., inscriptions, tokenized assets, dust-he= avy >> protocols). This article proposes a soft-fork mechanism that treats UTXO >> set size as a controlled variable: a slowly rising target size is define= d, >> and a PID-style feedback controller, updated every difficulty epoch, >> dynamically raises a minimum-value floor beneath which old UTXOs become >> unspendable. The result is bounded, predictable growth of the UTXO set w= ith >> ample warning periods, no hard caps on monetary use, and strong resistan= ce >> to bloat attacks=E2=80=94all while remaining fully compatible with a sof= t-fork >> deployment. >> >> ## Introduction >> >> Bitcoin, the pioneering decentralized digital currency, operates as a >> complex dynamic system governed by consensus rules that ensure security, >> immutability, and permissionless participation. At its core, Bitcoin >> maintains a distributed ledger known as the blockchain, which records al= l >> transactions in a sequence of blocks. Each transaction involves inputs a= nd >> outputs: inputs reference previously unspent outputs from prior >> transactions, while outputs create new spendable units called Unspent >> Transaction Outputs (UTXOs). The UTXO set represents the aggregate state= of >> all currently spendable coins in the network, serving as a critical >> component for transaction validation and wallet management. >> >> As Bitcoin has evolved, the UTXO set has grown significantly, influenced >> by increasing adoption and diverse usage patterns. This growth, while >> indicative of the network's vitality, poses challenges to its long-term >> scalability and decentralization. Nodes=E2=80=94computers that validate = and relay >> transactions=E2=80=94must store and process the entire UTXO set in memor= y for >> efficient operation, which can strain resources such as random access >> memory (RAM). Unchecked expansion could lead to higher barriers for runn= ing >> full nodes, potentially centralizing control among fewer, well-resourced >> participants. This article explores a proposed mechanism to address thes= e >> concerns: an adaptive control system inspired by >> Proportional-Integral-Derivative (PID) feedback principles, designed to >> impose bounded growth on the UTXO set while preserving Bitcoin's >> foundational properties. >> >> ## Motivations for UTXO Set Management >> >> To appreciate the need for enhanced UTXO management, it is essential to >> understand the current status quo and its vulnerabilities. Bitcoin's des= ign >> prioritizes efficiency and security through mechanisms like block size >> limits, which cap the amount of data added per block (approximately 1 MB >> base size, expandable to about 4 MB with Segregated Witness). These limi= ts >> help control the overall blockchain size, ensuring predictable hardware >> requirements for storage on hard disk drives (HDDs). Similarly, the >> difficulty adjustment algorithm maintains a consistent block production >> rate of roughly one every 10 minutes by dynamically scaling the >> computational challenge for miners based on network hashrate. >> >> However, the UTXO set lacks comparable built-in constraints. It >> accumulates as users create new outputs=E2=80=94often in small >> denominations=E2=80=94without a mandatory mechanism to consolidate or pr= une them. >> This can result from various activities, including high-frequency >> microtransactions, the embedding of non-monetary data (such as through >> protocols like Ordinals, which inscribe arbitrary information onto >> satoshis, Bitcoin's smallest unit), or the anchoring of sidechain or >> tokenized assets that leverage Bitcoin's security but operate externally= . >> While these innovations expand Bitcoin's utility, they can inadvertently >> increase the UTXO count disproportionately to their economic value, >> elevating transaction fees during congestion and raising operational cos= ts >> for nodes. >> >> Existing incentives, such as transaction fees, partially mitigate this b= y >> encouraging users to consolidate low-value UTXOs to avoid higher costs. >> Yet, these market-driven forces are insufficient against sustained patte= rns >> of low-value output creation, particularly when driven by external syste= ms >> that do not bear the full cost of network maintenance. Over time, this >> leads to bloat: as of recent estimates, the UTXO set exceeds several >> gigabytes when loaded into memory, complicating node synchronization and >> validation. Without intervention, projected growth could undermine >> decentralization, as fewer individuals or entities might afford to >> participate fully in the network. >> >> The motivation for reform, therefore, stems from a desire to balance >> innovation with sustainability. An ideal solution would allow gradual UT= XO >> expansion to accommodate genuine monetary usage while introducing feedba= ck >> to curb excessive accumulation, all without compromising Bitcoin's >> permissionless nature or requiring a hard fork that could fragment the >> community. >> >> ## A PID-Inspired Feedback Controller for UTXO Control >> >> Drawing from control theory, which studies how systems maintain desired >> behaviors through feedback, this proposal introduces a dynamic mechanism= to >> regulate UTXO set size. PID controllers, widely used in engineering for >> processes like temperature regulation or autopilot systems, combine thre= e >> terms: Proportional \(P\) for immediate response to errors, Integral (I) >> for correcting persistent deviations, and Derivative (D) for anticipatin= g >> changes based on trends. In Bitcoin's context, we adapt this framework t= o >> compute a minimum value threshold=E2=80=94or "floor"=E2=80=94below which= UTXOs become >> ineligible for spending, effectively deprecating them over time. >> >> Importantly, this deprecation rule constitutes a soft fork: it restricts >> the set of valid transactions to a subset of those previously allowed, >> without introducing new capabilities. Upgraded nodes enforce the rule, >> rejecting deprecated UTXOs as invalid inputs, ensuring backward >> compatibility and minimizing disruption. >> >> ### Mechanism Overview >> >> The controller activates at each difficulty adjustment epoch, >> approximately every two weeks (2016 blocks), aligning with Bitcoin's >> existing periodic recalibrations for predictability. >> >> 1. **Define a Target UTXO Size Trajectory**: Establish an increasing >> maximum target for the UTXO set size, denoted as $T(t)$, where $t$ >> represents the epoch number. This could grow sublinearly with time or >> blockchain height to reflect organic adoption=E2=80=94 for instance, $T(= t) =3D U_0 >> \cdot (1 + r)^t$, with $U_0$ as the initial size and $r$ a small annual >> growth rate (e.g., less than 1%). This permits expansion while preventin= g >> unbounded divergence. >> >> 2. **Measure the Error**: At each epoch, compute the error $e(t) =3D S(t= ) - >> T(t)$, where $S(t)$ is the current UTXO set size (measured by count or >> aggregate data footprint). >> >> 3. **Compute the Floor Adjustment**: Apply the PID formula to adjust the >> value floor $F(t)$: >> >> $$ \Delta F(t) =3D K_p \cdot e(t) + K_i \cdot \sum_{k=3D0}^{t} e(k)= + >> K_d \cdot \frac{e(t) - e(t-1)}{\Delta t} $$ >> >> Here, $K_p, K_i, K_d$ are tunable gains, selected conservatively >> through simulations to ensure stability (e.g., using root locus analysis= to >> avoid oscillatory or unstable roots in the system's characteristic >> equation). The floor $F(t)$ increases only if $e(t) > 0$, applying to UT= XOs >> below this threshold. >> >> 4. **Deprecation Process**: UTXOs with values below $F(t)$ are marked as >> unspendable in future transactions. To provide fairness and predictabili= ty, >> implement a grace period (e.g., 4-12 months) during which owners can >> consolidate affected UTXOs. >> >> 5. **Safeguards and Tuning**: Incorporate clamps on $\Delta F(t)$ >> (similar to Bitcoin's difficulty adjustment limits) to prevent abrupt >> changes that might cause network congestion from mass consolidations. >> Periodic reviews of gains and targets via community governance would ada= pt >> the system to evolving conditions. >> >> ### Stability and Dynamics Considerations >> >> In dynamic systems terms, the UTXO set can be modeled as an integrator >> accumulating outputs minus expenditures. The PID controller introduces >> negative feedback to dampen this accumulation, driving $S(t)$ toward $T(= t)$ >> with minimal overshoot. Simulations would verify robustness against >> adversarial scenarios, such as deliberate UTXO spam, ensuring the system= 's >> poles remain in the stable region of the complex plane. >> >> Potential benefits include reduced node resource demands, lower fees >> during normal operation, and enhanced decentralization. Risks, such as >> over-deprecation affecting small holders, can be mitigated through caref= ul >> gain selection. >> >> ## Conclusion >> >> This PID-inspired approach offers a principled path to managing Bitcoin'= s >> UTXO set, fostering sustainable growth while upholding core principles. = By >> integrating feedback control, Bitcoin can evolve as a resilient dynamic >> system, better equipped for widespread adoption. Further research, >> including detailed modeling and community discourse, is essential to ref= ine >> and potentially implement such enhancements. >> >> -- > You received this message because you are subscribed to the Google Groups > "Bitcoin Development Mailing List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to bitcoindev+unsubscribe@googlegroups.com. > To view this discussion visit > https://groups.google.com/d/msgid/bitcoindev/049858f4-28ba-4ffb-8966-f8be= f09035b7n%40googlegroups.com > > . > --=20 You received this message because you are subscribed to the Google Groups "= Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/= CAJowKg%2B6%3D%3Dq%3DWa4LRWRHdiOfcV6RPtYdCzVjY%2BJJrmEgbR6c2A%40mail.gmail.= com. --000000000000b1a87106459e19f2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Core: Keep UTXO lookups in RAM as much= as possible. This gives predictable validation under adversarial load and = trades memory for protection from pathological disk latency.
Libbitcoin: Represent history in append-only, memory-mapped files. Let the = operating system decide what stays hot. Accept more disk I/O in return for = simpler concurrency and no explicit UTXO map.

To create 200 million outputs as an attack, the cost is roughly 48 BTC b= ased on eight vb per output at low feerates.

Given the operational problems that a very large UTXO set creates, it ma= y be worthwhile to develop a clear storage abstraction layer.

The g= oal would be to ensure that validation correctness does not depend on the t= ransitional behavior of a specific storage engine.

Once the validat= or has a well defined storage contract, alternate backends such as a libbit= coin style layout could exist as storage formats while still benefiting fro= m the reliability of the rest of the Bitcoin implementation.

(I wo= uld be interested in something like this.)

On Wed, Dec 1= 0, 2025 at 10:34=E2=80=AFAM Eric Voskuil <eric@voskuil.org> wrote:
> Giv= en the increasing=C2=A0RAM requirements, due to the increasing=C2=A0UTXO se= t

I have been assured multiple times recently that this probl= em does not exist in Bitcoin Core. I'm not sure about that, but I can s= ay for sure that this is not a problem in Libbitcoin and is therefore not p= roblem with the Bitcoin protocol.

Best,
Eric

On Wednesday, December= 10, 2025 at 1:12:31=E2=80=AFPM UTC-5 uuowwpevskfcordh wrote:
Given the increasing=C2=A0RAM requirements, d= ue to the increasing=C2=A0UTXO set, I suggest seeing the UTXO set size as a= controlled variable. A feedback mechanism sets a dynamic dust level, below= from which UTXOs are removed/discarded and thus freeing RAM.

Below is an overview essay bet= ter expressed by grok, which can also be seen in here:
=

# Enhancing= Bitcoin's Scalability: A PID-Inspired Approach to Managing UTXO Set Gr= owth

## Abstract
Bitcoin=E2=80=99s UTXO set is currently an unbounded accu= mulator that risks long-term centralization as node RAM requirements grow w= ithout limit. Existing fee incentives have proven insufficient against sust= ained low-value output creation (e.g., inscriptions, tokenized assets, dust= -heavy protocols). This article proposes a soft-fork mechanism that treats = UTXO set size as a controlled variable: a slowly rising target size is defi= ned, and a PID-style feedback controller, updated every difficulty epoch, d= ynamically raises a minimum-value floor beneath which old UTXOs become unsp= endable. The result is bounded, predictable growth of the UTXO set with amp= le warning periods, no hard caps on monetary use, and strong resistance to = bloat attacks=E2=80=94all while remaining fully compatible with a soft-fork= deployment.

## Introduction

Bitcoin, the pioneering decentralized digital= currency, operates as a complex dynamic system governed by consensus rules= that ensure security, immutability, and permissionless participation. At i= ts core, Bitcoin maintains a distributed ledger known as the blockchain, wh= ich records all transactions in a sequence of blocks. Each transaction invo= lves inputs and outputs: inputs reference previously unspent outputs from p= rior transactions, while outputs create new spendable units called Unspent = Transaction Outputs (UTXOs). The UTXO set represents the aggregate state of= all currently spendable coins in the network, serving as a critical compon= ent for transaction validation and wallet management.

=
As Bitcoin has evolved, the UTXO set has grown significant= ly, influenced by increasing adoption and diverse usage patterns. This grow= th, while indicative of the network's vitality, poses challenges to its= long-term scalability and decentralization. Nodes=E2=80=94computers that v= alidate and relay transactions=E2=80=94must store and process the entire UT= XO set in memory for efficient operation, which can strain resources such a= s random access memory (RAM). Unchecked expansion could lead to higher barr= iers for running full nodes, potentially centralizing control among fewer, = well-resourced participants. This article explores a proposed mechanism to = address these concerns: an adaptive control system inspired by Proportional= -Integral-Derivative (PID) feedback principles, designed to impose bounded = growth on the UTXO set while preserving Bitcoin's foundational properti= es.

## Motivations for UTXO Set Manag= ement

To appreciate the need for enha= nced UTXO management, it is essential to understand the current status quo = and its vulnerabilities. Bitcoin's design prioritizes efficiency and se= curity through mechanisms like block size limits, which cap the amount of d= ata added per block (approximately 1 MB base size, expandable to about 4 MB= with Segregated Witness). These limits help control the overall blockchain= size, ensuring predictable hardware requirements for storage on hard disk = drives (HDDs). Similarly, the difficulty adjustment algorithm maintains a c= onsistent block production rate of roughly one every 10 minutes by dynamica= lly scaling the computational challenge for miners based on network hashrat= e.

However, the UTXO set lacks compar= able built-in constraints. It accumulates as users create new outputs=E2=80= =94often in small denominations=E2=80=94without a mandatory mechanism to co= nsolidate or prune them. This can result from various activities, including= high-frequency microtransactions, the embedding of non-monetary data (such= as through protocols like Ordinals, which inscribe arbitrary information o= nto satoshis, Bitcoin's smallest unit), or the anchoring of sidechain o= r tokenized assets that leverage Bitcoin's security but operate externa= lly. While these innovations expand Bitcoin's utility, they can inadver= tently increase the UTXO count disproportionately to their economic value, = elevating transaction fees during congestion and raising operational costs = for nodes.

Existing incentives, such = as transaction fees, partially mitigate this by encouraging users to consol= idate low-value UTXOs to avoid higher costs. Yet, these market-driven force= s are insufficient against sustained patterns of low-value output creation,= particularly when driven by external systems that do not bear the full cos= t of network maintenance. Over time, this leads to bloat: as of recent esti= mates, the UTXO set exceeds several gigabytes when loaded into memory, comp= licating node synchronization and validation. Without intervention, project= ed growth could undermine decentralization, as fewer individuals or entitie= s might afford to participate fully in the network.

The motivation for reform, therefore, stems from a desire to= balance innovation with sustainability. An ideal solution would allow grad= ual UTXO expansion to accommodate genuine monetary usage while introducing = feedback to curb excessive accumulation, all without compromising Bitcoin&#= 39;s permissionless nature or requiring a hard fork that could fragment the= community.

## A PID-Inspired Feedbac= k Controller for UTXO Control

Drawing= from control theory, which studies how systems maintain desired behaviors = through feedback, this proposal introduces a dynamic mechanism to regulate = UTXO set size. PID controllers, widely used in engineering for processes li= ke temperature regulation or autopilot systems, combine three terms: Propor= tional \(P\) for immediate response to errors, Integral (I) for correcting = persistent deviations, and Derivative (D) for anticipating changes based on= trends. In Bitcoin's context, we adapt this framework to compute a min= imum value threshold=E2=80=94or "floor"=E2=80=94below which UTXOs= become ineligible for spending, effectively deprecating them over time.

Importantly, this deprecation rule cons= titutes a soft fork: it restricts the set of valid transactions to a subset= of those previously allowed, without introducing new capabilities. Upgrade= d nodes enforce the rule, rejecting deprecated UTXOs as invalid inputs, ens= uring backward compatibility and minimizing disruption.
### Mechanism Overview

<= span>The controller activates at each difficulty adjustment epoch, approxim= ately every two weeks (2016 blocks), aligning with Bitcoin's existing p= eriodic recalibrations for predictability.

= 1. **Define a Target UTXO Size Trajectory**: Establish an increasing = maximum target for the UTXO set size, denoted as $T(t)$, where $t$ represen= ts the epoch number. This could grow sublinearly with time or blockchain he= ight to reflect organic adoption=E2=80=94 for instance, $T(t) =3D U_0 \cdot= (1 + r)^t$, with $U_0$ as the initial size and $r$ a small annual growth r= ate (e.g., less than 1%). This permits expansion while preventing unbounded= divergence.

2. **Measure the Error**= : At each epoch, compute the error $e(t) =3D S(t) - T(t)$, where $S(t)$ is = the current UTXO set size (measured by count or aggregate data footprint).<= /span>

3. **Compute the Floor Adjustment**: = Apply the PID formula to adjust the value floor $F(t)$:
=C2=A0 =C2=A0
=C2=A0 =C2=A0 $$ =C2=A0\Delta F(t)= =3D K_p \cdot e(t) + K_i \cdot \sum_{k=3D0}^{t} e(k) + K_d \cdot \frac{e(t= ) - e(t-1)}{\Delta t} $$
=C2=A0 =C2=A0
<= div>=C2=A0 =C2=A0Here, $K_p, K_i, K_d$ are tunable gains, selected co= nservatively through simulations to ensure stability (e.g., using root locu= s analysis to avoid oscillatory or unstable roots in the system's chara= cteristic equation). The floor $F(t)$ increases only if $e(t) > 0$, appl= ying to UTXOs below this threshold.

4= . **Deprecation Process**: UTXOs with values below $F(t)$ are marked as uns= pendable in future transactions. To provide fairness and predictability, im= plement a grace period (e.g., 4-12 months) during which owners can consolid= ate affected UTXOs.

5. **Safeguards a= nd Tuning**: Incorporate clamps on $\Delta F(t)$ (similar to Bitcoin's = difficulty adjustment limits) to prevent abrupt changes that might cause ne= twork congestion from mass consolidations. Periodic reviews of gains and ta= rgets via community governance would adapt the system to evolving condition= s.

### Stability and Dynamics Conside= rations

In dynamic systems terms, the= UTXO set can be modeled as an integrator accumulating outputs minus expend= itures. The PID controller introduces negative feedback to dampen this accu= mulation, driving $S(t)$ toward $T(t)$ with minimal overshoot. Simulations = would verify robustness against adversarial scenarios, such as deliberate U= TXO spam, ensuring the system's poles remain in the stable region of th= e complex plane.

Potential benefits i= nclude reduced node resource demands, lower fees during normal operation, a= nd enhanced decentralization. Risks, such as over-deprecation affecting sma= ll holders, can be mitigated through careful gain selection.

## Conclusion

This= PID-inspired approach offers a principled path to managing Bitcoin's U= TXO set, fostering sustainable growth while upholding core principles. By i= ntegrating feedback control, Bitcoin can evolve as a resilient dynamic syst= em, better equipped for widespread adoption. Further research, including de= tailed modeling and community discourse, is essential to refine and potenti= ally implement such enhancements.

--
You received this message because you are subscribed to the Google Groups &= quot;Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.googl= e.com/d/msgid/bitcoindev/049858f4-28ba-4ffb-8966-f8bef09035b7n%40googlegrou= ps.com.

--
You received this message because you are subscribed to the Google Groups &= quot;Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoind= ev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.googl= e.com/d/msgid/bitcoindev/CAJowKg%2B6%3D%3Dq%3DWa4LRWRHdiOfcV6RPtYdCzVjY%2BJ= JrmEgbR6c2A%40mail.gmail.com.
--000000000000b1a87106459e19f2--