These commits revise the layout of a few key classes to eliminate padding, eliminating useless memory overhead.
-This reduces CTransaction from 96 bytes to 88 bytes
These commits revise the layout of a few key classes to eliminate padding, eliminating useless memory overhead.
-This reduces CTransaction from 96 bytes to 88 bytes
432 | @@ -433,15 +433,14 @@ class CTransaction 433 | 434 | void UpdateHash() const; 435 | }; 436 | -
nit: we mostly have newlines between classes.
109 | + // Space that is typically used as padding. Commented out so that it doesn't cause extra memory use on different architectures. 110 | + // char PADDING_FREE_SPACE[2]; 111 | + 112 | + // Bool flags 113 | + struct CTxMemPoolEntryFlags { 114 | + int UNUSED_BITS : 6;
What's the purpose of UNUSED_BITS?
Maybe a uint32_t bitmap would be the better choice here?
There's no purpose, just to demarcate that these bits are available for future programmers. A uint32_t would be 4 bytes whereas this struct should be 1 byte.
Sorry.. overlooked that you are using a c++ bitfield. But I guess the struct will still consume 4 bytes.
Bitfields are standard C.
The size of the struct will be 1 byte.
However, the alignment of the entire CTxMempool object will still be 4, so there will be 3 padding bytes at the end of CTxMempool, I think?
@sipa are you sure of the alignment? On my system it is 8 bytes, which leaves no padding for the CTxMemPoolEntry. Perhaps you're on OSX where size_t is 4 bytes not 8 (I didn't look at OSX).
@jeremyrubin size_t is the size of a pointer, which is 32 bits on 32 bits architectures and 64 bits on 64 bits architectures. You're right that the overall alignment is 8 bytes (on my architecture at least).
It seems the flags field becomes 4 bytes anyway, so there is no padding afterwards.
Concept ACK
No, I'm not. I haven't actually tried. But I assume the int before the bitfield is 4-aligned, which would make the end of the bitfield byte be at a multiple of 4 plus 1?
Needs rebase. Concept ACK, on the placeholder bitfield, perhaps better to make that a comment?
rebased and addressed feedback
Rebased! (removing priority conflicted with this)
88 | @@ -94,7 +89,17 @@ class CTxMemPoolEntry 89 | uint64_t nSizeWithAncestors; 90 | CAmount nModFeesWithAncestors; 91 | int64_t nSigOpCostWithAncestors; 92 | + size_t nTxWeight; //!< ... and avoid recomputing tx weight (also used for GetTxSize()) 93 | + size_t nModSize; //!< ... and modified size for priority
In commit "Re-pack the CTxMempoolEntry to be closer to optimally packed":
This isn't used, guessing bad rebase.
yeah... hmm :/
Will fix.
96 | + mutable size_t vTxHashesIdx; //!< Index in mempool's vTxHashes 97 | +private: 98 | + unsigned int entryHeight; //!< Chain height when entering the mempool 99 | 100 | + // Bool flags 101 | + bool hadNoDependencies; //!< Not dependent on any other txs when it entered the mempool
In commit "Re-pack the CTxMempoolEntry to be closer to optimally packed":
Also unused.
75 | @@ -76,9 +76,12 @@ uint256 CTransaction::GetWitnessHash() const 76 | } 77 | 78 | /* For backward compatibility, the hash is initialized to 0. TODO: remove the need for this default constructor entirely. */ 79 | -CTransaction::CTransaction() : nVersion(CTransaction::CURRENT_VERSION), vin(), vout(), nLockTime(0), hash() {} 80 | -CTransaction::CTransaction(const CMutableTransaction &tx) : nVersion(tx.nVersion), vin(tx.vin), vout(tx.vout), nLockTime(tx.nLockTime), hash(ComputeHash()) {}
In commit "Reorder C{,Mutable}Transaction for better packing":
Seems like you could remove ComputeHash now, it doesn't appear to be called anywhere else.
105 | // Bool flags 106 | - bool hadNoDependencies; //!< Not dependent on any other txs when it entered the mempool 107 | - bool spendsCoinbase; //!< keep track of transactions that spend a coinbase 108 | + struct CTxMemPoolEntryFlags { 109 | + //int UNUSED_BITS : 7; 110 | + bool spendsCoinbase : 1; //!< keep track of transactions that spend a coinbase
Kind of a lonely bitfield now with just one bool. But maybe it will have some more flags later.
yeah, hopefully inspires whoever adds a flag in the future...
19 | @@ -20,9 +20,10 @@ 20 | 21 | CTxMemPoolEntry::CTxMemPoolEntry(const CTransactionRef& _tx, const CAmount& _nFee, 22 | int64_t _nTime, unsigned int _entryHeight, 23 | - bool _spendsCoinbase, int64_t _sigOpsCost, LockPoints lp): 24 | - tx(_tx), nFee(_nFee), nTime(_nTime), entryHeight(_entryHeight), 25 | - spendsCoinbase(_spendsCoinbase), sigOpCost(_sigOpsCost), lockPoints(lp) 26 | + bool _spendsCoinbase, int64_t _sigOpsCost, LockPoints lp): 27 | +lockPoints(lp), tx(_tx), nFee(_nFee)
In commit "Re-pack the CTxMempoolEntry to be closer to optimally packed"
Unusual indentation here, maybe reformat.
105 | + // Bool flags 106 | + struct CTxMemPoolEntryFlags { 107 | + //int UNUSED_BITS : 7; 108 | + bool spendsCoinbase : 1; //!< keep track of transactions that spend a coinbase 109 | + }; 110 | + CTxMemPoolEntryFlags flags;
Ehh, why not just have the single bool spendsCoinbase : 1 here instead of making a struct for it?
It's somewhat vestigial; iirc when I originally wrote this patch last year there was another bool that got removed in a later PR.
The only real remaining reason is to make the code "self documenting" so that when/if someone later adds another flag to mempool (or even adds the flag back), they are encouraged to dtrt rather than adding more memory bloat.
Is there harm in having bool spendsCoinbase : 1; bool newFlags : 1; instead of encapsulating the booleans in a flags struct?
Is that valid c++? If so, o_0
In what way is that different from what you're doing now?
This reduces CTransaction from 96 bytes to 88 bytes for me (great), but increases CTxMemPoolEntry from 160 bytes to 168 bytes (not so great). NACK as long as that is the case.
@sipa I rewrote this PR to repack the CTransaction versions via a minimal patch.
I'm not sure what was going on with the CTxMemPoolEntry version, I just think it became stale. At the time I wrote the patch initially they were 192 bytes, so 160 bytes/entry is a good improvement anyways (perhaps from removing priority).
utACK 37495e0d8d4bd98ae04364eae2f4ecb7084a9234. Verified with printf("%i\n", sizeof(CTransaction)) before and after that this indeed reduces the size of the type.
utACK 37495e0d8d4bd98ae04364eae2f4ecb7084a9234
utACK 37495e0d8d4bd98ae04364eae2f4ecb7084a9234. Should update outdated PR description though to avoid misleading text in git history.
PR text updates done. Original text preserved below.
These commits revise the layout of a few key classes to eliminate padding, eliminating useless memory overhead.
CTxMemPoolEntry is decreased to 184 bytes from 192. CTransaction is decreased to 112 bytes from 120.
Furthermore, it is clarified in CTxMempoolEntry where the extra space is (2 bytes) for future overhead-free use.
The introduction of CTxMemPoolEntryFlags is used to bit-pack the bool flags, making room for 6 more future-flags. I have not performance tested this, but it should not be much. It does not save any space at the moment due to padding, but if more bools are added it would compared to naive.
To finish this work, the following should be done: profiling of access patterns to class fields to guarantee cache-line co-residency, cross architecture padding optimizations.
utACK
@laanwj merge?