contrib: Count entry differences in asmap-tool diff summary

fjahr commented at 1:39 pm on November 24, 2025: contributor

Currently the output of asmap-tool.py diff returns the total number of addresses that has changed at the end of the list.

Example output currently:

02602:feda:c0::/48 AS1029 # was AS43126
12604:7c00:100::/40 AS29802 # was AS40244
2# 0 IPv4 addresses changed; 79552154633921058212365205504 (2^96.01) IPv6 addresses changed

This is good indicator but in case of a longer list I would like the number of changed entries as well, since that is an easier number to parse and for debugging of certain issues also the more relevant value. This PR adds the count of changed entries to this summary output at the end. There as also a bit more structure so it’s easier to parse as well.

Example new output:

02602:feda:c0::/48 AS1029 # was AS43126
12604:7c00:100::/40 AS29802 # was AS40244
2# Summary
3IPv4: 0 entries with 0 addresses changed
4IPv6: 12 entries with 79552154633921058212365205504 (2^96.01) addresses changed

DrahtBot added the label Scripts and tools on Nov 24, 2025

DrahtBot commented at 1:39 pm on November 24, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33939.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	janb84, hodlinator

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

fanquake commented at 4:20 pm on November 25, 2025: member

cc @hodlinator

in contrib/asmap/asmap-tool.py:158 in 996660c1e8

153             elif isinstance(net, ipaddress.IPv6Network):
154                 ipv6_changed += 1 << (128 - net.prefixlen)
155+                ipv6_entries_changed += 1
156             if new_asn == 0:
157                 print(f"# {net} was AS{old_asn}")
158             elif old_asn == 0:

janb84 commented at 8:33 am on November 26, 2025:

NIT: since we are touching the code, it would be nice to update the code to be a little bit more performant (and modern)

 0            net = asmap.prefix_to_net(prefix)
 1            num_addrs = net.num_addresses
 2            if isinstance(net, ipaddress.IPv4Network):
 3                ipv4_changed += num_addrs
 4                ipv4_entries_changed += 1
 5            else:  # IPv6Network
 6                ipv6_changed += num_addrs
 7                ipv6_entries_changed += 1
 8            if new_asn == 0:
 9                print(f"# {net} was AS{old_asn}")
10            elif old_asn == 0:

fjahr commented at 2:46 pm on November 26, 2025:

Ah, neat, I am sure I knew num_addresses existed. Updated the code to use it now and confirmed the result is the same as before for a diff between two different asmap files.

janb84 commented at 8:55 am on November 26, 2025: contributor

utACK 996660c1e8b0f221012ac0662fd05c3c85b96888

PR changes format of and adds counters of ip entries to te summary.

During my code review I noticed that it uses bit shifting in stead of the built-in property of Python’s standard library ipaddress module. Since we are already importing it why not use it? (should be slightly more performant also) And we are already touching the code, this would be a nice time to change it. Therefor I added the NIT.

contrib: Count entry differences in asmap-tool diff summary

Also uses num_addresses from ipaddress instead of calculating it
ourselves.

fd4ce55121

fjahr force-pushed on Nov 26, 2025

janb84 commented at 3:14 pm on November 26, 2025: contributor

utACK fd4ce55121e7b0fe0e5b1ecf648dc3178ed37fd8

changes since last ACK:

changes to use net.num_addresses

(thanks for using my suggestion! )

in contrib/asmap/asmap-tool.py:167 in fd4ce55121

162@@ -159,8 +163,9 @@ def main():
163         ipv6_change_str = "" if ipv6_changed == 0 else f" (2^{math.log2(ipv6_changed):.2f})"
164 
165         print(
166-            f"# {ipv4_changed}{ipv4_change_str} IPv4 addresses changed; "
167-            f"{ipv6_changed}{ipv6_change_str} IPv6 addresses changed"
168+f"""# Summary
169+IPv4: {ipv4_entries_changed} entries with {ipv4_changed}{ipv4_change_str} addresses changed

hodlinator commented at 4:10 pm on November 26, 2025:

suggestion: Would be nice to output how many entries there are in total, something like

0Total entries - {args.infile1.name}: {len(state1.to_entries(overlapping=False))}, {args.infile2.name}: {len(state2.to_entries(overlapping=False))}
1->
2Total entries - 1762865730.dat: 706503, 1762878275.dat: 698844

But adding that makes it run 7 seconds slower on top of the ~29 seconds I got otherwise. Maybe there’s some other way to get an approximate value.

fjahr commented at 0:35 am on November 27, 2025:

I will look into this if I have to retouch. Usually the total count isn’t that interesting of a number to me in analysis. But I can see it’s a nice to have maybe with some percentage value as well. The calculation could certainly be much faster if the file is text based, then it’s just a line count. But for the binary it’s trickier of course and I am not sure we can do better.

hodlinator approved

hodlinator commented at 4:11 pm on November 26, 2025: contributor

ACK fd4ce55121e7b0fe0e5b1ecf648dc3178ed37fd8

Tested by diffing 2 previously recorded¹ and encoded² Kartograf outputs:

0./contrib/asmap/asmap-tool.py diff 1762865730.dat 1762878275.dat

master

0...
1# 96634298 (2^26.53) IPv4 addresses changed; 2116571614109896006051667873103669 (2^110.71) IPv6 addresses changed

fd4ce55121e7b0fe0e5b1ecf648dc3178ed37fd8

0...
1# Summary
2IPv4: 35383 entries with 96634298 (2^26.53) addresses changed
3IPv6: 37688 entries with 2116571614109896006051667873103669 (2^110.71) addresses changed

kartograf/run map -rv -irr ↩︎
/contrib/asmap/asmap-tool.py encode ~/kartograf/out/1762865730/final_result.txt 1762865730.dat ↩︎

contrib: Count entry differences in asmap-tool diff summary #33939

Code Coverage & Benchmarks

Reviews

master

fd4ce55121e7b0fe0e5b1ecf648dc3178ed37fd8