Fetch Headers over DNS

TheBlueMatt commented at 8:35 pm on September 8, 2019: member

This allows us to fetch headers from the DNS. While this isn’t very useful on its own, its a somewhat cool demo of how low the review burden could be to add new fetch methods in Rust code, and could be a useful to know whether we’re behind (eg for stale tip checking). You can test with -headersfetchdns=headers.bitcoinheaders.net which is served from a standard bind named with the following generation scripts:

0for I in {0..100}; do
1	echo "" > headers-$I.zone
2done
3COUNT=$(~/bitcoin-cli getblockcount)
4
5for I in `seq 0 2000 $COUNT`; do
6	HASH=$(~/bitcoin-cli getblockhash $I)
7	wget -q -O - http://127.0.0.1/rest/headers/2000/$HASH.hex | ./split $I $COUNT >> headers-$(($I / 10000)).zone
8done

 0#include <stdio.h>
 1#include <string.h>
 2#include <stdlib.h>
 3#include <assert.h>
 4
 5// Primes with 2 replaced with 1:
 6static const int PRIMES[] = {1, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379, 383, 389, 397, 401, 409, 419, 421, 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491, 499, 503, 509, 521, 523, 541, 547, 557, 563, 569, 571, 577, 587, 593, 599, 601, 607, 613, 617, 619, 631, 641, 643, 647, 653, 659, 661, 673, 677, 683, 691, 701, 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773, 787, 797, 809, 811, 821, 823, 827, 829, 839, 853, 857, 859, 863, 877, 881, 883, 887, 907, 911, 919, 929, 937, 941, 947, 953, 967, 971, 977, 983, 991, 997, 1009, 1013, 1019, 1021, 1031, 1033, 1039, 1049, 1051, 1061, 1063, 1069, 1087, 1091, 1093, 1097, 1103, 1109, 1117, 1123, 1129, 1151, 1153, 1163, 1171, 1181, 1187, 1193, 1201, 1213, 1217, 1223, 1229, 1231, 1237, 1249, 1259, 1277, 1279, 1283, 1289, 1291, 1297, 1301, 1303, 1307, 1319, 1321, 1327, 1361, 1367, 1373, 1381, 1399, 1409, 1423, 1427, 1429, 1433, 1439, 1447, 1451, 1453, 1459, 1471, 1481, 1483, 1487, 1489, 1493, 1499, 1511, 1523, 1531, 1543, 1549, 1553, 1559, 1567, 1571, 1579, 1583, 1597, 1601, 1607, 1609, 1613, 1619, 1621, 1627, 1637, 1657, 1663, 1667, 1669, 1693, 1697, 1699, 1709, 1721, 1723, 1733, 1741, 1747, 1753, 1759, 1777, 1783, 1787, 1789, 1801, 1811, 1823, 1831, 1847, 1861, 1867, 1871, 1873, 1877, 1879, 1889, 1901, 1907, 1913, 1931, 1933, 1949, 1951, 1973, 1979, 1987, 1993, 1997, 1999, 2003, 2011, 2017, 2027, 2029, 2039, 2053, 2063, 2069, 2081, 2083, 2087, 2089, 2099, 2111, 2113, 2129, 2131, 2137, 2141, 2143, 2153, 2161, 2179, 2203, 2207, 2213, 2221, 2237, 2239, 2243, 2251, 2267, 2269, 2273, 2281, 2287, 2293, 2297, 2309, 2311, 2333, 2339, 2341, 2347, 2351, 2357, 2371, 2377, 2381, 2383, 2389, 2393, 2399, 2411, 2417, 2423, 2437, 2441, 2447, 2459, 2467, 2473, 2477, 2503, 2521, 2531, 2539, 2543, 2549, 2551, 2557, 2579, 2591, 2593, 2609, 2617, 2621, 2633, 2647, 2657, 2659, 2663, 2671, 2677, 2683, 2687, 2689, 2693, 2699, 2707, 2711, 2713, 2719, 2729, 2731, 2741, 2749, 2753, 2767, 2777, 2789, 2791, 2797, 2801, 2803, 2819, 2833, 2837, 2843, 2851, 2857, 2861, 2879, 2887, 2897, 2903, 2909, 2917, 2927, 2939, 2953, 2957, 2963, 2969, 2971, 2999, 3001, 3011, 3019, 3023, 3037, 3041, 3049, 3061, 3067, 3079, 3083, 3089, 3109, 3119, 3121, 3137, 3163, 3167, 3169, 3181, 3187, 3191, 3203, 3209, 3217, 3221, 3229, 3251, 3253, 3257, 3259, 3271, 3299, 3301, 3307, 3313, 3319, 3323, 3329, 3331, 3343, 3347, 3359, 3361, 3371, 3373, 3389, 3391, 3407, 3413, 3433, 3449, 3457, 3461, 3463, 3467, 3469, 3491, 3499, 3511, 3517, 3527, 3529, 3533, 3539, 3541, 3547, 3557, 3559, 3571, 3581, 3583, 3593, 3607, 3613, 3617, 3623, 3631, 3637, 3643, 3659, 3671, 3673, 3677, 3691, 3697, 3701, 3709, 3719, 3727, 3733, 3739, 3761, 3767, 3769, 3779, 3793, 3797, 3803, 3821, 3823, 3833, 3847, 3851, 3853, 3863, 3877, 3881, 3889, 3907, 3911, 3917, 3919, 3923, 3929, 3931, 3943, 3947, 3967, 3989, 4001, 4003, 4007, 4013, 4019, 4021, 4027, 4049, 4051, 4057, 4073, 4079, 4091, 4093, 4099, 4111, 4127, 4129, 4133, 4139, 4153, 4157, 4159, 4177, 4201, 4211, 4217, 4219, 4229, 4231, 4241, 4243, 4253, 4259, 4261, 4271, 4273, 4283, 4289, 4297, 4327, 4337, 4339, 4349, 4357, 4363, 4373, 4391, 4397, 4409, 4421, 4423, 4441, 4447, 4451, 4457, 4463, 4481, 4483, 4493, 4507, 4513, 4517, 4519, 4523, 4547, 4549, 4561, 4567, 4583, 4591, 4597, 4603, 4621, 4637, 4639, 4643, 4649, 4651, 4657, 4663, 4673, 4679, 4691, 4703, 4721, 4723, 4729, 4733, 4751, 4759, 4783, 4787, 4789, 4793, 4799, 4801, 4813, 4817, 4831, 4861, 4871, 4877, 4889, 4903, 4909, 4919, 4931, 4933, 4937, 4943, 4951, 4957, 4967, 4969, 4973, 4987, 4993, 4999, 5003, 5009, 5011, 5021, 5023, 5039, 5051, 5059, 5077, 5081, 5087, 5099, 5101, 5107, 5113, 5119, 5147, 5153, 5167, 5171, 5179, 5189, 5197, 5209, 5227, 5231, 5233, 5237, 5261, 5273, 5279, 5281, 5297, 5303, 5309, 5323, 5333, 5347, 5351, 5381, 5387, 5393, 5399, 5407, 5413, 5417, 5419, 5431, 5437, 5441, 5443, 5449, 5471, 5477, 5479, 5483, 5501, 5503, 5507, 5519, 5521, 5527, 5531, 5557, 5563, 5569, 5573, 5581, 5591, 5623, 5639, 5641, 5647, 5651, 5653, 5657, 5659, 5669, 5683, 5689, 5693, 5701, 5711, 5717, 5737, 5741, 5743, 5749, 5779, 5783, 5791, 5801, 5807, 5813, 5821, 5827, 5839, 5843, 5849, 5851, 5857, 5861, 5867, 5869, 5879, 5881, 5897, 5903, 5923, 5927, 5939, 5953, 5981, 5987, 6007, 6011, 6029, 6037, 6043, 6047, 6053, 6067, 6073, 6079, 6089, 6091, 6101, 6113, 6121, 6131, 6133, 6143, 6151, 6163, 6173, 6197, 6199, 6203, 6211, 6217, 6221, 6229, 6247, 6257, 6263, 6269, 6271, 6277, 6287, 6299, 6301, 6311, 6317, 6323, 6329, 6337, 6343, 6353, 6359, 6361, 6367, 6373, 6379, 6389, 6397, 6421, 6427, 6449, 6451, 6469, 6473, 6481, 6491, 6521, 6529, 6547, 6551, 6553, 6563, 6569, 6571, 6577, 6581, 6599, 6607, 6619, 6637, 6653, 6659, 6661, 6673, 6679, 6689, 6691, 6701, 6703, 6709, 6719, 6733, 6737, 6761, 6763, 6779, 6781, 6791, 6793, 6803, 6823, 6827, 6829, 6833, 6841, 6857, 6863, 6869, 6871, 6883, 6899, 6907, 6911, 6917, 6947, 6949, 6959, 6961, 6967, 6971, 6977, 6983, 6991, 6997, 7001, 7013, 7019, 7027, 7039, 7043, 7057, 7069, 7079, 7103, 7109, 7121, 7127, 7129, 7151, 7159, 7177, 7187, 7193, 7207, 7211, 7213, 7219, 7229, 7237, 7243, 7247, 7253, 7283, 7297, 7307, 7309, 7321, 7331, 7333, 7349, 7351, 7369, 7393, 7411, 7417, 7433, 7451, 7457, 7459, 7477, 7481, 7487, 7489, 7499, 7507, 7517, 7523, 7529, 7537, 7541, 7547, 7549, 7559, 7561, 7573, 7577, 7583, 7589, 7591, 7603, 7607, 7621, 7639, 7643, 7649, 7669, 7673, 7681, 7687, 7691, 7699, 7703, 7717, 7723, 7727, 7741, 7753, 7757, 7759, 7789, 7793, 7817, 7823, 7829, 7841, 7853, 7867, 7873, 7877, 7879, 7883, 7901, 7907, 7919, 7927, 7933, 7937, 7949, 7951, 7963, 7993, 8009, 8011, 8017, 8039, 8053, 8059, 8069, 8081, 8087, 8089, 8093, 8101, 8111, 8117, 8123, 8147, 8161, 8167, 8171, 8179, 8191, 8209, 8219, 8221, 8231, 8233, 8237, 8243, 8263, 8269, 8273, 8287, 8291, 8293, 8297, 8311, 8317, 8329, 8353, 8363, 8369, 8377, 8387, 8389, 8419, 8423, 8429, 8431, 8443, 8447, 8461, 8467, 8501, 8513, 8521, 8527, 8537, 8539, 8543, 8563, 8573, 8581, 8597, 8599, 8609, 8623, 8627, 8629, 8641, 8647, 8663, 8669, 8677, 8681, 8689, 8693, 8699, 8707, 8713, 8719, 8731, 8737, 8741, 8747, 8753, 8761, 8779, 8783, 8803, 8807, 8819, 8821, 8831, 8837, 8839, 8849, 8861, 8863, 8867, 8887, 8893, 8923, 8929, 8933, 8941, 8951, 8963, 8969, 8971, 8999, 9001, 9007, 9011, 9013, 9029, 9041, 9043, 9049, 9059, 9067, 9091, 9103, 9109, 9127, 9133, 9137, 9151, 9157, 9161, 9173, 9181, 9187, 9199, 9203, 9209, 9221, 9227, 9239, 9241, 9257, 9277, 9281, 9283, 9293, 9311, 9319, 9323, 9337, 9341, 9343, 9349, 9371, 9377, 9391, 9397, 9403, 9413, 9419, 9421, 9431, 9433, 9437, 9439, 9461, 9463, 9467, 9473, 9479, 9491, 9497, 9511, 9521, 9533, 9539, 9547, 9551, 9587, 9601, 9613, 9619, 9623, 9629, 9631, 9643, 9649, 9661, 9677, 9679, 9689, 9697, 9719, 9721, 9733, 9739, 9743, 9749, 9767, 9769, 9781, 9787, 9791, 9803, 9811, 9817, 9829, 9833, 9839, 9851, 9857, 9859, 9871, 9883, 9887, 9901, 9907, 9923, 9929, 9931, 9941, 9949, 9967, 9973, 10007, 10009, 10037, 10039, 10061, 10067, 10069, 10079, 10091, 10093, 10099, 10103, 10111, 10133, 10139, 10141, 10151, 10159, 10163, 10169, 10177, 10181, 10193, 10211, 10223, 10243, 10247, 10253, 10259, 10267, 10271, 10273, 10289, 10301, 10303, 10313, 10321, 10331, 10333, 10337, 10343, 10357, 10369, 10391, 10399, 10427, 10429, 10433, 10453, 10457, 10459, 10463, 10477, 10487, 10499, 10501, 10513, 10529, 10531, 10559, 10567, 10589, 10597, 10601, 10607, 10613, 10627, 10631, 10639, 10651, 10657, 10663, 10667, 10687, 10691, 10709, 10711, 10723, 10729, 10733, 10739, 10753, 10771, 10781, 10789, 10799, 10831, 10837, 10847, 10853, 10859, 10861, 10867, 10883, 10889, 10891, 10903, 10909, 10937, 10939, 10949, 10957, 10973, 10979, 10987, 10993, 11003, 11027, 11047, 11057, 11059, 11069, 11071, 11083, 11087, 11093, 11113, 11117, 11119, 11131, 11149, 11159, 11161, 11171, 11173, 11177, 11197, 11213, 11239, 11243, 11251, 11257, 11261, 11273, 11279, 11287, 11299, 11311, 11317, 11321, 11329, 11351, 11353, 11369, 11383, 11393, 11399, 11411, 11423, 11437, 11443, 11447, 11467, 11471, 11483, 11489, 11491, 11497, 11503, 11519, 11527, 11549, 11551, 11579, 11587, 11593, 11597, 11617, 11621, 11633, 11657, 11677, 11681, 11689, 11699, 11701, 11717, 11719, 11731, 11743, 11777, 11779, 11783, 11789, 11801, 11807, 11813, 11821, 11827, 11831, 11833, 11839, 11863, 11867, 11887, 11897, 11903, 11909, 11923, 11927, 11933, 11939, 11941, 11953, 11959, 11969, 11971, 11981, 11987, 12007, 12011, 12037, 12041, 12043, 12049, 12071, 12073, 12097, 12101, 12107, 12109, 12113, 12119, 12143, 12149, 12157, 12161, 12163, 12197, 12203, 12211, 12227, 12239, 12241, 12251, 12253, 12263, 12269, 12277, 12281, 12289, 12301, 12323, 12329, 12343, 12347, 12373, 12377, 12379, 12391, 12401, 12409, 12413, 12421, 12433, 12437, 12451, 12457, 12473, 12479, 12487, 12491, 12497, 12503, 12511, 12517, 12527, 12539, 12541, 12547, 12553, 12569, 12577, 12583, 12589, 12601, 12611, 12613, 12619, 12637, 12641, 12647, 12653, 12659, 12671, 12689, 12697, 12703, 12713, 12721, 12739, 12743, 12757, 12763, 12781, 12791, 12799, 12809, 12821, 12823, 12829, 12841, 12853, 12889, 12893, 12899, 12907, 12911, 12917, 12919, 12923, 12941, 12953, 12959, 12967, 12973, 12979, 12983, 13001, 13003, 13007, 13009, 13033, 13037, 13043, 13049, 13063, 13093, 13099, 13103, 13109, 13121, 13127, 13147, 13151, 13159, 13163, 13171, 13177, 13183, 13187, 13217, 13219, 13229, 13241, 13249, 13259, 13267, 13291, 13297, 13309, 13313, 13327, 13331, 13337, 13339, 13367, 13381, 13397, 13399, 13411, 13417, 13421, 13441, 13451, 13457, 13463, 13469, 13477, 13487, 13499, 13513, 13523, 13537, 13553, 13567, 13577, 13591, 13597, 13613, 13619, 13627, 13633, 13649, 13669, 13679, 13681, 13687, 13691, 13693, 13697, 13709, 13711, 13721, 13723, 13729, 13751, 13757, 13759, 13763, 13781, 13789, 13799, 13807, 13829, 13831, 13841, 13859, 13873, 13877, 13879, 13883, 13901, 13903, 13907, 13913, 13921, 13931, 13933, 13963, 13967, 13997, 13999, 14009, 14011, 14029, 14033, 14051, 14057, 14071, 14081, 14083, 14087, 14107, 14143, 14149, 14153, 14159, 14173, 14177, 14197, 14207, 14221, 14243, 14249, 14251, 14281, 14293, 14303, 14321, 14323, 14327, 14341, 14347, 14369, 14387, 14389, 14401, 14407, 14411, 14419, 14423, 14431, 14437, 14447, 14449, 14461, 14479, 14489, 14503, 14519, 14533, 14537, 14543, 14549, 14551, 14557, 14561, 14563, 14591, 14593, 14621, 14627, 14629, 14633, 14639, 14653, 14657, 14669, 14683, 14699, 14713, 14717, 14723, 14731, 14737, 14741, 14747, 14753, 14759, 14767, 14771, 14779, 14783, 14797, 14813, 14821, 14827, 14831, 14843, 14851, 14867, 14869, 14879, 14887, 14891, 14897, 14923, 14929, 14939, 14947, 14951, 14957, 14969, 14983, 15013, 15017, 15031, 15053, 15061, 15073, 15077, 15083, 15091, 15101, 15107, 15121, 15131, 15137, 15139, 15149, 15161, 15173, 15187, 15193, 15199, 15217, 15227, 15233, 15241, 15259, 15263, 15269, 15271, 15277, 15287, 15289, 15299, 15307, 15313, 15319, 15329, 15331, 15349, 15359, 15361, 15373, 15377, 15383, 15391, 15401, 15413, 15427, 15439, 15443, 15451, 15461, 15467, 15473, 15493, 15497, 15511, 15527, 15541, 15551, 15559, 15569, 15581, 15583, 15601, 15607, 15619, 15629, 15641, 15643, 15647, 15649, 15661, 15667, 15671, 15679, 15683, 15727, 15731, 15733, 15737, 15739, 15749, 15761, 15767, 15773, 15787, 15791, 15797, 15803, 15809, 15817, 15823, 15859, 15877, 15881, 15887, 15889, 15901, 15907, 15913, 15919, 15923, 15937, 15959, 15971, 15973, 15991, 16001, 16007, 16033, 16057, 16061, 16063, 16067, 16069, 16073, 16087, 16091, 16097, 16103, 16111, 16127, 16139, 16141, 16183, 16187, 16189, 16193, 16217, 16223, 16229, 16231, 16249, 16253, 16267, 16273, 16301, 16319, 16333, 16339, 16349, 16361, 16363, 16369, 16381, 16411, 16417, 16421, 16427, 16433, 16447, 16451, 16453, 16477, 16481, 16487, 16493, 16519, 16529, 16547, 16553, 16561, 16567, 16573, 16603, 16607, 16619, 16631, 16633, 16649, 16651, 16657, 16661, 16673, 16691, 16693, 16699, 16703, 16729, 16741, 16747, 16759, 16763, 16787, 16811, 16823, 16829, 16831, 16843, 16871, 16879, 16883, 16889, 16901, 16903, 16921, 16927, 16931, 16937, 16943, 16963, 16979, 16981, 16987, 16993, 17011, 17021, 17027, 17029, 17033, 17041, 17047, 17053, 17077, 17093, 17099, 17107, 17117, 17123, 17137, 17159, 17167, 17183, 17189, 17191, 17203, 17207, 17209, 17231, 17239, 17257, 17291, 17293, 17299, 17317, 17321, 17327, 17333, 17341, 17351, 17359, 17377, 17383, 17387, 17389, 17393, 17401, 17417, 17419, 17431, 17443, 17449, 17467, 17471, 17477, 17483, 17489, 17491, 17497, 17509, 17519, 17539, 17551, 17569, 17573, 17579, 17581, 17597, 17599, 17609, 17623, 17627, 17657, 17659, 17669, 17681, 17683, 17707, 17713, 17729, 17737, 17747, 17749, 17761, 17783, 17789, 17791, 17807, 17827, 17837, 17839, 17851, 17863, 17881, 17891, 17903, 17909, 17911, 17921, 17923, 17929, 17939, 17957, 17959, 17971, 17977, 17981, 17987, 17989, 18013, 18041, 18043, 18047, 18049, 18059, 18061, 18077, 18089, 18097, 18119, 18121, 18127, 18131, 18133, 18143, 18149, 18169, 18181, 18191, 18199, 18211, 18217, 18223, 18229, 18233, 18251, 18253, 18257, 18269, 18287, 18289, 18301, 18307, 18311, 18313, 18329, 18341, 18353, 18367, 18371, 18379, 18397, 18401, 18413, 18427, 18433, 18439, 18443, 18451, 18457, 18461, 18481, 18493, 18503, 18517, 18521, 18523, 18539, 18541, 18553, 18583, 18587, 18593, 18617, 18637, 18661, 18671, 18679, 18691, 18701, 18713, 18719, 18731, 18743, 18749, 18757, 18773, 18787, 18793, 18797, 18803, 18839, 18859, 18869, 18899, 18911, 18913, 18917, 18919, 18947, 18959, 18973, 18979, 19001, 19009, 19013, 19031, 19037, 19051, 19069, 19073, 19079, 19081, 19087, 19121, 19139, 19141, 19157, 19163, 19181, 19183, 19207, 19211, 19213, 19219, 19231, 19237, 19249, 19259, 19267, 19273, 19289, 19301, 19309, 19319, 19333, 19373, 19379, 19381, 19387, 19391, 19403, 19417, 19421, 19423, 19427, 19429, 19433, 19441, 19447, 19457, 19463, 19469, 19471, 19477, 19483, 19489, 19501, 19507, 19531, 19541, 19543, 19553, 19559, 19571, 19577, 19583, 19597, 19603, 19609, 19661, 19681, 19687, 19697, 19699, 19709, 19717, 19727, 19739, 19751, 19753, 19759, 19763, 19777, 19793, 19801, 19813, 19819, 19841, 19843, 19853, 19861, 19867, 19889, 19891, 19913, 19919, 19927, 19937, 19949, 19961, 19963, 19973, 19979, 19991, 19993, 19997, 20011, 20021, 20023, 20029, 20047, 20051, 20063, 20071, 20089, 20101, 20107, 20113, 20117, 20123, 20129, 20143, 20147, 20149, 20161, 20173, 20177, 20183, 20201, 20219, 20231, 20233, 20249, 20261, 20269, 20287, 20297, 20323, 20327, 20333, 20341, 20347, 20353, 20357, 20359, 20369, 20389, 20393, 20399, 20407, 20411, 20431, 20441, 20443, 20477, 20479, 20483, 20507, 20509, 20521, 20533, 20543, 20549, 20551, 20563, 20593, 20599, 20611, 20627, 20639, 20641, 20663, 20681, 20693, 20707, 20717, 20719, 20731, 20743, 20747, 20749, 20753, 20759, 20771, 20773, 20789, 20807, 20809, 20849, 20857, 20873, 20879, 20887, 20897, 20899, 20903, 20921, 20929, 20939, 20947, 20959, 20963, 20981, 20983, 21001, 21011, 21013, 21017, 21019, 21023, 21031, 21059, 21061, 21067, 21089, 21101, 21107, 21121, 21139, 21143, 21149, 21157, 21163, 21169, 21179, 21187, 21191, 21193, 21211, 21221, 21227, 21247, 21269, 21277, 21283, 21313, 21317, 21319, 21323, 21341, 21347, 21377, 21379, 21383, 21391, 21397, 21401, 21407, 21419, 21433, 21467, 21481, 21487, 21491, 21493, 21499, 21503, 21517, 21521, 21523, 21529, 21557, 21559, 21563, 21569, 21577, 21587, 21589, 21599, 21601, 21611, 21613, 21617, 21647, 21649, 21661, 21673, 21683, 21701, 21713, 21727, 21737, 21739, 21751, 21757, 21767, 21773, 21787, 21799, 21803, 21817, 21821, 21839, 21841, 21851, 21859, 21863, 21871, 21881, 21893, 21911, 21929, 21937, 21943, 21961, 21977, 21991, 21997, 22003, 22013, 22027, 22031, 22037, 22039, 22051, 22063, 22067, 22073, 22079, 22091, 22093, 22109, 22111, 22123, 22129, 22133, 22147, 22153, 22157, 22159, 22171, 22189, 22193, 22229, 22247, 22259, 22271, 22273, 22277, 22279, 22283, 22291, 22303, 22307};
 7
 8
 9int main(int argc, char* argv[]) {
10	assert(argc >= 3);
11	int start_height = atoi(argv[1]);
12	int total_height = atoi(argv[2]);
13	// We don't want to constantly update old header zones, but we do want their TTLs
14	// to get updated slowly, so we update zones on prime intervals of block count:
15	int chunk = start_height / 10000;
16	int chunk_count = total_height / 10000;
17	int update_every = PRIMES[(chunk_count - chunk) * 5];
18	int ttl = ((total_height - start_height) / update_every) * update_every;
19	if (ttl > update_every * 60 * 10 && ttl > 86400/2) {
20		fprintf(stderr, "Bad TTL %d for updating every %d blocks (%d sec)\n", ttl, update_every, update_every * 60 * 10);
21		ttl = update_every * 60 * 10;
22	}
23
24	char* hex = NULL;
25	size_t linelen = 0;
26	getline(&hex, &linelen, stdin);
27	size_t len = strlen(hex);
28	for (int i = 0; i < len / 80 / 2; i++) {
29		for (int j = 0; j < 6; j++) {
30			char fmt[7*5];
31			memset(fmt, 0, sizeof(fmt));
32			size_t offs = 0;
33			if (j == 0) {
34				fmt[0] = '0';
35				fmt[1] = '0';
36				fmt[2] = hex[i * 80*2];
37				offs = 1;
38			} else {
39				offs = (j-1)*27 + 25;
40				memcpy(fmt, &hex[i * 80*2 + offs], 3);
41				offs += 3;
42			}
43			memcpy(fmt + 1*5, &hex[i * 80*2 + 0*4 + offs], 4);
44			memcpy(fmt + 2*5, &hex[i * 80*2 + 1*4 + offs], 4);
45			memcpy(fmt + 3*5, &hex[i * 80*2 + 2*4 + offs], 4);
46			memcpy(fmt + 4*5, &hex[i * 80*2 + 3*4 + offs], 4);
47			memcpy(fmt + 5*5, &hex[i * 80*2 + 4*4 + offs], 4);
48			memcpy(fmt + 6*5, &hex[i * 80*2 + 5*4 + offs], 4);
49			int zttl = ttl - i;
50			if (zttl < 60) { zttl = 60; }
51			printf("%d	%d	IN	AAAA	2001:%d%s:%s:%s:%s:%s:%s:%s\n",
52				start_height + i, zttl, j,
53				&fmt[0 * 5],
54				&fmt[1 * 5],
55				&fmt[2 * 5],
56				&fmt[3 * 5],
57				&fmt[4 * 5],
58				&fmt[5 * 5],
59				&fmt[6 * 5]);
60		}
61	}
62}

DrahtBot commented at 9:10 pm on September 8, 2019: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#17783 (util: Fix -norpcwhitelist, -norpcallowip, and similar corner case behavior by ryanofsky)
#17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
#17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
#17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)
#17452 (test: update fuzz directory in .gitignore by jonatack)
#17398 (build: Update leveldb to 1.22+ by laanwj)
#17383 (Refactor: Move consts to their correct translation units by jnewbery)
#17227 (Qt: Add Android packaging support by icota)
#16722 (build: Disable warnings for leveldb subtree by default by hebasto)
#15367 (feature: Added ability for users to add a startup command by benthecarman)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

practicalswift commented at 9:28 pm on September 8, 2019: contributor

Concept ACK: really like these ideas/proof-of-concepts on how to improve block/block header transport diversity. Thanks for working on this @TheBlueMatt!

This “headers over DNS” idea could be combined with DNS over HTTPS (DoH) using widely used DoH servers.

DNS lookups are trivial to block/filter for a MITM adversary but if the user is instead doing lookups using DoH with a widely used DoH-server (think Mozilla, Google or CloudFlare or similar large scale operation) – that would be hard to null route/block without upsetting non-Bitcoin users.

DrahtBot added the label Build system on Sep 8, 2019

DrahtBot added the label P2P on Sep 8, 2019

DrahtBot added the label Validation on Sep 8, 2019

fanquake removed the label P2P on Sep 8, 2019

fanquake added the label Needs Conceptual Review on Sep 8, 2019

fanquake removed the label Build system on Sep 8, 2019

fanquake added the label P2P on Sep 8, 2019

TheBlueMatt force-pushed on Sep 9, 2019

laanwj commented at 2:14 pm on September 9, 2019: member

Interesting idea! I think this needs to be disabled when -proxy is in use, as the DNS lookups would bypass the proxy.

laanwj added this to the milestone 0.20.0 on Sep 9, 2019

laanwj commented at 5:06 pm on September 9, 2019: member

Tried to test this, but doesn’t look like bitcoinheaders.net provides headers after 60000, is this still in progress?

0$ host 59999.5.bitcoinheaders.net
159999.5.bitcoinheaders.net has IPv6 address 2001:53ef:f60e:4c5c:670f:1c16:2d0a:300
2...
3$ host 60000.6.bitcoinheaders.net
4Host 60000.6.bitcoinheaders.net not found: 2(SERVFAIL)

Can bind (or DNS, for that matter) handle huge numbers of hosts like this?

TheBlueMatt commented at 5:30 pm on September 9, 2019: member

Right, I’m currently updating the zones. Bind can’t handle huge zones so well hence why it got split up into chunks of 10k, and I’m currently slowly churning through dnssec signing for the larger zones. They should be up in a few hours.

TheBlueMatt commented at 10:04 pm on September 9, 2019: member

Should be good now. Turns out bind’s inline dnssec signing has some hangs on big zones, worked fine once I manually signed them.

On Sep 9, 2019, at 13:30, Matt Corallo notifications@github.com wrote:

Right, I’m currently updating the zones. Bind can’t handle huge zones so well hence why it got split up into chunks of 10k, and I’m currently slowly churning through dnssec signing for the larger zones. They should be up in a few hours.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

laanwj added the label Feature on Sep 30, 2019

TheBlueMatt force-pushed on Sep 30, 2019

TheBlueMatt commented at 11:48 pm on September 30, 2019: member

No longer based on #16762 (since this is simpler, so a better first-land), plus the build issue was resolved (thanks to @theuni!). Should be ready for review/merge now!

fanquake commented at 11:06 am on October 5, 2019: member

Concept ACK. Discussing with @theuni, there might be the possibility to simplify this even further by dropping the use of Cargo.

TheBlueMatt force-pushed on Oct 6, 2019

TheBlueMatt force-pushed on Oct 7, 2019

in src/rusty/src/bridge.rs:94 in a4b948b5d1 outdated

84+    }
85+
86+    pub fn hash(&self) -> [u8; 32] {
87+        let hashptr = unsafe { rusty_IndexToHash(self.index) };
88+        let mut res = [0u8; 32];
89+        unsafe { std::ptr::copy(hashptr, res.as_mut_ptr(), 32) };

elichai commented at 12:39 pm on October 8, 2019:

You use the pointer without checking for nullness, I know that you already asserted that on the C++ side, but I think we should be consistent on which side of the FFI checks for null. (and if the rust side then using NonNull everywhere forces you to check)

TheBlueMatt commented at 9:44 pm on October 10, 2019:

I added an if is_null() { unreachable!(); } which I think is the right approach - it clearly isn’t allowed to be null, but its safe and will panic inside rust.

in src/rusty/src/dns_headers.rs:24 in a4b948b5d1 outdated

11+
12+fn map_addrs_to_header(ips: &mut [Ipv6Addr]) -> [u8; 80] {
13+    let mut header = [0u8; 80];
14+    if ips.len() != 6 { return header; }
15+    ips.sort_unstable_by(|a, b| {
16+        (&(a.octets()[2] & 0xf0)).cmp(&(b.octets()[2] & 0xf0))

elichai commented at 12:54 pm on October 8, 2019:

Could you add a comment on what this sort is doing/for?

in src/rusty/src/dns_headers.rs:68 in a4b948b5d1 outdated

55+
56+static THREAD_COUNT: AtomicUsize = AtomicUsize::new(0);
57+
58+#[no_mangle]
59+pub extern "C" fn init_fetch_dns_headers(domain: *const c_char) -> bool {
60+    let domain_str: String = match unsafe { CStr::from_ptr(domain) }.to_str() {

elichai commented at 1:01 pm on October 8, 2019:

Check for nullness?

TheBlueMatt commented at 9:50 pm on October 10, 2019:

Meh? I mean its called during init from C++, it would be kinda hard to get a bug here.

in src/rusty/src/dns_headers.rs:75 in a4b948b5d1 outdated

62+        Err(_) => return false,
63+    };
64+    std::thread::spawn(move || {
65+        // Always catch panics so that even if we have some bug in our parser we don't take the
66+        // rest of Bitcoin Core down with us:
67+        THREAD_COUNT.fetch_add(1, Ordering::AcqRel);

elichai commented at 1:04 pm on October 8, 2019:

Could this also panic somehow? (it calls atomic_xadd_acqrel and I can’t find the source code for it)

TheBlueMatt commented at 9:49 pm on October 10, 2019:

I dont think so? Certainly in practice it can’t panic since it just compiles down to a few instructions, but the docs even say it wraps, which seems like the only type of panic I could imagine.

in src/rusty/src/bridge.rs:44 in a4b948b5d1 outdated

39+    let index = unsafe { rusty_ConnectHeaders(headers.as_ptr(), 80, headers.len() / 80) };
40+    if index.is_null() { None } else { Some(BlockIndex { index }) }
41+}
42+
43+/// Processes a new block, in serialized form.
44+/// blockindex_requested_by_state shouild be set *only* if the given BlockIndex was provided by

elichai commented at 1:21 pm on October 8, 2019:

nit: *should

elichai commented at 2:42 pm on October 8, 2019: contributor

Personally I really like https://doc.rust-lang.org/std/ptr/struct.NonNull.html as it gives a little bit more compile time assurance (requires you to check for nullness and after you checked you easily know you don’t need to check anymore)

Edit: I’m actually not sure anymore. Because with NonNull it’s hard to identify const VS mut pointers.

MarcoFalke commented at 7:59 pm on October 10, 2019: member

None of the travis jobs built this, nor gitian

TheBlueMatt force-pushed on Oct 10, 2019

TheBlueMatt commented at 10:43 pm on October 10, 2019: member

Should be run in travis now, but waiting to see it run.

TheBlueMatt commented at 11:03 pm on October 10, 2019: member

Cool, looks like the travis changes successfully ran the in-rust tests:

0running 1 test
1
2test dns_headers::test_map_addrs ... ok
3
4test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

in .gitignore:142 in 129880fcd8 outdated

136@@ -137,3 +137,7 @@ db4/
137 osx_volname
138 dist/
139 *.background.tiff
140+
141+# Rust
142+src/rusty/Cargo.lock

stevenroose commented at 9:06 am on October 11, 2019:

I feel like Core is an end product and thus should commit this (as opposed to libraries). This ensures this project can’t be targeted by managers of dependent crates.

fanquake commented at 12:13 pm on October 11, 2019:

We’ve dropped using Cargo entirely now, so this line can probably go away.

in src/Makefile.am:396 in 129880fcd8 outdated

363@@ -352,6 +364,13 @@ libbitcoin_wallet_tool_a_SOURCES = \
364   wallet/wallettool.cpp \
365   $(BITCOIN_CORE_H)
366 
367+LIBBITCOIN_RUSTY_SRCS= \
368+  rusty/src/lib.rs
369+
370+$(LIBBITCOIN_RUSTY): $(LIBBITCOIN_RUSTY_SRCS)

fanquake commented at 12:58 pm on October 11, 2019:

Could you add some comments here on the options your passing to rustc, like panic=unwind or --edition. Similar to some of the discussion in #17090.

in src/Makefile.am:717 in 129880fcd8 outdated

713@@ -676,6 +714,8 @@ $(top_srcdir)/$(subdir)/config/bitcoin-config.h.in:  $(am__configure_deps)
714 clean-local:
715 	-$(MAKE) -C secp256k1 clean
716 	-$(MAKE) -C univalue clean
717+	-rm -rf rusty/release

fanquake commented at 1:15 pm on October 11, 2019:

Will need to add src/rusty_test and src/rusty_test.dSYM/ somewhere in here.

in src/init.cpp:379 in 129880fcd8 outdated

374@@ -367,6 +375,9 @@ void SetupServerArgs()
375 #endif
376     gArgs.AddArg("-blockreconstructionextratxn=<n>", strprintf("Extra transactions to keep in memory for compact block reconstructions (default: %u)", DEFAULT_BLOCK_RECONSTRUCTION_EXTRA_TXN), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
377     gArgs.AddArg("-blocksonly", strprintf("Whether to reject transactions from network peers. Transactions from the wallet, RPC and relay whitelisted inbound peers are not affected. (default: %u)", DEFAULT_BLOCKSONLY), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
378+#if ENABLE_RUSTY
379+    gArgs.AddArg("-headersfetchdns=<domain>", "A domain name from which to fetch headers. eg bitcoinheaders.net", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);

fanquake commented at 1:21 pm on October 11, 2019:

I think I’d prefer we didn’t have an example domain name in here.

TheBlueMatt commented at 5:07 am on October 12, 2019:

I actually kinda prefer not, just cause you need something that works. At some point after a few of these things land I want to write up a AntiCensorship.md doc, which I suppose can contain targets for this that include a bit more text (eg “run by X, wont log, but you may get logged by Y”), at which point we can use example.com, but…whatever.

in src/rusty/src/bridge.rs:22 in 129880fcd8 outdated

17+
18+    /// Gets a CBlockIndex* pointer (casted to a c_void) representing the genesis block.
19+    /// Guaranteed to never be NULL
20+    fn rusty_GetGenesisIndex() -> *const c_void;
21+
22+    #[allow(dead_code)]

fanquake commented at 1:36 pm on October 11, 2019:

There’s a lot of #[allow(dead_code)] usage here. I assume some of it is because there’s code that was/is part of #16762? I’d prefer we didn’t introduce anything we aren’t actually using here and instead bring it in in subsequent PRs.

TheBlueMatt commented at 4:55 am on October 12, 2019:

Admittedly I was being lazy, fixed now.

in src/rusty/src/lib.rs:24 in 129880fcd8 outdated

16+//
17+// In the future, as upstream Rust explores a safer allocation API (eg the Alloc API which
18+// returns Results instead of raw pointers, or redefining the GlobalAlloc API to allow
19+// panic!()s inside of alloc calls), we should switch to those, however these APIs are
20+// currently unstable.
21+const TOTAL_MEM_LIMIT_BYTES: usize = 128 * 1024 * 1024;

fanquake commented at 2:56 pm on October 11, 2019:

refusing new allocations if it exceeds 128MB.

So when Rust tries to allocate more than 128mb bitcoind will crash? I dropped the limit to something tiny and bitcoind would crash on startup.

elichai commented at 3:28 pm on October 11, 2019:

The reason it crashed all of bitcoind when this is tiny is because this allocation is outside out catch_unwind https://github.com/bitcoin/bitcoin/pull/16834/files#diff-57a33003fdd95417a0d7d83201893c62R69

How long does https://github.com/bitcoin/bitcoin/pull/16834/files#diff-c865a8939105e6350a50af02766291b7R1841 lives?

TheBlueMatt commented at 5:04 am on October 12, 2019:

Indeed, I think its fine, but just to make your testing easier, I went ahead and wrapped the whole function in a unwind. I think we’d be ok with holding the pointer longer, but trying to think about how long a pointer lives is exactly the reason we’re writing rust code to begin with, so, please no :)

fanquake commented at 2:56 pm on October 11, 2019: member

Still Concept ACK for 0.20.0. Left some high level comments. Running some more tests / review over the weekend.

TheBlueMatt force-pushed on Oct 11, 2019

TheBlueMatt force-pushed on Oct 12, 2019

in src/rusty/src/dns_headers.rs:48 in e86a034eee outdated

42+        }
43+    }
44+    header
45+}
46+
47+#[test]

elichai commented at 2:34 pm on October 12, 2019:

feature gate this? #[cfg(test)]

TheBlueMatt commented at 7:25 pm on October 12, 2019:

Done. Double checked that you can slip non-compiling code into the function and it won’t fail to build until you run make check, and that make check still fails if you change the header.

TheBlueMatt force-pushed on Oct 12, 2019

elichai commented at 10:33 am on October 15, 2019: contributor

Concept ACK. I hope to actually test the usability of this in the following days

TheBlueMatt force-pushed on Oct 17, 2019

TheBlueMatt commented at 1:12 am on October 17, 2019: member

Fixed the outstanding issue where rust code changes (outside of lib.rs) doesn’t result in rebuild of the rust lib (caused by the migration away from cargo to rustc).

theuni commented at 6:11 pm on October 25, 2019: member

Concept ACK.

I’d really like to do a cleanup of the buildsystem changes here, as it’s a bit of a Frankenstein after a bunch of back+forth on the rust side.

But I haven’t had the chance yet, and I’ve held this up long enough already, so I can do that post-merge (assuming it’s merged) if necessary.

DrahtBot added the label Needs rebase on Oct 26, 2019

TheBlueMatt force-pushed on Oct 26, 2019

DrahtBot removed the label Needs rebase on Oct 26, 2019

TheBlueMatt force-pushed on Oct 29, 2019

in configure.ac:486 in 22de67982c outdated

481@@ -476,6 +482,24 @@ AC_ARG_WITH([daemon],
482   [build_bitcoind=$withval],
483   [build_bitcoind=yes])
484 
485+if test "x$use_rust" = xyes; then
486+  AC_PATH_PROG(RUSTC,rustc)

dongcarl commented at 7:25 pm on October 29, 2019:

I think with the check below… We neeed:

0  AC_PATH_PROG(RUSTC, rustc, none)

TheBlueMatt commented at 5:20 pm on October 30, 2019:

I don’t know what this means…. @theuni or should I just take it?

TheBlueMatt commented at 7:51 pm on November 1, 2019:

Oh, I upated the if to check for empty string, so I think its fine now?

theuni commented at 9:03 pm on November 1, 2019:

Thinking about this in terms of actually merging it, it’s probably a bad idea for rust support to be opportunistically compiled in if it’s auto-detected.

It’s a significant change, and IMO builders shouldn’t have to check build logs to see if they’re using rust features or not.

So… how about making this explicitly opt-in or opt-out?

TheBlueMatt commented at 3:25 am on November 3, 2019:

Hmm, lets kick this one back to a meeting. I figure some folks will squawk if suddenly Bitcoin Core fails to build cause its missing rustc.

in configure.ac:502 in 22de67982c outdated

488+    use_rust=no
489+    AC_MSG_ERROR("rustc required to enable experimental rust support.")
490+  fi
491+
492+  case $host in
493+    ${host_cpu}-w64-mingw*) RUST_TARGET=${host_cpu}-pc-windows-gnu ;;

dongcarl commented at 8:16 pm on October 29, 2019:

Let’s perform these checks with *mingw*, *linux*, and *darwin* like we do in other places, or at the very least do the right thing for triples like aarch64-linux-gnu (notice that there’s no -pc)

theuni commented at 9:09 pm on November 1, 2019:

Agree with moving this into the other sections. Want to create a patch for that?

Note that these are already expanded triples, though:

0 $ ./config.sub aarch64-linux-gnu
1aarch64-unknown-linux-gnu

in configure.ac:500 in 22de67982c outdated

495+    ${host_cpu}-apple-darwin*) RUST_TARGET=${host_cpu}-apple-darwin ;;
496+    *) RUST_TARGET="${host}" ;;
497+  esac
498+
499+  dnl TODO: create a rust test project and check host_alias as well as host.
500+  AC_DEFINE(ENABLE_RUSTY, [true], [Use Rusty])

dongcarl commented at 8:19 pm on October 29, 2019:

0  AC_DEFINE(ENABLE_RUSTY, [true], [Enable experimental Rust support.])

in configure.ac:529 in 22de67982c outdated

526@@ -503,6 +527,7 @@ case $host in
527      AC_CHECK_LIB([shlwapi],      [main],, AC_MSG_ERROR(libshlwapi missing))
528      AC_CHECK_LIB([iphlpapi],      [main],, AC_MSG_ERROR(libiphlpapi missing))
529      AC_CHECK_LIB([crypt32],      [main],, AC_MSG_ERROR(libcrypt32 missing))
530+     AC_CHECK_LIB([userenv],      [main],, AC_MSG_ERROR(libuserenv missing))

dongcarl commented at 8:22 pm on October 29, 2019:

Do we need libuserenv for anything other than Rust?

theuni commented at 9:10 pm on November 1, 2019:

Good point, this should be gated. It may not even be necessary now that we’ve slimmed down.

theuni commented at 10:27 pm on November 1, 2019:

mingw+rust builds fine without this now, so I guess we can just remove it.

in configure.ac:666 in 22de67982c outdated

663@@ -639,6 +664,14 @@ if test x$use_pkgconfig = xyes; then
664   ])
665 fi
666 
667+dnl Find the additional lib (if any) responsible for handling

dongcarl commented at 8:24 pm on October 29, 2019:

Do we need this for anything other than Rust?

theuni commented at 9:12 pm on November 1, 2019:

No, and I’m not actually sure why we need it for rust. It would be nice if we could disable the functionality. Again, maybe this isn’t necessary now that we’ve slimmed down and aren’t using cargo.

theuni commented at 10:11 pm on November 1, 2019:

This also seems to be unnecessary now, at least for linux/win/osx. After removing it (as well as manually removing -ldl from the link line, which some library has rudely added), builds still succeed.

TheBlueMatt commented at 7:27 pm on November 3, 2019:

Oops, looks like we do - I unset this then got a build failure on Debian Stable. I did move the block into the if use_rust block.

in src/Makefile.am:269 in 22de67982c outdated

261@@ -255,6 +262,11 @@ obj/build.h: FORCE
262 	  "$(abs_top_srcdir)"
263 libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h
264 
265+# libbitcoin_rustybridge should not need headers other than Core's internal ones.
266+libbitcoin_rustybridge_a_CPPFLAGS = $(AM_CPPFLAGS) -I$(builddir) $(LEVELDB_CPPFLAGS)
267+libbitcoin_rustybridge_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
268+libbitcoin_rustybridge_a_SOURCES =
269+libbitcoin_rustybridge_a_SOURCES += rusty/src/cpp_bridge.cpp

dongcarl commented at 8:41 pm on October 29, 2019:

I probably don’t know Makefile syntax minutiae enough, but why not just:

0libbitcoin_rustybridge_a_SOURCES = rusty/src/cpp_bridge.cpp

TheBlueMatt commented at 5:28 pm on October 30, 2019:

I think its just historical, fixed.

theuni commented at 9:12 pm on November 1, 2019:

Fwiw, I’ve done this as a convention so that future diffs are 100% straightforward when adding/removing files and keeping them in alphabetical order. I don’t really care much either way though.

in src/Makefile.am:374 in 22de67982c outdated

367@@ -356,6 +368,25 @@ libbitcoin_wallet_tool_a_SOURCES = \
368   wallet/wallettool.cpp \
369   $(BITCOIN_CORE_H)
370 
371+# rustc only reqires the main .rs file as input, but rustc must also be invoked
372+# if any of the other files change. List the main file first, then targets can
373+# depend on $(LIBBITCOIN_RUSTY_SRCS), but only build the first source file ($<).
374+

dongcarl commented at 8:54 pm on October 29, 2019:

TheBlueMatt force-pushed on Oct 30, 2019

TheBlueMatt force-pushed on Oct 31, 2019

in src/Makefile.am:388 in 74520b47af outdated

383+# an alloc and refuse it, generating a panic instead).
384+# Also force panics to unwind (instead of calling abort()) as we want to catch
385+# panics in the Rust threads and simply close the thread, instead of bringing
386+# down the rest of Bitcoin Core.
387+$(LIBBITCOIN_RUSTY): $(LIBBITCOIN_RUSTY_SRCS)
388+	$(RUSTC) --crate-name rusty $< --crate-type=staticlib --emit=link -g -O -C overflow-checks=on -C panic=unwind --out-dir $(@D)

theuni commented at 10:19 pm on November 1, 2019:

Whoops, we dropped the –target arg here when switching from cargo. My bad :(

--target=$(RUST_TARGET)

ariard commented at 2:17 am on November 2, 2019: member

@dongcarl Compile with --enable-rust-, rust library is finely compiled in generated src/rusty dir but not linked in binary.

TheBlueMatt force-pushed on Nov 3, 2019

TheBlueMatt force-pushed on Nov 4, 2019

dongcarl commented at 6:37 pm on November 4, 2019: member

The latest push seems to be working.

A few tips for other reviewers:

Empty out your ~/.bitcoin
Comment out https://github.com/bitcoin/bitcoin/blob/b060ca707b6eaa7cff26587b3b5822c1ba8c4b7f/src/rusty/src/dns_headers.rs#L79
./src/bitcoind --headersfetchdns=bitcoinheaders.net --connect=0
Marvel at blockheaders sync progress in stdout, or watch sudo tcpdump udp port 53

TheBlueMatt force-pushed on Nov 7, 2019

TheBlueMatt commented at 7:59 pm on November 7, 2019: member

Rejiggered the defaults again: today’s meeting discussion was “wait, why are we building by default, dont do that”, so we’re back to off-by-default, which does neatly solve the issues @theuni had above. Should be ready to go (tm) now.

DrahtBot added the label Needs rebase on Nov 8, 2019

TheBlueMatt force-pushed on Nov 12, 2019

DrahtBot removed the label Needs rebase on Nov 12, 2019

in src/Makefile.am:395 in f2564a6170 outdated

390+# an alloc and refuse it, generating a panic instead).
391+# Also force panics to unwind (instead of calling abort()) as we want to catch
392+# panics in the Rust threads and simply close the thread, instead of bringing
393+# down the rest of Bitcoin Core.
394+$(LIBBITCOIN_RUSTY): $(LIBBITCOIN_RUSTY_SRCS)
395+	$(RUSTC) --crate-name rusty $< --target=$(RUST_TARGET) --crate-type=staticlib --emit=link -g -O -C overflow-checks=on -C panic=unwind --out-dir $(@D)

icota commented at 3:15 am on November 16, 2019:

Debugging with gdb doesn’t seem to work with -O. Perhaps drop it when --enable-debug?

icota changes_requested

TheBlueMatt force-pushed on Nov 17, 2019

in src/Makefile.am:703 in 5c50c0e8f6 outdated

699+LIBBITCOIN_RUST_LIBS += $(LIBBITCOIN_RUSTY)
700+LIBBITCOIN_RUST_LIBS += $(LIBBITCOIN_RUSTY_BRIDGE)
701+
702+RUSTY_TEST_BIN=rusty/test_rusty
703+$(RUSTY_TEST_BIN): $(LIBBITCOIN_RUSTY_SRCS) $(LIBBITCOIN_RUSTY_TEST_SRCS)
704+	$(RUSTC) --crate-name test_rusty $< --target=$(RUST_TARGET) --emit=link --test -C debuginfo=2 -C overflow-checks=on --out-dir $(@D)

icota commented at 7:03 am on November 18, 2019:

RUST_FLAGS instead of RUST_TARGET here?

icota changes_requested

TheBlueMatt force-pushed on Nov 18, 2019

icota commented at 9:02 am on November 19, 2019: contributor

tACK 0a3c6d2e8ed1b62c60e62df3f75f4ac30618597e It does what it says on the tin. It’s great to have redundancy. IMO Rust is a very welcome addition and this PR (being optional at compile and runtime) is the least controvesial way to introduce it.

DrahtBot added the label Needs rebase on Nov 19, 2019

TheBlueMatt force-pushed on Nov 19, 2019

DrahtBot removed the label Needs rebase on Nov 20, 2019

DrahtBot added the label Needs rebase on Nov 20, 2019

TheBlueMatt force-pushed on Nov 21, 2019

DrahtBot removed the label Needs rebase on Nov 21, 2019

DrahtBot added the label Needs rebase on Dec 4, 2019

Adds a rust library to bitcoin to demonstrate linking and building rust code.

The demonstration library compiles a rust hello world example and auto-generates
a header which can be included in C++ code.

Co-Authored-By: Jeremy Rubin <j@rubin.io>
Co-Authored-By: Cory Fields <cory-nospam-@coryfields.com>
Various changes by: Matt Corallo <git@bluematt.me>

100ce28f5a

build: show rust enabled in configure output 4830d9e0c8

build: gitignore src/rusty_test 828a9822a0

Add basic ffi bindings for Rust header download

Also, break circular dependency with a new helper lib courtesy of
Cory Fields <cory-nospam-@coryfields.com>.

9bd5f6fba0

Add headers-over-DNS fetcher in Rust f0763b28ea

Limit total memory allocated by all Rust code to 128MB eb21b69810

Enable rustc building + testing on xenial Tsan Travis job 94051addd9

TheBlueMatt force-pushed on Dec 5, 2019

DrahtBot removed the label Needs rebase on Dec 5, 2019

DrahtBot commented at 11:43 pm on January 7, 2020: member

DrahtBot added the label Needs rebase on Jan 7, 2020

TheBlueMatt commented at 7:48 pm on March 5, 2020: member

Closing due to lack of interest.

TheBlueMatt closed this on Mar 5, 2020

gdassori commented at 10:42 am on November 9, 2020: none

Aw, sadly I’ve missed this at the time. As I told to Matt elsewhere this reminded me of chaindnsd, a POC I did some years ago… and I would loved that feature into the reference impl :-)

DrahtBot locked this on Feb 15, 2022

Fetch Headers over DNS #16834

Conflicts