Fix broken lists.linuxfoundation.org URLs #1698

pull kanzure wants to merge 2 commits into bitcoin:master from kanzure:fix-linuxfoundation-urls changing 46 files +104 −104
  1. kanzure commented at 2:14 pm on November 9, 2024: contributor

    This pull request updates lists.linuxfoundation.org urls to gnusha.org/url.

    Background

    Unfortunately, lists.linuxfoundation.org is no longer hosting the bitcoin-dev static email archives, nor any others for that matter. This has caused many links around the web to become broken.

    Redirect web service

    gnusha.org/url is a web service that redirects old style archive urls to a new location where the content can be viewed.

    For example, a bitcoin-dev mailing list archive url like:

    https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html

    using https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html

    is redirected to:

    https://gnusha.org/pi/bitcoindev/ogoa6o$o4s$1@blaine.gmane.org/

    The redirect mappings were generated from my archives and you can review the scripts I made to do that here: https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b

    Alternative

    Instead of using gnusha.org/url, another option would be to resolve all of the archive urls to their email Message-ID redirects for the bitcoin-dev mailing list and use those urls instead. In fact, either gnusha.org/pi/bitcoindev or https://mailing-list.bitcoindevs.xyz/bitcoindev/ could be used for this purpose.

    Those links would look like:

    0https://gnusha.org/pi/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_ZUiBg1HGFyA@mail.gmail.com/
    1https://mailing-list.bitcoindevs.xyz/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_ZUiBg1HGFyA@mail.gmail.com/
    

    If this is the preferred solution for the BIPs, then let me know how I can help. Take a look here.

    Finally, one other alternative is to wait and do nothing. Maybe someone can convince Linux Foundation to put up the static HTML archives for posterity?

    More background

    For more information see:

  2. update lists.linuxfoundation.org urls to gnusha.org/url
    Unfortunately, lists.linuxfoundation.org is no longer hosting the
    bitcoin-dev mailing list static archives nor any other mailing list
    static archives.
    
    gnusha.org/url is a web service that redirects old style archive urls to
    a new location where the content can be viewed.
    
    For more information see:
    https://gnusha.org/url/
    https://x.com/kanzure/status/1853779672514826334
    https://github.com/bitcoin/bitcoin/pull/29782#issuecomment-2460974096
    https://github.com/bitcoin/bitcoin/pull/31240
    https://github.com/bitcoinops/bitcoinops.github.io/pull/2018
    55a92a59c1
  3. jlopp commented at 7:59 am on November 10, 2024: contributor
    tACK
  4. jonatack commented at 2:52 am on November 11, 2024: member

    Nice, concept ack.

    Looks like the http urls need updating.

    0bip-0099.mediawiki:11:  Post-History: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/008936.html
    1bip-0101.mediawiki:90:[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/008820.html Simulations show] that with the current peer-to-peer protocol, miners behind high-latency or low-bandwidth links are at a disadvantage compared to miners connected to a majority of hashpower via low-latency, high-bandwidth links. Larger blocks increase the advantage of miners with high-bandwidth connections, although that advantage can be minimized with changes to the way new blocks are announced (e.g. http://bitcoinrelaynetwork.org/ ).
    2bip-0106.mediawiki:55:These two proposals have been derived after discussion on [https://bitcointalk.org/index.php?topic=1154536.0 BitcoinTalk] and [http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010285.html bitcoin-dev mailing list]. The original idea and its evolution in the light of various arguments can be found [http://upalc.com/maxblocksize.php here].
    3bip-0111.mediawiki:30:to no privacy<ref>http://eprint.iacr.org/2014/763</ref>, as well as being a large DoS risk on some nodes<ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2013-July/003044.html] is one example where the issues were found, though others independently discovered issues as well. Sample DoS exploit code available at https://github.com/petertodd/bloom-io-attack.</ref>.
    4bip-0112.mediawiki:381:[http://lists.linuxfoundation.org/pipermail/lightning-dev/2015-July/000021.html HTLCs using OP_CHECKSEQUENCEVERIFY/OP_LOCKTIMEVERIFY and revocation hashes]
    5bip-0112.mediawiki:389:[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010396.html Softfork deployment considerations]
    6bip-0113.mediawiki:119:[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010396.html Softfork deployment considerations]
    7bip-0124.mediawiki:13:  Post-History: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-November/011795.html
    
  5. jonatack added the label Bug fix on Nov 11, 2024
  6. also update http:// linuxfoundation urls.
    See 55a92a59c1e5ddb7c165327d5af4fd0c93d2a2a9.
    9834559b3c
  7. kanzure commented at 12:36 pm on November 11, 2024: contributor

    @jonatack, fixed that. thanks.

    I am seeking feedback on gnusha.org/url/ vs resolving everything to urls like https://gnusha.org/pi/bitcoindev/ogoa6o$o4s$1@blaine.gmane.org/

    With the resolved urls, the IDs are more verbose– instead of an integer ID for the message, it’s the Message-ID from the actual email. Then in the future someone can use that Message-ID to find the email on a mirror, like https://mailing-list.bitcoindevs.xyz/bitcoindev/ogoa6o$o4s$1@blaine.gmane.org/.

    This seems to be the likely direction that bitcoin.git is going in. They only have four urls in the codebase.

  8. kanzure commented at 12:42 pm on November 11, 2024: contributor

    I’ll elaborate on gnusha.org/url vs gnusha.org/pi/bitcoindev style urls.

    It is nice and simple to use a find-and-replace with gnusha.org/url/ because anyone can easily repeat that exercise and verify the git diff before merging the pull request. However, the actual routing is not something that is verified by just checking that the git diff looks sane.

    If I was to provide my mapping (I haven’t uploaded this yet, but I did publish the scripts I used to make it), then someone could write a script and/or cross-verify that all of the replacements are valid according to the mapping, if we were to use the destination urls instead of the redirect service for this pull request. Then someone would be able to verify that kind of pull request.

    There are other further issues though, like content malleability, wrong redirect link injection, service provider compromise… instead of referencing by Message-ID maybe in the future we can reference by email hash…. Just dreaming here.

  9. kanzure commented at 7:49 pm on November 11, 2024: contributor

    Instead of using gnusha.org/url, another option would be to resolve all of the archive urls to their email Message-ID redirects for the bitcoin-dev mailing list and use those urls instead. In fact, either gnusha.org/pi/bitcoindev or https://mailing-list.bitcoindevs.xyz/bitcoindev/ could be used for this purpose.

    this is implemented here: https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b#file-resolver-py

    LMK if that’s the preference instead of the current version of this pull request (gnusha.org/url inserts).

  10. jonatack commented at 9:45 pm on November 13, 2024: member
    Thanks @kanzure. No strong opinion, but I think I mildly prefer the destination urls over the redirects.
  11. murchandamus commented at 4:34 pm on November 14, 2024: contributor

    LGTM.

    I also don’t have a strong opinion regarding the two approaches. I’d prefer a quicker reestablishing functionality over a delayed perfect solution.

  12. kanzure commented at 5:15 pm on November 14, 2024: contributor
    There would be no delay. Either will work now. See the gist.
  13. ajtowns commented at 6:49 am on November 19, 2024: contributor

    The lists.linuxfoundation.org site seems to redirect automatically now:

    0$ curl -I https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2012-June/001511.html
    1HTTP/1.1 301 Moved Permanently
    2Content-length: 0
    3Location: https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2012-June/001511.html
    

    (also applies to lightning-dev mails)

    Is there still a good reason to churn a bunch of bip texts, rather than just restrict this to updating the mailing list location in bip 2?

  14. kanzure commented at 6:13 pm on November 19, 2024: contributor

    The lists.linuxfoundation.org site seems to redirect automatically now:

    I haven’t announced this yet. It’s a strange situation because LF has not agreed to keep these redirects (perhaps they don’t even know at the moment about these redirects), and LF is not running that server. Given the strangeness I haven’t been sure what to say or announce precisely.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bips. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-07 00:10 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me