Optimization: utilize 100% resources of a computer in long operations like rescan/initial block download #27939

1. 
   GregTonoski commented at 8:52 am on June 23, 2023: none

   Please describe the feature you’d like to see added.

   As a user, I want Bitcoin Core to utilize maximum resources (CPU, network bandwith, memory) so that intensive operations are perfomed faster (take shorter time), e.g. rescan after importing a key to a wallet opened in a pruned node.

   Is your feature related to a problem, if so please describe it.

   There is the problem that intensive operations take long while computer resources are not utilized in full:

   • CPU utilization never exceeds 30%,
   • bandwith utilization rarely exceed a few MB (approx. 10-20% of capacity, wired),
   • RAM < 50%,
   • disk (SSD) rarely exceeds a few %.

   The computer specification: [CPU] 11th Gen Intel(R) Core(TM) i5-11400H @ 2.70GHz Base speed: 2.69 GHz Sockets: 1 Cores: 6 Logical processors: 12 Virtualization: Enabled L1 cache: 480 KB L2 cache: 7.5 MB L3 cache: 12.0 MB [RAM] 16.0 GB Speed: 3200 MHz Slots used: 2 of 2 Form factor: SODIMM Hardware reserved: 280 MB [SSD] NVMe Micron 2300 NVMe 512GB Capacity: 477 GB Formatted: 477 GB System disk: Yes Page file: Yes Type: SSD [Connectivity] 1Gb Ethernet, 600 Mbit Internet bandwith. [Software] Windows 11, offical build of Bitcoin Core 25.0 (altough the same concerns the older versions), default configuration (e.g. block storage pruned to 2GB).

   Describe the solution you’d like

   No response

   Describe any alternatives you’ve considered

   No response

   Please leave any additional context

   There is 100% CPU utilization when Prime95 stress test is run on the machine.

2. GregTonoski added the label Feature on Jun 23, 2023
3. 
   maflcko commented at 9:54 am on June 23, 2023: member

   Seems to be a duplicate of #23727, #11756, and #21267, etc, no?
4. 
   GregTonoski commented at 1:06 pm on June 23, 2023: none

   No, I don’t think that this request is a duplicate. Let’s analyse it separately from the other changes which seem not to increase computer resources utilization. They would decrease demand for (if implement faster algorithms) while not increase/maximize utilization of the computer resources (as far as I understand from the descriptions).
5. 
   maflcko commented at 1:12 pm on June 23, 2023: member

   They would decrease demand for resources while not increasing/maximizng utilization of them if I understand correctly.

   It seems a bit premature to claim they don’t improve resource utilization and open an issue for this. If they don’t you can leave a review comment or open a specific issue when the work is done.

6. 
   furszy commented at 1:35 pm on June 23, 2023: member

   No, I don’t think that this request is a duplicate. Let’s analyse it separately from the other changes which seem not to increase computer resources utilization. They would decrease demand for (if implement faster algorithms) while not increase/maximize utilization of the computer resources (as far as I understand from the descriptions).

   I’m actually doing both for the #21267 working path. Check #26966 for a resource optimization work.

   And I agree with Marko on opening specific issues for each process. This way, we can address each specific case, have focused discussions, and provide updates on it independently from other improvements.

7. 
   maflcko commented at 1:50 pm on June 23, 2023: member

   Closing for now, per discussion. Feel free to open a new specific issue about a specific shortcoming where a specific improvement can be made, making sure to first check existing open issues.
8. maflcko closed this on Jun 23, 2023

   ---

9. bitcoin locked this on Jun 22, 2024

Contributors
GregTonoski maflcko furszy

Labels
Feature