When syn'ing blocks, the progress is incredibly slow. The CPU usage is less than 10%, network traffic is less than 10%, load factor is around 3 The whole system is very unresponsive; it takes sometimes a minute or more to run a cli command.

The system (ODROID MC1; ARM SBC) has 8 cores and 2GB RAM; RAM usage is around 1.1 GB, no swap storage is a 480GB SSD Bitcoin Core Daemon version v0.15.1.0-g7b57bc998f3 I verified that the system is not CPU throttling because of temperature.

e.g:

2017-12-05 00:55:26 UpdateTip: new best=000000000000000000d00911c00d26abb641d6366082522f5562f715cde92d53 height=417300 version=0x20000001 log2_work=84.865016 tx=137411252 date='2016-06-21 04:05:21' progress=0.496572 cache=36.7MiB(328035txo) 2017-12-05 00:55:26 - Connect postprocess: 2.31ms [0.12s] 2017-12-05 00:55:26 - Connect block: 23700.01ms [3525.67s] 2017-12-05 00:55:37 - Load block from disk: 10556.34ms [293.62s] 2017-12-05 00:55:37 - Sanity checks: 7.94ms [0.41s] 2017-12-05 00:55:37 - Fork checks: 0.19ms [0.01s] 2017-12-05 01:00:16 - Connect 1472 transactions: 279395.86ms (189.807ms/tx, 53.854ms/txin) [3520.09s] 2017-12-05 01:00:16 - Verify 5188 txins: 279396.04ms (53.854ms/txin) [3520.11s] 2017-12-05 01:00:16 - Index writing: 0.09ms [0.00s] 2017-12-05 01:00:16 - Callbacks: 0.07ms [0.00s] 2017-12-05 01:00:16 - Connect total: 279752.57ms [3521.30s] 2017-12-05 01:00:16 - Flush: 17.99ms [0.94s] 2017-12-05 01:00:16 - Writing chainstate: 0.18ms [0.01s]

So it looks to me like processing this block took about 5 minutes. AFAIK a new block is created about every 10 minutes. No wonder that the syncing "never" finishes. This is going on for over a week now.

Maybe this is some kind of DOS attack from malicious peer nodes?!? Or there is some multithreading related /(and/or ARM related) bug in the code; because CPU and network load is very low; yet the system makes little progress and is never the less VERY unresponsive.

Whatever the reason, this is as it is, not usable at all.

Are you guys testing your software on ARM SBC? This is what I'm using, it's $50: (ODROID HC1; 8 cores, 2GB RAM, SATA interface) http://www.hardkernel.com/main/products/prdt_info.php?g_code=G150229074080

2017-12-05 00:55:37 - Load block from disk: 10556.34ms [293.62s] <-- it took 10 seconds to load the block to connect from disk. You should look into the speed/latency of your disk (are you storing the blockchain on some SD card/device storage instead of your SATA-attached SSD?)

It's slow because your system is slow. This level of performance is expected if you don't have an SSD drive. This is definitively not a bug.

hdparm -tT /dev/sda

/dev/sda: Timing cached reads: 1714 MB in 2.00 seconds = 857.28 MB/sec Timing buffered disk reads: 776 MB in 3.01 seconds = 258.18 MB/sec

@unsystemizer: I wrote above that I have a 480GB SSD ???

@PolarGo No need to be hostile. Can you provide the output of mount? Is the Bitcoin datadir on the internal SD card or is it being stored on the SSD?

@unsystemizer On almost any system it's completely unreasonable that loading a block from disk takes 10 seconds, or loading the inputs from the chainstate takes 5 minutes - regardless of what disk setup you have. Please don't claim that this is normal or expected without an SSD (even my odroid-c2 with microsd storage is much faster than that). It may be expected given other hardware or software characteristics OP's system may have, but we don't know that yet.

yes, datadir is on ssd. I use ext4 on lvm logical volume.

below is the smartmontools output, which looks ok to me:

smartctl --all /dev/sda
=== START OF INFORMATION SECTION ===
Model Family:     SandForce Driven SSDs
Device Model:     MKNSSDEC480GB
LU WWN Device Id: 0 015030 1a1739104
Firmware Version: 603ABBF0
User Capacity:    480,103,981,056 bytes [480 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS, ACS-2 T13/2015-D revision 3
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Dec  5 21:28:34 2017 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

[...]

  1 Raw_Read_Error_Rate     0x0032   095   095   050    Old_age   Always       -       0/3112556
  5 Retired_Block_Count     0x0033   099   099   003    Pre-fail  Always       -       25
  9 Power_On_Hours_and_Msec 0x0032   099   099   000    Old_age   Always       -       1450h+22m+01.500s
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       96
171 Program_Fail_Count      0x000a   100   100   000    Old_age   Always       -       0
172 Erase_Fail_Count        0x0032   100   100   000    Old_age   Always       -       0
174 Unexpect_Power_Loss_Ct  0x0030   000   000   000    Old_age   Offline      -       34
177 Wear_Range_Delta        0x0000   000   000   000    Old_age   Offline      -       0
181 Program_Fail_Count      0x000a   100   100   000    Old_age   Always       -       0
182 Erase_Fail_Count        0x0032   100   100   000    Old_age   Always       -       0
187 Reported_Uncorrect      0x0012   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0000   052   057   000    Old_age   Offline      -       52 (Min/Max 19/57)
194 Temperature_Celsius     0x0022   052   057   000    Old_age   Always       -       52 (Min/Max 19/57)
195 ECC_Uncorr_Error_Count  0x001c   120   120   000    Old_age   Offline      -       0/3112556
196 Reallocated_Event_Count 0x0033   099   099   003    Pre-fail  Always       -       25
201 Unc_Soft_Read_Err_Rate  0x001c   120   120   000    Old_age   Offline      -       0/3112556
204 Soft_ECC_Correct_Rate   0x001c   120   120   000    Old_age   Offline      -       0/3112556
230 Life_Curve_Status       0x0013   100   100   000    Pre-fail  Always       -       100
231 SSD_Life_Left           0x0013   089   089   010    Pre-fail  Always       -       21474836481
233 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       53797
234 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       1004
241 Lifetime_Writes_GiB     0x0032   000   000   000    Old_age   Always       -       1004
242 Lifetime_Reads_GiB      0x0032   000   000   000    Old_age   Always       -       863

@PolarGo what does the output of dd if=/path/to/bitcoin/datadir/blocks/blk00000.dat of=/dev/null say?

[root@alarm alarm]# dd if=/dev/mapper/ssd-data of=/dev/null count=15 bs=4096
15+0 records in
15+0 records out
61440 bytes (61 kB, 60 KiB) copied, 15.5446 s, 4.0 kB/s

So it seems to be indeed a hardware problem. I've never seen something like this before. Sorry for the trouble caused!