Incorrect operation with the hard disk #2909

If you run bitcoind under the Ubuntu VMWare virtual machine - size of the virtual machine's hard drive will be enormous large.

How to reproduce:

1. Make a new VMWare virtual machine with Ubuntu (I used Ubuntu 12.10)
2. Install bitcoind, launch it as the server and daemon
3. Wait until the bitcoind will load all the blocks
4. Check via any linux tool how much space on the hard drive is used (about 10 Gb at all in my case)
5. Stop the virtual machine
6. Check via host machine tools how big is the folder with the Bitcoind virtual machine - it will be about 70-90 Gb!!!
7. VMware tools, that help to compact/defragment virtual machine hard drive can't help

I've never seen any software that has such specific bug. Something very strange

Pretty sure this is to be expected, except for 7. A bug in the tools?

It sounds like you're using a sparse/growable vmdk virtual disk and for some reasons the guest OS keeps using new disk blocks. What's the capacity of your virtual disk btw?

wrt the compact/defragment function in the tools: the first step of this process is to write 0s to the disks but if you run out of space on the host the app may be unable to subsequently reclaim this unused free space. You may want to try playing with the option -k of vmware-vdiskmanager and see if that helps. Please ping me privately if you have any other question.

This is expected behavior -- the block chain is very large.

@jgarzik That is correct, but irrelevant. The block chain is only 11 GB and is only written once, and he complains about much larger usage.

My guess for the reason is that the chainstate database is continuously being updated, and in total a much larger amount of data is being written for this. Assuming the size of the virtual disk is large enough, then depending on the file system, the data may end up scattered over the virtual block device, making every sector dirty and require space.

I use growable disk (tried single and split in 2GB files). Capacity - 200Gb I think if i will use smaller disk - I will get error. But it need to be verified.

I alsoI think this is due to the constant creation and deletion of chainstate files. Size of the blockchain is irrelevenat. So big virtual disk is a very strange result. I use different apps inside virtual machines, some of them uses hard drive much-much-much more then bitcoind. I've never had such problems

Still this explains it perfectly. The virtual block device does not know which parts of it are used by the file system, so every area ever written to requires disk space. This has nothing to do with Bitcoin.

I think that vmware should know which part of the filesystem is currently used and which is not. This is the only way that would allow vmware utilities to work If you make virtual machine, create and then delete very big file, launch vmware tool - drive will be reduced on the size of of the file. I made it a lot of times - software creates files, I manually delete them

tried to use "vmware-vdiskmanager.exe -k" as maxime01 advised - disk size absolutely not changed.

Most of your blocks are dirtied. You need to either:

1. vmware-mount your guest partition on your host and use vmware-vdiskmanager -p,
2. do something like "dd if=/dev/zero of=/mnt/foo && rm -f /mnt/foo" from within the guest before running vmware-vdiskmanager -k.