Sstrhsrtj

How to clear unused space with zeros ? (I am asking for tool for btrfs filesystem)

I'm looking for something smarter than

cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero

Like FSArchiver is looking for "used space" and ignores unused, but opposite site.

Purpose: I'd like to compress partition images, so filling unused space with zeros is highly recommended.

Btw. For ext3,ext4 : Clear unused space with zeros (ext3,ext4) . Here I ask for btrfs

Clarification of "smarter algorithm": As I see that "smarter algorithm" leads to confusion, here I finally decided to add line about it. Dear reader, please think about filesystem as datastructure. For simplification purposes, let's imagine self balancing red-black tree holding map, that is used as association key->value array, where values are integers. Goal is to clear all negative values with zeros. "not smart" algorithm will iterate over it and add key:value pairs to new map (with eventual altering negative to zero). Smart algorithm will just detect places where negative values are stored and clear them in-place.

Here's a modified and much expanded repost of my answer to your previous question:

sfill from the secure-delete package can do what you want.

Unlike zerofree (which works only with ext2, ext3, and ext4 filesystems), sfill will work with any file system.

e.g.

sfill -l -l -z /mnt/X

The sfill/secure-delete home page is now seems to have vanished, but it is packaged for debian and ubuntu. probably other distros too. if you need source code, that can be found in the debian archives if you can't find it anywhere else.

NOTE: sfill only works on a mounted filesystem. If you're zero-filling the image files from the host system rather than from within a VM, you'll need to mount the fs image on the host. The exact method varies depending on what kind of image file it is (e.g. qcow2 or raw).

Mounting 'raw' images is easy and straight-forward. Just use kpartx on the image file to create a loopback device and /dev/mapper/ entries for each partition can then be mounted individually.

here's a partial script modified from a fragment of the script I use to mount net-bootable freedos hard-disk images (used for bios updates on machines where flashrom won't work):

image="myrawimagefile.img"



# use kpartx to build /dev/mapper device nodes

KP=$(kpartx -a -v "$image")



# now mount each partition under ./hd/

for p in $(echo "$KP" | awk '/^add map/ {print $3}') ; do

  dm="/dev/mapper/$p"

  fp=$(echo "$p" | sed -r -e 's/^loop[0-9]+//')

  mkdir -p "./hd/$fp"

  mount "$dm" "./hd/$fp"

  sfill -l -l -v "./hd/$fp"

  umount "$dm"

done



# now remove the loopback device and /dev/mapper entries

kpartx -d "$image"

NOTE: it assumes every partition in the image is mountable. true in my use-case (a freedos hd image with one partition). not true if one of the partitions is, say, swap space. Detecting such partitions and using dd to zero the swap partition is left as an exercise for the reader :)

This method, or a variant of it, should work for LVM volumes too.

If your images are qcow then you can use the qemu-nbd tool from the qemu-utils package, which will present the image file and its partitions as network block devices - e.g. /dev/nbd0 /dev/ndb0p1 - which can be used in a similar fashion to the /dev/mapper devices above.

It's possibly easier and less hassle (but more time-consuming) to use qemu-img to convert from qcow to raw, use the method for raw above, and then convert the modified raw image back to compressed qcow2. This will probably result in slightly smaller images than using qemu-nbd as you'll be compressing the new qcow2 image as it is being created.

The correct and "smart" way to clear the free space is using:

fstrim /mountpoint

Use this on a mounted filesystem.

It's important to note that there is no security advantage to this. If you wish to clear your disk for security, back up your files, zero the entire disk with a command like "shred" and then restore your files.

However the solution works fine if your disk is a thin provision on a sparse filesystem and you wish to recover the unused space.