Command-line incremental backup tools

up vote
6
down vote

favorite

Which command line backup tool(s) has the best compression ratio? I want to backup my entire system, including media files, text files etc.

I found this list on the Arch wiki site but I don't know how these tools compare to each other and which one would offer the best compression ratio overall. I realize that different tools might give better results for specific file types, but which tool and using which settings would result in the smallest archive given a mix of various input files?

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

4

covering all bases I see.
– mikeserv
Jun 24 '14 at 2:54

1

This question isn't opinion based; it's asking for an objective metric (compression ratio).
– goldilocks
Jun 24 '14 at 15:53

so I used tar so far, which is relatively slow
– polym
Jul 5 '14 at 23:42

add a comment |

up vote
6
down vote

favorite

Which command line backup tool(s) has the best compression ratio? I want to backup my entire system, including media files, text files etc.

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

4

covering all bases I see.
– mikeserv
Jun 24 '14 at 2:54

1

This question isn't opinion based; it's asking for an objective metric (compression ratio).
– goldilocks
Jun 24 '14 at 15:53

so I used tar so far, which is relatively slow
– polym
Jul 5 '14 at 23:42

add a comment |

up vote
6
down vote

favorite

Which command line backup tool(s) has the best compression ratio? I want to backup my entire system, including media files, text files etc.

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

Which command line backup tool(s) has the best compression ratio? I want to backup my entire system, including media files, text files etc.

command-line backup compression

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

edited Jun 24 '14 at 11:30

terdon♦

127k31244421

asked Jun 24 '14 at 2:33

polym

6,50643157

asked Jun 24 '14 at 2:33

polym

6,50643157

asked Jun 24 '14 at 2:33

polym

6,50643157

4

covering all bases I see.
– mikeserv
Jun 24 '14 at 2:54

1

This question isn't opinion based; it's asking for an objective metric (compression ratio).
– goldilocks
Jun 24 '14 at 15:53

so I used tar so far, which is relatively slow
– polym
Jul 5 '14 at 23:42

add a comment |

4

covering all bases I see.
– mikeserv
Jun 24 '14 at 2:54

1

This question isn't opinion based; it's asking for an objective metric (compression ratio).
– goldilocks
Jun 24 '14 at 15:53

so I used tar so far, which is relatively slow
– polym
Jul 5 '14 at 23:42

covering all bases I see.
– mikeserv
Jun 24 '14 at 2:54

This question isn't opinion based; it's asking for an objective metric (compression ratio).
– goldilocks
Jun 24 '14 at 15:53

so I used tar so far, which is relatively slow
– polym
Jul 5 '14 at 23:42

add a comment |

1 Answer
1

active

oldest

votes

up vote
1
down vote

The highest compression ratio also has some important drawbacks and is usually not recommended.

For a backup solution, it is often important to have a fast restore.

The compression ratio you are able to achieve depends on your data and the compression tool you are using. xz provides one of the highest compression ratios:

xz -z -c -9 -e /dev/sda2 > /path/file.xz

will compress your disk device to stdout (-c) with the highest compression ratio (-9) and the extreme switch (-e). This will take a very long time.

Another way to have good compression ratios and also a fast restore is using a compression-enabled file system like BTRFS, where you can store for example rsync backups.

To mount a compressed BTRFS volume:

mount /dev/sda2 -t btrfs -o noatime,nodiratime,compress=lzo /path

This is pretty convenient, because you don't need to deal with compression (it is automatically done by the filesystem) and have fast access to your backed-up data.

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

add a comment |

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f138862%2fcommand-line-incremental-backup-tools%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
1
down vote

The highest compression ratio also has some important drawbacks and is usually not recommended.

For a backup solution, it is often important to have a fast restore.

The compression ratio you are able to achieve depends on your data and the compression tool you are using. xz provides one of the highest compression ratios:

xz -z -c -9 -e /dev/sda2 > /path/file.xz

will compress your disk device to stdout (-c) with the highest compression ratio (-9) and the extreme switch (-e). This will take a very long time.

Another way to have good compression ratios and also a fast restore is using a compression-enabled file system like BTRFS, where you can store for example rsync backups.

To mount a compressed BTRFS volume:

mount /dev/sda2 -t btrfs -o noatime,nodiratime,compress=lzo /path

This is pretty convenient, because you don't need to deal with compression (it is automatically done by the filesystem) and have fast access to your backed-up data.

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

add a comment |

up vote
1
down vote

The highest compression ratio also has some important drawbacks and is usually not recommended.

For a backup solution, it is often important to have a fast restore.

The compression ratio you are able to achieve depends on your data and the compression tool you are using. xz provides one of the highest compression ratios:

xz -z -c -9 -e /dev/sda2 > /path/file.xz

will compress your disk device to stdout (-c) with the highest compression ratio (-9) and the extreme switch (-e). This will take a very long time.

Another way to have good compression ratios and also a fast restore is using a compression-enabled file system like BTRFS, where you can store for example rsync backups.

To mount a compressed BTRFS volume:

mount /dev/sda2 -t btrfs -o noatime,nodiratime,compress=lzo /path

This is pretty convenient, because you don't need to deal with compression (it is automatically done by the filesystem) and have fast access to your backed-up data.

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

add a comment |

up vote
1
down vote

The highest compression ratio also has some important drawbacks and is usually not recommended.

For a backup solution, it is often important to have a fast restore.

The compression ratio you are able to achieve depends on your data and the compression tool you are using. xz provides one of the highest compression ratios:

xz -z -c -9 -e /dev/sda2 > /path/file.xz

will compress your disk device to stdout (-c) with the highest compression ratio (-9) and the extreme switch (-e). This will take a very long time.

Another way to have good compression ratios and also a fast restore is using a compression-enabled file system like BTRFS, where you can store for example rsync backups.

To mount a compressed BTRFS volume:

mount /dev/sda2 -t btrfs -o noatime,nodiratime,compress=lzo /path

This is pretty convenient, because you don't need to deal with compression (it is automatically done by the filesystem) and have fast access to your backed-up data.

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

The highest compression ratio also has some important drawbacks and is usually not recommended.

For a backup solution, it is often important to have a fast restore.

The compression ratio you are able to achieve depends on your data and the compression tool you are using. xz provides one of the highest compression ratios:

xz -z -c -9 -e /dev/sda2 > /path/file.xz

will compress your disk device to stdout (-c) with the highest compression ratio (-9) and the extreme switch (-e). This will take a very long time.

Another way to have good compression ratios and also a fast restore is using a compression-enabled file system like BTRFS, where you can store for example rsync backups.

To mount a compressed BTRFS volume:

mount /dev/sda2 -t btrfs -o noatime,nodiratime,compress=lzo /path

This is pretty convenient, because you don't need to deal with compression (it is automatically done by the filesystem) and have fast access to your backed-up data.

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

edited Nov 27 at 23:45

n.st

5,13311742

edited Nov 27 at 23:45

n.st

5,13311742

edited Nov 27 at 23:45

n.st

5,13311742

answered Jul 24 '14 at 19:52

user55518

answered Jul 24 '14 at 19:52

user55518

answered Jul 24 '14 at 19:52

user55518

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

add a comment |

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

good point about the drawbacks and great example, I believe the reason why xz gained traction was because bzip2, while having a greater compression ratio than gzip, worked much slower
– AnonymousLurker
Nov 28 at 4:29

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Unix & Linux Stack Exchange!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Sstrhsrtj