rsync gets stuck on *.png files [on hold]
up vote
3
down vote
favorite
I have been running the following rsync command as a cronjob for the past month. Today I discovered that this has been getting stuck on a *.png file and none of my other data has been backing up! In the meantime I've excluded *.png files and rsync has been working again, but I'd like a comprehensive solution to back up ALL of my files if possible.
This is getting stuck on *.png files
rsync -rcvl --delete --progress /home/user/target /media/user/backup/source
This is my temporary fix
rsync -rcvl --delete --progress --exclude='*.png' /home/user/target /media/user/backup/source
Each image is only ~300k.
The filesystems of both the target and source directories are fuseblk. The source dir is a Western Digital 4TB external hard-drive that is connected via USB 3.0.
cron rsync xubuntu png
asked Nov 26 at 23:30
spacedSparking
1164
put on hold as unclear what you're asking by Ipor Sircer, Christopher, thrig, Isaac, G-Man Nov 27 at 4:45
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
|
show 1 more comment
up vote
3
down vote
favorite
I have been running the following rsync command as a cronjob for the past month. Today I discovered that this has been getting stuck on a *.png file and none of my other data has been backing up! In the meantime I've excluded *.png files and rsync has been working again, but I'd like a comprehensive solution to back up ALL of my files if possible.
This is getting stuck on *.png files
rsync -rcvl --delete --progress /home/user/target /media/user/backup/source
This is my temporary fix
rsync -rcvl --delete --progress --exclude='*.png' /home/user/target /media/user/backup/source
Each image is only ~300k.
The filesystems of both the target and source directories are fuseblk. The source dir is a Western Digital 4TB external hard-drive that is connected via USB 3.0.
cron rsync xubuntu png
asked Nov 26 at 23:30
spacedSparking
1164
put on hold as unclear what you're asking by Ipor Sircer, Christopher, thrig, Isaac, G-Man Nov 27 at 4:45
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
2
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
1
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
1
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08
|
show 1 more comment
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I have been running the following rsync command as a cronjob for the past month. Today I discovered that this has been getting stuck on a *.png file and none of my other data has been backing up! In the meantime I've excluded *.png files and rsync has been working again, but I'd like a comprehensive solution to back up ALL of my files if possible.
This is getting stuck on *.png files
rsync -rcvl --delete --progress /home/user/target /media/user/backup/source
This is my temporary fix
rsync -rcvl --delete --progress --exclude='*.png' /home/user/target /media/user/backup/source
Each image is only ~300k.
The filesystems of both the target and source directories are fuseblk. The source dir is a Western Digital 4TB external hard-drive that is connected via USB 3.0.
cron rsync xubuntu png
asked Nov 26 at 23:30
spacedSparking
1164
I have been running the following rsync command as a cronjob for the past month. Today I discovered that this has been getting stuck on a *.png file and none of my other data has been backing up! In the meantime I've excluded *.png files and rsync has been working again, but I'd like a comprehensive solution to back up ALL of my files if possible.
This is getting stuck on *.png files
rsync -rcvl --delete --progress /home/user/target /media/user/backup/source
This is my temporary fix
rsync -rcvl --delete --progress --exclude='*.png' /home/user/target /media/user/backup/source
Each image is only ~300k.
The filesystems of both the target and source directories are fuseblk. The source dir is a Western Digital 4TB external hard-drive that is connected via USB 3.0.
cron rsync xubuntu png
cron rsync xubuntu png
asked Nov 26 at 23:30
spacedSparking
1164
asked Nov 26 at 23:30
spacedSparking
1164
edited Nov 27 at 21:46
asked Nov 26 at 23:30
spacedSparking
1164
asked Nov 26 at 23:30
spacedSparking
1164
asked Nov 26 at 23:30
spacedSparking
1164
1164
put on hold as unclear what you're asking by Ipor Sircer, Christopher, thrig, Isaac, G-Man Nov 27 at 4:45
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
put on hold as unclear what you're asking by Ipor Sircer, Christopher, thrig, Isaac, G-Man Nov 27 at 4:45
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
2
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
1
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
1
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08
|
show 1 more comment
2
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
1
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
1
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08
2
2
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
1
1
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
1
1
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08
|
show 1 more comment
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
2
How large are the large files and what filesystem type does the destination directory live on?
– Kusalananda
Nov 26 at 23:53
Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results.
– spacedSparking
Nov 27 at 0:32
1
I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else.
– Rui F Ribeiro
Nov 27 at 8:52
Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems.
– spacedSparking
Nov 27 at 21:49
1
Is this to an NTFS filesystem?
– roaima
Nov 27 at 23:08