rejetto forum

Software => HFS ~ HTTP File Server => Topic started by: MarkV on November 26, 2008, 01:32:41 AM

Title: folder archive
Post by: MarkV on November 26, 2008, 01:32:41 AM
Just a little question concerning folder archive.

As I understood it, the tar archive is created and served on the fly. But what happens if data/files etc. are changed/created/deleted while serving a folder .tar? How will HFS react? Can the archive corrupt?
Title: Re: folder archive
Post by: rejetto on November 26, 2008, 09:29:33 AM
while the tar is served, the single file is open, thus changes(write) are not permitted by Windows.
but if you stop the download, and restart in the middle of a file, the resulting file will be corrupted.

what i could do is, if the request includes a date, check if the timestamp of file in the middle of the resume point is newer than the request...
in such case i may decide to not allow the resume.
this also mean that if you downloaded a Gig of other files, that will be lost.
opinions?
Title: Re: folder archive
Post by: maverick on November 26, 2008, 09:58:00 AM
what i could do is, if the request includes a date, check if the timestamp of file in the middle of the resume point is newer than the request...
in such case i may decide to not allow the resume.
this also mean that if you downloaded a Gig of other files, that will be lost.
opinions?

Please don't even think about not allowing resume when downloading.  That is an important feature IMHO.
Title: Re: folder archive
Post by: rejetto on November 26, 2008, 10:00:32 AM
in such case, the file will just result corrupted inside the archive
Title: Re: folder archive
Post by: maverick on November 26, 2008, 10:06:20 AM
in such case, the file will just result corrupted inside the archive

I see your point.  Maybe not allowing resume for "tar archives" might be the best solution.  To be honest, I haven't seen this type of problem with a tar archive and don't know if it is a common problem or not.
Title: Re: folder archive
Post by: rejetto on November 26, 2008, 10:13:15 AM
the point is: if you never meet the problem, then you'll also be always allowed to resume.
forbidding is only for cases where a file would get corrupted.
problem is: only that file, not the whole archive, that's why the question is debated.
hard or impossible to know if the downloader would prefer restarting or not the download from beginning.
IDK if asking is possible, but even asking would be very confusing for 90% of people, i guess.
Title: Re: folder archive
Post by: maverick on November 26, 2008, 11:17:19 AM
problem is: only that file, not the whole archive, that's why the question is debated.

Well then, why not let the user (downloader) decide on what he wants to do --  Accept the resumed download as is containing the corrupt file (that could be downloaded seperately at a different time) or re-download the whole archive again from the beginning.

Would HFS be smart enough to:
1. Notify the downloader with a popup or something indicating there is a corrupt file in the archive before resuming and also let him know what the corrupt filename is.
2. Give the downloader the option to either cancel the resumed download containing the corrupt file, continue the resumed download as is with the corrupt file,  or start the download process again from the beginning.


Title: Re: folder archive
Post by: MarkV on November 26, 2008, 04:58:00 PM
Thank heavens tar is not a solid archive type.

Another option would be to duplicate the corrupt file inside the archive (with a new name), I mean, d/l it a second time, this time uncorrupted.
Title: Re: folder archive
Post by: rejetto on November 26, 2008, 05:17:32 PM
Ok guys, i tested by using wget, firefox, downThemAll, FDM.
All of them didn't report any timestamp, asking to resume.
So there's no way for me to know if there would be a corruption or not.

Quote
Another option would be to duplicate the corrupt file inside the archive (with a new name), I mean, d/l it a second time, this time uncorrupted.

this is a very nice idea, although unsuitable due to my statement above.
Title: Re: folder archive
Post by: Wasserfloh on November 26, 2008, 06:56:57 PM
Ok guys, i tested by using wget, firefox, downThemAll, FDM.
All of them didn't report any timestamp, asking to resume.
So there's no way for me to know if there would be a corruption or not.
What speaks against one timestamp in the file name?
Title: Re: folder archive
Post by: rejetto on November 27, 2008, 10:30:26 AM
i didn't understand what you meant
Title: Re: folder archive
Post by: MarkV on November 27, 2008, 05:44:58 PM
Either the timestamp is asked after finishing the d/l, or it is not asked at all. :(

Couldn't you check at least the filesize?
Title: Re: folder archive
Post by: rejetto on November 27, 2008, 09:01:45 PM
and compare it with what?
Title: Re: folder archive
Post by: MarkV on November 28, 2008, 11:26:56 PM
With the filesize from the original listing? Possible?

Damn that's so easy with FTP...
Title: Re: folder archive
Post by: rejetto on November 28, 2008, 11:33:38 PM
i know only the filesize of the server-side file.
i know nothing about the archive you are trying to resume.
so i see nothing to compare with.
Title: Re: folder archive
Post by: MarkV on November 28, 2008, 11:38:13 PM
I mean the filesize of the (possibly) corrupted file. But maybe it's not possible to check...
Title: Re: folder archive
Post by: rejetto on November 28, 2008, 11:39:20 PM
the corrupted file is client-side
Title: Re: folder archive
Post by: MarkV on November 28, 2008, 11:46:59 PM
...and the original one is on the server. I give up.  :-[