@Mars: After doing several tests, everything seems to be working OK, but somehow, I've found a small rare bug/issue: if connection gets closed (or dropped) by the remote server (or by bad connection quality) while downloading a chunk, then the result was getting a corrupted file (I've ended up with a corrupted 15,7MB file, instead of being 12,6MB). It only happened once, and then I couldn't recreate the error. I don't know if we can get a workaround or modify the code to avoid bugs like this happen again, but I will leave you the logs of my tests:
GOOD download (normal log):10:25:05 127.0.0.1:1281 Requested GET /
10:25:13 127.0.0.1:1281 loading from 'http://distro.ibiblio.org/tinycorelinux/9.x/x86/release/Core-9.0.iso' 13256704 bytes
10:25:34 127.0.0.1:1281 chunk=10000000 from=0 size=10000000
10:26:08 127.0.0.1:1281 chunk=3256704 from=10000000 size=3256704
10:26:09 127.0.0.1:1281 saved file contain 13256704 bytes
10:26:09 127.0.0.1:1281 Requested POST /
BAD download (issue log):(Marked in bold = are the abnormal execution of code)
(Marked in red = comments added by me to describe the situation)
10:36:05 127.0.0.1:1257 Requested GET /
10:36:17 127.0.0.1:1257 loading from 'http://distro.ibiblio.org/tinycorelinux/9.x/x86/release/Core-9.0.iso' 13256704 bytes
10:36:53 127.0.0.1:1257 chunk=10000000 from=0 size=10000000 > Connection dropped!
10:37:16 127.0.0.1:1269 loading from 'http://distro.ibiblio.org/tinycorelinux/9.x/x86/release/Core-9.0.iso' 13256704 bytes
10:37:38 127.0.0.1:1269 chunk=10000000 from=0 size=10000000
10:37:47 127.0.0.1:1269 chunk=3256704 from=10000000 size=3256704
10:37:48 127.0.0.1:1269 saved file contain 13256704 bytes
10:37:48 127.0.0.1:1269 Requested POST /
10:37:48 127.0.0.1:1257 chunk=3256704 from=10000000 size=3256704 > WTF!?
10:37:49 127.0.0.1:1257 saved file contain 13256704 bytes > Not true, file was bigger, (exactly 16513408 bytes)!
10:37:49 127.0.0.1:1257 Requested POST /
Code used (RemoteUpload3.tpl):<form method='post'>
URL: <input name='url' value="http://distro.ibiblio.org/tinycorelinux/9.x/x86/release/Core-9.0.iso">
<br>Filename: <input name='dest' value="%folder%Core-9.0.iso">
<br><input type='submit'>
</form>
{.set|url|{.postvar|url.}.}
{.break|if={.not|{.^url.}.}.}
{.set|dest|{.or|{.filename|{.postvar|dest.}.}|{.filename|{.^url.}.}|downloaded.}.}
{.delete|{.^dest.}.}
{.add to log| loading from '{.^url.}' {.filesize|{.^url.}.} bytes .}
{.break|if={.not|{.filesize|{.^url.}.}.}|result=Source file can't be downloaded: server return nul size.}
{.set|from|0.}
{.save|{.^dest.}|.}
{.comment| define CHUNK with the min size, if nul then WHILE is never executed.}
{.set|chunk|{.min|{.filesize|{.^url.}.}|10000000.}.}
{.while|chunk|{:
{.load|{.^url.}|from={.^from.}|var=data|size={.^chunk.}.}
{.add to log| chunk={.^chunk.} from={.^from.} size={.length|var=data.}.}
{.if|{.length|var=data.}
| {:
{.append|{.^dest.}|var=data.}{.inc|from|{.length|var=data.}.}
{.comment| redefine CHUNK with the min size, if nul then WHILE is stoped.}
{.set|chunk|{.min|{.^chunk.}|{.sub|{.filesize|{.^url.}.}|{.^from.}/sub.}.}.}
:}
| {:{.set|chunk|0.}:}
/if.}:}
|timeout=0
|else={:{.add to log|Error during WHILE.}
:}/while.}
{.add to log| saved file contain {.^from.} bytes.}
» Ideas for solutions/enhancements: I don't know if there is a way to 'detect' if a connection was closed (or dropped) while downloading a chunk, but I think a simple verification of file size after finish the download, could be of help. Also, perhaps a 'progress' bar could be of help, meanwhile the file is downloading (if the download on the server takes too long).
I was trying to add a 'progress' bar, like HFS has by default for uploads (useful if the download on the server takes too long), but using %progress-files%, %total% and %speed-kb% doesn't seem to have effect.
Show Posts
