rejetto forum

Software => HFS ~ HTTP File Server => Bug reports => Topic started by: Wasserfloh on May 31, 2007, 07:08:13 PM

Title: Upload broken with small files
Post by: Wasserfloh on May 31, 2007, 07:08:13 PM
Problems with the Upload

I have here again and again problems with the uupload of files on HFS. The browser waits then up to the timeout for answer of the server. Sometimes the files are correctly uploaded. Often the files are however also incomplete and with the upload of several files are missing which and the files are partly joined.
 
The problem arises with the most different files and both locally and over the InterNet.

OS: Win2k
Browser: Opera 9.2x, FF 2.0.0.3
HFS: 2.1d build 88, 2.2 build 100, 2.2 build 101, default template, default preferences, vitual file system only upload folder

The Logs is inserted as attachments. When inserting as code a server error prevents the post.

Log1
The Upload in line 4 and 5 were actually 2 files. I do not exclude an error of the browser Opera here however.

Log2
Here Opera up to timeout waited for an answer of the server. Both files were completely transferred. More did not come in the log.
 
Log3
Opera hung also here again and waited for answer of the server. Were 3 files, about which both first were completely high-loaded and added together. Third is missing completely.

Log4
That was not an allegedly successful attempt with FF however 013vi.jpg is 120.79 KB largely. Here again 2 files were joined. That is no Opera problem thus probably nevertheless.
 
Result: The Upload is here nearly useless. Only if with the Upload only RAR-, ZIP- and TAR archives and executable files (* exe) were involved, gabs so far no problems.

[attachment deleted by admin]
Title: Upload broken with small files
Post by: rejetto on June 24, 2007, 03:24:30 PM
Log1
The Upload in line 4 and 5 were actually 2 files. I do not exclude an error of the browser Opera here however.

> 4:  22:20:13 127.0.0.1:1444 Uploading Mailansicht.png
> 5:  22:20:24 127.0.0.1:1444 Fully uploaded -  145,88 MB @ 13043 KB/s

you mean that this was not a 145MB png fle, but actually 2 files?

Quote
Log4
That was not an allegedly successful attempt with FF however 013vi.jpg is 120.79 KB largely. Here again 2 files were joined. That is no Opera problem thus probably nevertheless.

joined?

anyway.... i studied the upload code, and found no apparent problem. i can speculate about it not handling the upload if a disconnection is issued in the while, so i made a little change that may solve this.
Title: Upload broken with small files
Post by: Wasserfloh on June 26, 2007, 05:32:28 PM
Log1
 The Upload in line 4 and 5 were actually 2 files. I do not exclude an error of the browser Opera here however.
 

 > 4:  22:20:13 127.0.0.1:1444 Uploading Mailansicht.png
 > 5:  22:20:24 127.0.0.1:1444 Fully uploaded -  145,88 MB @ 13043 KB/s
 
 you mean that this was not a 145MB png fle, but actually 2 files?
yes
 
 
Quote
Log4
 That was not an allegedly successful attempt with FF however 013vi.jpg is 120.79 KB largely. Here again 2 files were joined. That is no Opera problem thus probably nevertheless.
 

 joined?
yes
 
 
anyway.... i studied the upload code, and found no apparent problem. i can speculate about it not handling the upload if a disconnection is issued in the while, so i made a little change that may solve this.
We want to hope that that solves the problem.
 
 Unfortunately I can reproduce it also only on a computer with Win2k. With Windows 98 on another this problem does not arise. Further Windows installations do not stand for me to the test for order. I am afraid that also because of the Windows installation is. Only which causes the problem? I use neither a on-acces virus scan still another personal firewall or the other data traffic filtering software. Router could the problem to cause?
Title: Upload broken with small files
Post by: rejetto on June 27, 2007, 01:05:59 AM
it's so strange i can't imagine anything

anyway, try the new build and see
Title: Upload broken with small files
Post by: Wasserfloh on June 28, 2007, 11:38:24 AM
anyway, try the new build and see
Unfortunately no change.
Title: Upload broken with small files
Post by: rejetto on June 28, 2007, 11:45:15 AM
what build did you test?
Title: Upload broken with small files
Post by: Wasserfloh on June 29, 2007, 11:55:50 AM
Build 107 and Build 108
Title: Upload broken with small files
Post by: rejetto on July 02, 2007, 10:29:42 PM
another user is experiencing your problem.
i was not able to reproduce it yet.
it may be related to some option. can you please furnish your options? menu -> other options -> show customized options
Title: Upload broken with small files
Post by: Wasserfloh on July 03, 2007, 03:09:15 PM
Code: [Select]
window-max=yes
default: no

easy=no
default: yes

log-date=yes
default: no

log-server-start=yes
default: no

log-server-stop=yes
default: no

log-connections=yes
default: no

log-disconnections=yes
default: no

log-replies=yes
default: no

log-progress=yes
default: no

log-banned=yes
default: no

custom-ip=192.168.30.20;127.0.0.1
default:

listen-on=127.0.0.1
default:

dynamic-dns-updater=......
default:

search-better-ip=no
default: yes

auto-save-vfs=yes
default: no

use-comment-as-realm=no
default: yes

modal-options=no
default: yes

rename-partial-uploads=%name%_part
default:

recursive-listing=no
default: yes

list-protected-items=yes
default: no

enable-no-default=yes
default: no

browse-localhost=no
default: yes

last-update-check=39229,0324770949
default: 0

speed-limit=8
default: -1

max-ips-downloading=4
default: 0

max-connections=64
default: 0

max-connections-by-ip=8
default: 0

max-contemporary-dls=4
default: 0

max-contemporary-dls-ip=1
default: 0

connections-timeout=180
default: 60

no-reply-ban=yes
default: no

last-file-open=N:\Programme\HFS\VFS\test.vfs
default:

reload-on-startup=no
default: yes

only-1-instance=no
default: yes

mime-types=*.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.txt|text/plain|*.css|text/css|*.zip|application/zip|*.rar|application/x-rar-compressed|*.7z|application/x-7z-compressed|*menu*.ini|application/x-opera-configuration-menu|*toolbar*.ini|application/x-opera-configuration-toolbar|*keybord*.ini|application/x-opera-configuration-keyboard|*.mp3|audio/mpeg|*.torrent|application/x-bittorrent
default: *.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.txt|text/plain|*.css|text/css

address2name=joerg|192.168.22.71|Validom|212.162.14.235
default:

recents=N:\Programme\HFS\VFS\test.vfs;N:\Programme\HFS\VFS\hfs.vfs;N:\Programme\HFS\VFS\00000002.vfs;N:\Programme\HFS\VFS\0000000000000001.vfs;N:\Programme\HFS\VFS\0000000000000002.vfs
default:

ips-ever-connected=127.0.0.1;87.188.224.192;91.5.223.223;87.188.207.76;87.188.216.254;212.162.14.235;192.168.30.20
default:

leave-disconnected-connections=yes
default: no

tray-instead-of-quit=yes
default: no

use-iso-date-format=yes
default: no

number-files-on-upload=no
default: yes

out-total=11942152
default: 0

in-total=212472822
default: 0

hits-total=273
default: 0

upload-total=66
default: 0

many-items-warning=no
default: yes

connections-columns=IP;179|Filename;180|Status;180|Speed;60|Time left;55|%;371|
default: IP;120|Filename;180|Status;180|Speed;60|Time left;55|%;50|

update-daily=no
default: yes

tester-updates=yes
default: no

copy-url-on-addition=no
default: yes
Title: Upload broken with small files
Post by: rejetto on July 03, 2007, 04:09:43 PM
shit, i used your settings and still it works perfectly here :(

the last chance i see, is for you to determine in exactly which build the problem was introduced, so i can study the changes.
all builds are available at www.rejetto.com/temp/hfsXXX.exe
i would be grateful if you find the time to do this, and will be happy to try to fix it.
Title: Upload broken with small files
Post by: Wasserfloh on July 03, 2007, 05:36:04 PM
the last chance i see, is for you to determine in exactly which build the problem was introduced, so i can study the changes.
The problem arises at least since Build 088. I have older Builds only briefly, but uses without Upload.

all builds are available at www.rejetto.com/test/hfsXXX.exe
Everything? No, only the test versions. In addition the www.rejetto.com/temp/hfsXXX.exe would have to be called.

i would be grateful if you find the time to do this, and will be happy to try to fix it.
Thus it looks probably bad. I do not know, how I am to otherwise still help. All Builds starting from 088 has here the problem. Both under Windows 98 and 2000 with different Browsern. On another computer with Win98 I cannot reconstruct that also.
Title: Upload broken with small files
Post by: KalleB on July 03, 2007, 10:25:34 PM
Let me try to help you with this.

My settings are:
Code: [Select]
easy=no
default: yes

port=8900
default:

log-font-name=Trebuchet MS
default:

log-font-size=8
default: 0

confirm-exit=yes
default: no

dynamic-dns-updater=......
default:

search-better-ip=no
default: yes

start-minimized=yes
default: no

files-stay-flagged-for-minutes=900
default: 0

auto-save-vfs=yes
default: no

modal-options=no
default: yes

delete-partial-uploads=yes
default: no

auto-save-vfs-every=3600
default: 0

last-update-check=39215,5131065509
default: 0

tray-shows=connections
default: downloads

tray-message=%url%\nLatauksia: %downloads%\nPyyntöjä: %hits%
default: %ip%\nUptime: %uptime%\nDownloads: %downloads%

speed-limit=800,1
default: -1

max-contemporary-dls=15
default: 0

max-contemporary-dls-ip=7
default: 0

open-in-browser=*.htm;*.html;*.jpg;*.jpeg;*.gif;*.png;*.txt;*.swf
default: *.htm;*.html;*.jpg;*.jpeg;*.gif;*.png;*.txt;*.swf;*.svg

graph-rate=100
default: 10

last-file-open=C:\Program Files\HFS\hfs.vfs
default:

mime-types=*.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.txt|text/plain
default: *.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.txt|text/plain|*.css|text/css

icon-masks=*.!ut|10001||
default:

min-disk-space=500
default: 0

out-total=396186180
default: 0

in-total=335991
default: 0

upload-total=1
default: 0

many-items-warning=no
default: yes

connections-columns=IP;170|Filename;209|Status;170|Speed;80|Time left;55|%;313|
default: IP;120|Filename;180|Status;180|Speed;60|Time left;55|%;50|

update-daily=no
default: yes

copy-url-on-addition=no
default: yes

This is my log when I try to upload 3 small txt files for the first time:
Code: [Select]
1:13:09 127.0.0.1:1945 Uploading small_test_file_1.txt
1:13:10 127.0.0.1:1945 Fully uploaded -  1,16 KB @ 37 KB/s
1:13:10 127.0.0.1:1945 Uploading small_test_file_2.txt

And this is what I experience:
Code: [Select]
- First file uploads is a sec
- Second file uploads until 99% and gets stuck there
- Stuck there for about a minute, then disappear
- Opera displays error that the page is not available

When I try second time after this I get this log:
Code: [Select]
1:19:26 127.0.0.1:2062 Uploading small_test_file_1.txt
1:19:26 127.0.0.1:2062 Fully uploaded -  1,16 KB @ 74 KB/s
1:19:26 127.0.0.1:2062 Uploading small_test_file_2.txt
1:19:26 127.0.0.1:2062 Fully uploaded -  647 B @ 20 KB/s
1:19:26 127.0.0.1:2062 Uploading small_test_file_3.txt

And this is what I experience:
Code: [Select]
- First file uploads is a sec
- Second file uploads is a sec
- At connections I see "receiving" for the third file but no percentage bar
- Stuck there for about a minute, then disappear
- Opera displays error that the page is not available

Third time is even more funnier:
Code: [Select]
1:21:42 127.0.0.1:2100 Uploading small_test_file_1.txt
1:21:42 127.0.0.1:2100 Fully uploaded -  1,16 KB @ 74 KB/s
1:21:42 127.0.0.1:2100 Uploading small_test_file_2.txt

And this is what I experience:
Code: [Select]
- First file uploads is a sec
- Second file is somehow skipped (!)
- Third file uploads until 99% and gets stuck there (!!!)
- Stuck there for about a minute, then disappear
- Opera displays error that the page is not available

Fourth time exactly the same as third, but I don't want to wait the minute so I cancel upload and try fifth time: All files SUCCEED!

Code: [Select]
1:23:48 127.0.0.1:2152 Uploading small_test_file_1.txt
1:23:48 127.0.0.1:2152 Fully uploaded -  1,16 KB @ 79 KB/s
1:23:48 127.0.0.1:2152 Uploading small_test_file_2.txt
1:23:48 127.0.0.1:2152 Fully uploaded -  647 B @ 40 KB/s
1:23:48 127.0.0.1:2152 Uploading small_test_file_3.txt
1:23:48 127.0.0.1:2152 Fully uploaded -  2,25 KB @ 153 KB/s
1:23:48 127.0.0.1:2152 Requested POST /folder/

Weird thing I'd say...Tomorrow I'll test on other machine and browser.

Here's an image what my upload folder looks like after all this:
Title: Upload broken with small files
Post by: rejetto on July 04, 2007, 11:35:15 AM
the last chance i see, is for you to determine in exactly which build the problem was introduced, so i can study the changes.
The problem arises at least since Build 088. I have older Builds only briefly, but uses without Upload.

so, can you confirm that upload in build #087 (http://www.rejetto.com/temp/hfs087.exe) works?

Quote
all builds are available at www.rejetto.com/test/hfsXXX.exe
Everything? No, only the test versions. In addition the www.rejetto.com/temp/hfsXXX.exe would have to be called.

you're right, sorry
Title: Upload broken with small files
Post by: Wasserfloh on July 04, 2007, 08:05:08 PM
The problem arises at least since Build 088. I have older Builds only briefly, but uses without Upload.
so, can you confirm that upload in build #087 (http://www.rejetto.com/temp/hfs087.exe) works?
In the meantime I tested the also following Builds: #067, #072, #083, #084 and #085. Everywhere there is the problem. That will be in Build #087 probably not different.
Title: Upload broken with small files
Post by: rejetto on July 04, 2007, 10:00:40 PM
thank you.
can you test 2.0b ? www.rejetto.com/files/hfs/hfs2.0b.zip

can i also ask you how did you set the default options for the test? the best way is to hold the SHIFT key while HFS starts. but this way you will loose your settings if you do not backup before.
Title: Upload broken with small files
Post by: Wasserfloh on July 04, 2007, 11:03:47 PM
can you test 2.0b ? www.rejetto.com/files/hfs/hfs2.0b.zip
The problem exists also here.

can i also ask you how did you set the default options for the test? the best way is to hold the SHIFT key while HFS starts. but this way you will loose your settings if you do not backup before.
That does not make a difference.
Title: Upload broken with small files
Post by: rejetto on July 04, 2007, 11:07:22 PM
version 2.0 is 2 years old
i wonder why only now people came with this problem  :-\

i see you are testing on 127.0.0.1
what if you try over a network? (LAN or internet)
Title: Upload broken with small files
Post by: Wasserfloh on July 04, 2007, 11:44:30 PM
i see you are testing on 127.0.0.1
what if you try over a network? (LAN or internet)
No difference.
Title: Upload broken with small files
Post by: rejetto on July 05, 2007, 12:37:27 AM
anyway... this means this bug is not a reason to delay official publishing of 2.2, since it is present by all previous versions.
i hope we'll find what it is.
Title: Upload broken with small files
Post by: KalleB on July 05, 2007, 08:37:40 AM
The problem persists here on other PC with HFS B111 with default settings and firefox..

Uploading small txt-files randomly gets stuck at 99% or "receiving". Only difference is that in this machine the files upload completely when "Uploading small_test_file_X.txt" appears...does not seem to need that "Fully uploaded" notification.
Title: Upload broken with small files
Post by: MarkV on July 05, 2007, 06:02:39 PM
version 2.0 is 2 years old
i wonder why only now people came with this problem  :-\

Maybe because Mikkisoft has been sending us broken 'updates' for the TCP/IP stack? Just guessing...
Title: Upload broken with small files
Post by: rejetto on July 06, 2007, 11:17:10 AM
nice try ...
kalleb, what service pack is installed on your win2000 ?
and can you try enabling: log what -> disconnection
and see who is actually breaking the connection. if no "by server" is specified, then it is the client.

Title: Upload broken with small files
Post by: KalleB on July 06, 2007, 01:01:05 PM
Um, I have WinXP SP2 with all the latest updates..on both my machines ..never have had Win2k.

Using Firefox and latest HFS 111 I made HFS log almost everything and this is what I got:

Code: [Select]
15:35:08 10.100.4.47:1799 Connected
15:35:08 10.100.4.47:1799 Got 407 bytes
15:35:08 10.100.4.47:1799 Requested GET /NewFolder/
15:35:08 10.100.4.47:1799 Request dump
15:35:08 10.100.4.47:1799 Sent 1049 bytes
15:35:08 10.100.4.47:1799 Served 1,02 KB
15:35:17 10.100.4.47:1799 Requested GET /NewFolder/~upload
15:35:17 10.100.4.47:1799 Request dump
15:35:17 10.100.4.47:1799 Sent 1279 bytes
15:35:17 10.100.4.47:1799 Served 1,30 KB
15:35:17 10.100.4.47:1799 Got 733 bytes
15:35:26 10.100.4.47:1799 Uploading small_test_file_1.txt
15:35:26 10.100.4.47:1799 Fully uploaded -  428 B @ 8 KB/s
15:35:26 10.100.4.47:1799 Uploading small_test_file_2.txt
15:35:26 10.100.4.47:1799 Fully uploaded -  1,72 KB @ 176 KB/s
15:35:26 10.100.4.47:1799 Uploading small_test_file_3.txt
15:35:26 10.100.4.47:1799 Got 1172 bytes
15:36:26 10.100.4.47:1799 Disconnected by server: inactivity - 4976 bytes sent
15:36:26 10.100.4.47:1821 Connected
15:36:26 10.100.4.47:1821 Got 453 bytes
15:36:26 10.100.4.47:1821 Uploading small_test_file_1.txt
15:36:26 10.100.4.47:1821 Fully uploaded -  428 B @ 21 KB/s
15:36:26 10.100.4.47:1821 Uploading small_test_file_2.txt
15:36:26 10.100.4.47:1821 Fully uploaded -  1,72 KB @ 58 KB/s
15:36:26 10.100.4.47:1821 Uploading small_test_file_3.txt
15:36:26 10.100.4.47:1821 Fully uploaded -  5,53 KB @ 283 KB/s
15:36:26 10.100.4.47:1821 Requested POST /NewFolder/
15:36:26 10.100.4.47:1821 Request dump
15:36:26 10.100.4.47:1821 Sent 1026 bytes
15:36:26 10.100.4.47:1821 Served 1,00 KB
15:36:26 10.100.4.47:1821 Disconnected by server - 2413 bytes sent
15:36:26 10.100.4.47:1822 Connected
15:36:26 10.100.4.47:1822 Got 407 bytes
15:36:26 10.100.4.47:1822 Sent 809 bytes
15:36:26 10.100.4.47:1822 Disconnected by server - 809 bytes sent
15:36:26 10.100.4.47:1823 Connected
15:36:26 10.100.4.47:1823 Got 407 bytes
15:36:26 10.100.4.47:1823 Sent 809 bytes
15:36:26 10.100.4.47:1823 Disconnected by server - 809 bytes sent


The uploading got stuck "receiving" on the third file again...it was there for a minute and then all of a sudden uploaded all three files again and showed the OK notifications. So now I had those uploded files duplicated:
Title: Upload broken with small files
Post by: KalleB on July 06, 2007, 01:13:55 PM
Tested second time with the same setup:

Code: [Select]
16:08:00 10.100.4.47:2457 Requested GET /NewFolder/
16:08:00 10.100.4.47:2457 Sent 1201 bytes
16:08:00 10.100.4.47:2457 Served 1,03 KB
16:08:00 10.100.4.47:2457 Got 363 bytes
16:08:01 10.100.4.47:2457 Requested GET /NewFolder/~upload
16:08:01 10.100.4.47:2457 Sent 2331 bytes
16:08:01 10.100.4.47:2457 Got 370 bytes
16:08:01 10.100.4.47:2457 Served 1,30 KB
16:08:01 10.100.4.47:2473 Connected
16:08:01 10.100.4.47:2473 Got 370 bytes
16:08:01 10.100.4.47:2474 Connected
16:08:01 10.100.4.47:2474 Got 370 bytes
16:08:07 10.100.4.47:2457 Uploading small_test_file_1.txt
16:08:07 10.100.4.47:2457 Got 7430 bytes
16:08:07 10.100.4.47:2457 Fully uploaded -  2,14 KB @ 43 KB/s
16:08:07 10.100.4.47:2457 Uploading small_test_file_3.txt
16:09:02 10.100.4.47:2473 Disconnected by server: inactivity - 623 bytes sent
16:09:02 10.100.4.47:2474 Disconnected by server: inactivity - 111 bytes sent
16:09:08 10.100.4.47:2457 Disconnected by server: inactivity - 5448 bytes sent
16:09:08 10.100.4.47:2491 Connected
16:09:08 10.100.4.47:2491 Got 453 bytes
16:09:08 10.100.4.47:2491 Uploading small_test_file_1.txt
16:09:08 10.100.4.47:2491 Fully uploaded -  428 B @ 20 KB/s
16:09:08 10.100.4.47:2491 Uploading small_test_file_2.txt
16:09:08 10.100.4.47:2491 Fully uploaded -  1,72 KB @ 88 KB/s
16:09:08 10.100.4.47:2491 Uploading small_test_file_3.txt
16:10:08 10.100.4.47:2491 Disconnected by server: inactivity

HFS received the first one and got stuck on the third (skipped second) file. Then after a minute it uploaded all three again successfully...but this time no OK notifications..only white screen in Firefox.

Edit: Out of curiosity, I also tested with small (>1kB) binary files and got about the same results as so far with text files.

Edit2: Same thing with a bit bigger (~10kB) files...

Edit3: Big (~1MB) files still upload is a split second without any problems:

Code: [Select]
16:25:53 10.100.4.47:2804 Connected
16:25:53 10.100.4.47:2804 Got 407 bytes
16:25:53 10.100.4.47:2804 Requested GET /NewFolder/
16:25:53 10.100.4.47:2804 Sent 1051 bytes
16:25:53 10.100.4.47:2804 Served 1,03 KB
16:25:54 10.100.4.47:2804 Requested GET /NewFolder/~upload
16:25:54 10.100.4.47:2804 Sent 1279 bytes
16:25:54 10.100.4.47:2804 Served 1,30 KB
16:25:54 10.100.4.47:2804 Got 733 bytes
16:26:04 10.100.4.47:2804 Uploading big_binary_file_1.pdf
16:26:04 10.100.4.47:2804 Got 8236 bytes
16:26:04 10.100.4.47:2804 Fully uploaded -  1,33 MB @ 11642 KB/s
16:26:04 10.100.4.47:2804 Uploading big_binary_file_2.pdf
16:26:04 10.100.4.47:2804 Fully uploaded -  1,81 MB @ 10566 KB/s
16:26:04 10.100.4.47:2804 Uploading big_binary_file_3.pdf
16:26:04 10.100.4.47:2804 Fully uploaded -  1,98 MB @ 9397 KB/s
16:26:04 10.100.4.47:2804 Requested POST /NewFolder/
16:26:04 10.100.4.47:2804 Sent 2650 bytes
16:26:04 10.100.4.47:2804 Served 1,00 KB
16:26:05 10.100.4.47:2804 Disconnected by server - 7396 bytes sent
16:26:05 10.100.4.47:2809 Connected
16:26:05 10.100.4.47:2809 Got 407 bytes
16:26:05 10.100.4.47:2809 Sent 809 bytes
16:26:05 10.100.4.47:2809 Disconnected by server - 809 bytes sent
16:26:05 10.100.4.47:2810 Connected
16:26:05 10.100.4.47:2810 Got 407 bytes
16:26:05 10.100.4.47:2810 Sent 809 bytes
16:26:05 10.100.4.47:2810 Disconnected by server - 809 bytes sent


So this obviously seems to be a size related issue...
Title: Upload broken with small files
Post by: rejetto on July 06, 2007, 05:25:22 PM
hmm, the point is: i tried with 3byte files, and got no problem :(
do you have a sniffer, so you could send me a dump of the whole traffic? you can use even smaller files, to make it easy to read.
you find many free sniffers on snapfiles (http://www.snapfiles.com/Freeware/network/fwpacketsniffer.html).
Title: Upload broken with small files
Post by: KalleB on July 07, 2007, 06:24:08 PM
I tried with 3 byte files and got pretty much the same results as so far... sent you the sniff via gmail.
Title: Upload broken with small files
Post by: MarkV on July 07, 2007, 06:38:30 PM
rejetto you should split the topic as this problems is with later builds also.
Title: Upload broken with small files
Post by: rejetto on July 07, 2007, 06:40:34 PM
as far as they say, it's with every version since 2.0
Title: Re: Upload broken with small files
Post by: Foggy on July 08, 2007, 08:01:53 AM
I dont quite understand what the problem is. Is it where uploads to HFS get broken? Sorry but im a little thick, just askin so that i may try and reproduce it.
Title: Re: Upload broken with small files
Post by: KalleB on July 08, 2007, 08:46:05 AM
Basically uploading small files (0,001-10 kB) does not work. Try uploading three of these files and only first and second one might upload... normally only the first one. Sometimes (on my other PC) HFS even uploads the files twice so that first round gets stuck on some file, but then all of a sudden all three upload again successfully. This has to be an OS related issue since the results vary a bit on different machines.
Title: Re: Upload broken with small files
Post by: Foggy on July 08, 2007, 09:24:53 AM
Thanks for explaining the problem, I tested uploading 4 different files locally they were of varying size between 42bytes and 10.6kb.
I uploaded 2 of them seperatly and all was fine.
I also uploaded all 4 at once and everything was sucessful.
I was using build 088 on an XP Pro machine with SP2.

I just dont get why it works for some and not for others. ???
Title: Re: Upload broken with small files
Post by: maverick on July 08, 2007, 11:11:09 AM
I did some testing here too.  Latest build 112.  Uploaded 4 small files ranging in size from 65b to 15kb.  Tried this a few times with different small files.  All files uploaded successfully without error.  WinXP Pro SP2.

I have never seen the problem described in this thread with any version or build.
Title: Re: Upload broken with small files
Post by: Foggy on July 08, 2007, 01:01:32 PM
KalleB you said you tried on different computers, was there any antivirus\firewall that is on all the computers you tried? it could just be you have some software blocking the upload for some reason.
Title: Re: Upload broken with small files
Post by: KalleB on July 08, 2007, 01:31:22 PM
No I don't think those have any effect since bigger files ~500kB and above always succeed...it's only the small ones that fail.

I've tested on two machines with WinXP SP2 Home, HFS b110-112, Firefox, Opera & IE,
other has F-Secure suite and the another Avast & Kerio.

I've mostly used the same machine as client (connecting to 127.0.0.1) but I also tried Kubuntu Feisty with Firefox as client and got the same results... will try it soon also as host using WINE.

EDIT: Using Kubuntu as host..surprise, surprise..uploading works perfect.
Title: Re: Upload broken with small files
Post by: rejetto on July 17, 2007, 05:59:59 PM
Please, test the last build and tell me...
Title: Re: Upload broken with small files
Post by: KalleB on July 17, 2007, 07:24:09 PM
I wish I could say it's fixed...and I almost did since the first time I tried with 119 it worked! ..BUT any of the tests after that not. No matter if I restart HFS. It's still almost impossible to upload three small files, uploading only two succeeds in about 60% of the cases and one in about 90%.  :-\

But I'm quite happy with the current situation since afterall it's not really HFS bug but something about Windows XP because it works with WINE and not many people (Wasserfloh and me) have reported this...don't let this delay next stable release.

I'll just have to figure out what is the common factor in three of my machines that causes this in them...
Title: Re: Upload broken with small files
Post by: Foggy on July 18, 2007, 02:42:53 AM
An idea I had but I was too lazy to go back and read the whole thread to see if its been answered already.
What browser are you using?
Title: Re: Upload broken with small files
Post by: rejetto on July 18, 2007, 02:44:57 AM
it's written in the first post of the thread
Title: Re: Upload broken with small files
Post by: Foggy on July 18, 2007, 03:45:14 AM
I downloaded the latest version of opera and I started to get the same results as KalleB with some uploading succesfully and others not.
Title: Re: Upload broken with small files
Post by: rejetto on July 18, 2007, 03:46:28 AM
i use the last opera version too
Title: Re: Upload broken with small files
Post by: KalleB on July 18, 2007, 09:18:08 AM
I've tested with Opera9, FF2 and IE7 so I don't think it's has much to do with the browser...
And only thing these machines have in common it the Finnish language setting really :D
Title: Re: Upload broken with small files
Post by: rejetto on August 18, 2007, 10:02:24 AM
too many people is getting this problem.
we must find what is causing it, so i can fix it  :'(
Title: Re: Upload broken with small files
Post by: rejetto on August 31, 2007, 03:13:01 PM
Guys... we need to test build #128
there is a chance it was fixed.
Title: Re: Upload broken with small files
Post by: Wasserfloh on September 01, 2007, 12:49:28 AM
Problem continues to exist here unfortunately.
Title: Re: Upload broken with small files
Post by: rejetto on September 01, 2007, 01:39:24 PM
 :'(
someone else reported it was fixed
can you confirm you tested build #128 please?
Title: Re: Upload broken with small files
Post by: KalleB on September 01, 2007, 08:08:18 PM
No change here with #128, sorry :'(

I think what we need is a totally different approach to this problem. Could you make a separate beta version of HFS with the uploading part totally rewritten and using totally different "methods" for delivering the data? Don't mind if it would break everything else in HFS... We need something radical to beat this irritating little bug.
Title: Re: Upload broken with small files
Post by: TSG on September 02, 2007, 02:04:31 AM
Tested again, it appears fixed to me...
Title: Re: Upload broken with small files
Post by: maverick on September 02, 2007, 03:51:09 AM

Strange, but I never had that problem with any build.  Everything is working just fine.
Title: Re: Upload broken with small files
Post by: Giant Eagle on September 02, 2007, 09:55:53 AM
Yes, it seems to be fixed on my end aswell, however, i have noticed something weird.

When uploading a small .gif file of around ~300 bytes. The status on HFS displays that the file has been received for 99%, the byte counter displays: 450/450 and it just hangs there for a minute. After that, the file has been fully uploaded.

In that minute when HFS 'hangs', you can already access the uploaded file with no problems. But if you do, hfs creates a second file image(1).gif when the minute is over.

very strange :P
Title: Re: Upload broken with small files
Post by: maverick on September 02, 2007, 11:06:43 AM
Yes, it seems to be fixed on my end aswell, however, i have noticed something weird.

When uploading a small .gif file of around ~300 bytes. The status on HFS displays that the file has been received for 99%, the byte counter displays: 450/450 and it just hangs there for a minute. After that, the file has been fully uploaded.

In that minute when HFS 'hangs', you can already access the uploaded file with no problems. But if you do, hfs creates a second file image(1).gif when the minute is over.

Tested here again, this time with small thumbs (gif) 10-12 kb in size.  No problem.  All uploaded successfully.

I didn't see anything in HFS status saying 99% received or HFS hanging for a minute.  Nothing like that at all.  As the files I uploaded were small, the uploads happened very quickly and the log shows 'Fully Uploaded' with the correct filesize for all of files.

What I find strange is that some people report problems uploading small files and others like myself have never seen this problem.  To me, it seems system specific for some reason.  As everyone is running the same HFS engine, it might have something to do with the o/s, browser, or maybe even the template used.

Edit:
I don't think this was mentioned in this thread or not, but for the ones who noticed problems did they appear when testing locally, remotely, or both?
Title: Re: Upload broken with small files
Post by: Giant Eagle on September 02, 2007, 12:13:05 PM
Try 1kb at most when testing images, cause everything above 1kb seems to upload fine (or a very low chance to get an error)

I tested it under the following circumstances:

both locally and remotely on 4 different pc's.
windows 2k pro sp4 - windows xp pro sp2 - windows server 2k3 enterprise
default template - terayon - tog

speed of the pc's varies from 500mhz to dual core 2.4ghz
memory from 384 to 2048mb

browser: IE6 and FireFox 2.0

Seems like it has nothing to do with the system..
Title: Re: Upload broken with small files
Post by: Foggy on September 02, 2007, 01:06:41 PM
When uploading a small .gif file of around ~300 bytes. The status on HFS displays that the file has been received for 99%, the byte counter displays: 450/450 and it just hangs there for a minute. After that, the file has been fully uploaded.

I got the same but not too sure about the last part about accessing it and a image(1) being created because I've been uploading the same 800byte gif.

I also got a screen shot of my hfs while it is stuck on 99%

also some of my tests have been timing out without hfs receiving any data and some are succesful
Title: Re: Upload broken with small files
Post by: maverick on September 02, 2007, 03:08:46 PM
Just tested with smaller files.  Uploaded successfully.  No errors.  HFS not stuck at 99%.  No second file created with (1) added to filename.

Don't know why some of you are having problems.  I can't think of anything else to suggest.

Title: Re: Upload broken with small files
Post by: KalleB on September 02, 2007, 06:09:50 PM
Tested locally with three small files using build #128. Here's the log w/ comments:

Quote
20:53:07 127.0.0.1:3180 Requested GET /upload folder/~upload
20:53:07 127.0.0.1:3180 Served 1,30 KB
20:53:07 127.0.0.1:3180 Got 601 bytes
20:53:16 127.0.0.1:3181 Disconnected - 92642 bytes sent
20:53:16 127.0.0.1:3186 Got 712 bytes
20:53:16 127.0.0.1:3186 Uploading 9B_test_file.txt
 ### only 9B_test_file.txt visible in the upload folder
 ### 76B_test_file.txt stuck at 99% in the connections table
20:54:08 127.0.0.1:3180 Disconnected by server: inactivity - 46061 bytes sent
20:54:17 127.0.0.1:3186 Disconnected by server: inactivity
 ### Browser returns "page not found"
 ### 9B_test_file.txt disappears from the upload folder, folder empty

So not much change from my previous logs reports.

Second test:
Quote
21:00:19 127.0.0.1:3207 Got 580 bytes
21:00:19 127.0.0.1:3207 Requested GET /upload folder/
21:00:19 127.0.0.1:3207 Served 1,02 KB
21:00:19 127.0.0.1:3208 Got 594 bytes
21:00:19 127.0.0.1:3209 Got 594 bytes
21:00:19 127.0.0.1:3210 Got 594 bytes
21:00:27 127.0.0.1:3207 Disconnected - 2668 bytes sent
21:00:27 127.0.0.1:3211 Got 712 bytes
21:00:27 127.0.0.1:3211 Uploading 9B_test_file.txt
21:00:27 127.0.0.1:3211 Fully uploaded -  9 B @ 0 KB/s
21:00:27 127.0.0.1:3211 Uploading 76B_test_file.txt
 ### 9B_test_file.txt & 76B_test_file.txt visible in the upload folder
 ### The last file, 261B_test_file.txt, stuck at 99%
21:01:20 127.0.0.1:3208 Disconnected by server: inactivity - 583 bytes sent
21:01:20 127.0.0.1:3209 Disconnected by server: inactivity - 118 bytes sent
21:01:20 127.0.0.1:3210 Disconnected by server: inactivity - 578 bytes sent
21:01:28 127.0.0.1:3211 Disconnected by server: inactivity
 ### Browser returns "page not found"
 ### 76B_test_file.txt disappears from the upload folder, only 9B_test_file.txt left there

Third test same as the second.

Fourth one:

Quote
21:04:32 127.0.0.1:3215 Got 555 bytes
21:04:32 127.0.0.1:3215 Requested GET /upload folder/
21:04:32 127.0.0.1:3215 Served 1,02 KB
21:04:39 127.0.0.1:3215 Disconnected - 1046 bytes sent
21:04:39 127.0.0.1:3216 Got 712 bytes
21:04:39 127.0.0.1:3216 Uploading 9B_test_file.txt
21:04:39 127.0.0.1:3216 Fully uploaded -  9 B @ 0 KB/s
21:04:39 127.0.0.1:3216 Uploading 76B_test_file.txt
21:04:39 127.0.0.1:3216 Fully uploaded -  76 B @ 5 KB/s
21:04:39 127.0.0.1:3216 Uploading 261B_test_file.txt
 ### All three files visible in the upload folder
 ### 261B_test_file.txt stuck "receiving" (no %-bar visible)
21:05:39 127.0.0.1:3216 Disconnected by server: inactivity
 ### Browser returns "page not found"
 ### 261B_test_file.txt disappears from the upload folder, 9B_test_file.txt and 76B_test_file.txt succesfully uploaded


Fifth test same as the first one.

Sixth test same as the second one.

Seventh & eighth test same as the first one... so it's totally random.

...exactly the same results as before.
Title: Re: Upload broken with small files
Post by: Guest on September 02, 2007, 06:15:12 PM
Small file bug
All small files do upload suscessfully for me, but
I notice that anything 200 bytes or below doesn't give upload
sucess page, nor are they added to upload total. 204 bytes does.
Note that when no upload sucess page comes up I must after
a few seconds use cancel upload to return to the folder uploaded to.
 I also notice that the file size will not show in the windows
folder window status bar for any 200 bytes or smaller, yet
when I copy and paste a 195 byte file it does show. This
might be because of the cancel?
This is build 128 on Win 98se uploaded on local address (lan),
not across internet. Using mod template, but with default template
upload page. I noticed some time ago, when I used a mod (js) upload
page, I had the files being combined problem, so I went back to default.
For me this goes back a long time. I always thought the transaction
just occured to fast to be recorded when local, and never considered
it a real problem, as very small files are rarely sent to me.
If I get time I'll try the same on Vista.
Best test would be between one with the problem and one
without across open internet, though?
Hope this info is of some meaning to someone.
Title: Re: Upload broken with small files
Post by: maverick on September 02, 2007, 08:41:41 PM
All small files do upload suscessfully for me, but
I notice that anything 200 bytes or below doesn't give upload
sucess page

I get the upload success page.  Here is a screenshot of my upload success page for a 45 byte file.

Maybe the other problems you mention have something to do with Win98.
Title: Re: Upload broken with small files
Post by: rejetto on September 02, 2007, 10:06:23 PM
the bugs i already fixed depend on how the stream data is splitted in chunks while transfered.
this depends both on the underlying network and the client application.
i will try to make a test application that sends the data byte by byte... it may help discovering more bugs.
Title: Re: Upload broken with small files
Post by: rejetto on September 03, 2007, 03:47:23 AM
i made the test, and it was very useful because i found some bugs, all possibly affecting the upload procedure.
i fixed them, so let me know if the problem is fixed for those of you still experiencing the problem.
www.rejetto.com/forum/?topic=4890
Title: Re: Upload broken with small files
Post by: Foggy on September 03, 2007, 10:18:46 AM
I just tried uploading some files one 800byte file and some around 40-100kb and all worked 100% of the time.
Title: Re: Upload broken with small files
Post by: KalleB on September 03, 2007, 02:38:05 PM
No change here with #129, randomly stuck 99% or "receiving" on some file. On first test I got the List index out of bounds (1) when HFS was disconnecting a timed out upload ...but not after that anymore.

Here's the log of the third test:
Quote
17:28:56 127.0.0.1:1280 Got 555 bytes
17:28:56 127.0.0.1:1280 Requested GET /upload folder/
17:28:56 127.0.0.1:1280 Served 1,02 KB
17:29:03 127.0.0.1:1280 Disconnected - 1047 bytes sent
17:29:03 127.0.0.1:1281 Got 712 bytes
17:29:03 127.0.0.1:1281 Uploading 9B_test_file.txt
17:29:03 127.0.0.1:1281 Fully uploaded -  9 B @ 0 KB/s
17:29:03 127.0.0.1:1281 Uploading 76B_test_file.txt
17:29:04 127.0.0.1:1281 Fully uploaded -  76 B @ 4 KB/s
17:29:04 127.0.0.1:1281 Uploading 261B_test_file.txt
 ## All three files visible in the upload folder
 ## 261B_test_file.txt stuck "receiving"
*** I think here should be "Disconnected by server: inactivity" but it gave it only in the first test (along with List index out of bounds (1)).
 ## Browser returns "page not found"
 ## 127.0.0.1 displaying "disconnected" in connections dialog, cannot be kicked!
 ## Unable to delete 261B_test_file.txt from the upload folder, other two delete OK

..had to restart HFS to be able to delete the file and get 127.0.0.1 away from the connections dialog.
Title: Re: Upload broken with small files
Post by: rejetto on September 03, 2007, 03:37:54 PM
thanks for trying kalleb
Title: Re: Upload broken with small files
Post by: Giant Eagle on September 05, 2007, 03:18:26 PM
Hmm, seems like we have ourselves another problem. The exact oposite of this topic.

I tried to upload a file that was around 4,4 gigs today and got stuck at 99% with the status 'Receiving' and disconnected after 2 minutes. This is not even the strangest part.

HFS showed that the file that was being received was only 265mb in size. :S I figured that the upload speed of my school was VERY fast (~1150KBp/s), so i waited till those 265mb were uploaded.

Maybe someone else can confim this.


BTW, about the file that is being transferred.. ::) It's a renamed .rar archive that contains my homework, its not the original movie in HD quality or something... really!
Title: Re: Upload broken with small files
Post by: mike on September 05, 2007, 04:47:47 PM
I tried to upload a file that was around 4,4 gigs today
.
.
.
It's a renamed .rar archive that contains my homework
You got 4.4 gigs of homework!  WOW!  I'm surprised your school allows such transfers.
Title: Re: Upload broken with small files
Post by: rejetto on September 05, 2007, 06:21:00 PM
didn't you think about the browser you used?
i tested upload with IE7, Opera 9.2 and FF2, with a 5GB file.

FF does... NOTHING! very strange, when i click upload it just doesn't send anything to the server.
Other 2 browsers send the upload request, but with wrong file size. They don't support 4GB+ for sure. I didn't test for a 3GB file.

Doesn't matter how i try to support 4GB+, the browsers don't want to... :(
Title: Re: Upload broken with small files
Post by: Giant Eagle on September 05, 2007, 09:14:56 PM
I used IE6, so that explains it :P

Next time, ill just have to rar it up and split it in half or something.