Problem with BIG files

[Caco]

I have to move big files to my server, but no browser seem capable to do that. Rejetto once said HFS is tested succesfully even with 10GB+ files.

I tried with a 5 GB avi file:

Internet Explorer 7 - FAILED

Mozilla Firefox - FAILED

Google Chrome - FAILED

How can i solve? 

[maverick]

The problem isn't with HFS but with the limitations of the Browser used.  Firefox shouldn't have a problem with a 5GB file.

[Caco]

Shouldn't... but somehow it has! I also had problem with a 2,6 GB file, no problems with files <2GB. Firefox 3.0.10.

[SilentPliz]

   
On what version of Windows running your server?
Your file system should it not be in fat32?

FAT32 impose a limits for file size of less than 2GB... This maybe the cause of your problems.   

[Caco]

Win XP SP3. All partitions are NTFS 

[rejetto]

you say "to move", that's not very clear.
you should report exactly what action/phase is failing in handling the big file, and report some information, like you should prove it to a jury.
it is useful for us to understand how to solve.

[Caco]

TEST 1

environment: dedicated gigabit LAN (no internet traffic, only HFS uploads/downloads)

test file: test.avi 4.486.729.532 byte

hfs build: #239

hfs template: RAWR template 0.1.1

server OS: Windows XP SP3 (NTFS)

client OS:  Windows XP SP2

client browser: Mozilla Firefox 3.0.10

action: upload test.avi to HFS

result: upload doesn't start, no error shown on browser , no entries in ExceptionsLog on HFS, HFS still works (in other tests crashed)


----at this point i put test.avi manually on server via windows shared folder----

action: download test.avi from HFS

result: download success , 10,5 MB/s average speed, about 6 minutes to complete

 

[rejetto]

we got it.
it's the upload.
and it's a problem of the browser.
indeed, from my latest tests (many months ago) i found that common browsers were limited to 2GB.
i found a way to rise this limit to 4GB with a trick
but that's the max i could get, and your file is over 4GB.
http://www.google.com/search?q=firefox+post+upload+limit+gb

let me know if you find any way to overcome this limit.
maybe using a flash applet?

if you don't need to use a browser, you may think of a solution using CURL. (but i don't know if it supports 4GB+ uploads)

[Caco]

let me know if you find any way to overcome this limit.

Yes, using Opera browser! 

I think the problem is that almost all browsers store the file size in a 32-bit integer, so the limit is 4GB anyways. I don't know how Opera overcomes this problem, but it does! 

I must say that before the beginning of upload, Opera seemed frozen for about 30 secs, but after that all went good.

Now i'm ready for the ultimate test: uploading (and flash encoding) a 10 GB monster file 

[Caco]

I tried to upload a 27 GB file 

The "freeze time" of Opera lasted about 15 minutes, then upload began but when it was at 50% the speed dropped from 10,5 MB/s to 90 KB/s, and i'm sure that no other connections were established to the server.

[rejetto]

yes, i used opera for many years, and remember it had that delay problem with big uploads.
considered CURL ?

[Caco]

Successfully uploaded, encoded and downloaded again 10.8 GB file in 58 minutes. For me is acceptable 

Well , it would be wonderful to have a system in which user can select a portion of video file to download, but i have no idea at all about its implementation

[80063r]

I use Vista x64 and ntfs and I know that when I use explorer to copy/paste from drive to drive, sets of files exceeding 2GB often fail.  My workaround is using a media browser (ACDSee) to copy paste collections of files.  I realize this is a slightly different scenario.

Anyway, what about Internet Explorer 64-bit.  Has anyone tried that?  I could possibly set up to test it but it would take me a couple of weeks to move files to backup and verify that my IP and DNS and everything is still working.  Sorry.

[MarkV]

i found a way to rise this limit to 4GB with a trick Cheesy

Didn't find anything. What trick?

[rejetto]

i found a way to rise this limit to 4GB with a trick Cheesy

Didn't find anything. What trick?

it's all in this line

Code: [Select]

  // the browser may not support 2GB+ files. This workaround works only for files under 4GB.
  if post.length < 0 then inc(post.length, int64(2) shl 31);

when the server receives a negative length, it tries to fix this number, hoping it is between 2 and 4 gigs.
(the same number would be sent to the server for a file between 6 and