On-the-fly zipped folders

[mrT]

First of all, a very useful piece of software, thanks a lot for sharing it.

The following feature would make it even more usefull in my opinion: Directly sharing a folder using HFS is very easy, but downloading the folder has to be done on a per-file basis. It would be great if the folders could be archived and/or zipped on-the-fly (either with compression for size, or without for speed) so that directory structures can be downloaded with ease

[Anonymous]

If you are planning to share a complete folder, to someone who actually wants your complete folder, why don't you just zip it up first yourself then include it the virtual file system?  Very easy and quick to do.

As for zipping on the fly I doubt very much that would happen.  IMHO that would be a useless feature.

[mrT]

I disagree. More often than not, one wants to download multiple files. "Real" file transfer clients (FTP, SCP) allow you to select multiple files (and folders) for dowload. The advantage of using HFS is that one does not need a specific client but one can a normal browser. The main disadvantage, as i see it, is that selection of multiple files is nearly impossible.

We could think of creating a smart AJAX http://en.wikipedia.org/wiki/AJAX javascript interface that can queue files, but that would be a rather large project.

Instead, a simpler approuch is to have a link "download folder" next to each folder that calls zips the respective folder (without compression this is quite fast) and passes it as a file. This is not that difficult. We have implemented it in some of our PHP application and I am sure that zip libraries are easy find, either as (c,c++ ?) source code or as linkable dll's

I disagree with the notion that it is just as easy to first zip the file by hand.
The lazy argument is that it saves one the trouble of creating a zip file.
The better argument is that in many cases, one does not know which directory is wanted by the client.

Consider:

- My documents
   - My Music  
     - Artist A
       - CD 1
       - CD 2
     - Artist B
       - CD 1
   - My Images

It could very well be that this person either wants 1) all files 2) all music 3) music from one artist 4) a particular CD.

[Druidor]

Why not add a tick box feature where you can browse the folders & tick the relevent files/folders you want to download & then just press a single download button to activate the download ofthose selected.

May be I am talking ou of my arse but I am new to this  :lol:

[~GeeS~]

Why not just use a decent browser and/or wget and/or teleport (pro) to rip the complete site with one click. Why bringing all the load to the serverside?
Javascript? You are free to put as much as you want on a site, but for me it's a waste of time and bandwith, because I filter or block it anyway. :evil:

[mrT]

@~GeeS~

Please show me which browser (without plugins) that can (recursively) traverse the page structure as it is created by HFS, and download the linked files in a way that reflects (a part of) the (virtual) file system indexed by HFS. I use Mozilla and IE and I have not yet found that feature. I am sure that there are site leechers that can achieve this, but what would be the point?

In my opinion, the main purpose of HFS is to make files sharable in the easiest possible manner and indeed, it works nearly perfect for this. If I want to download something from behind any computer from this is possible, as long as it has a browser (even lynx would do). I won't be lured into the discussion what a "decent browser" is ;-) , but changes are, the browser available is IE. Off course, you could first download firefox/netscape/opera/teleport etc, but in that case you may just as well use a FTP/SHHD server and download a SFTP/SCP client!

However, I do agree with you that this feature would add (significant?) server load, and it would be great if the HFS administrator would be able to turn the feature on and off. For me, the number of downloads is fairly low. I use HFS to share parts of my disk with a few not-so-tech-savy persons and for me this feature would be great.

@Druidor

This is very close to my javascript suggestion: In order for your idea to work, a javascript should pass all files to your browser. However, it would be up to the user to individually save the files and put them in the correct directory. For multiple levels of directories and perhaps 100-1000's of files, this would become tedious. An archive would require only a single download and could contain the full directory structure.

Anyway, I will stop defending this idea. Anyone can have his/her opinion about it and it is up to rejetto to see if he thinks it is worth implementing (or anybody else, for that matter).

[mrT]

Just one last thing (I know, I promissed :-) ).

Zipping (with no compression) 108 Mb in 225 in 34 folders on my PC took 12 seconds. I think this is not to bad. Performance would improve a lot, if the zipped folders where to be cached. In fact, when using cached folders the server rather than no zipped folders, load would even decrease, since a single file GET would suffice rather than in my case, 225 (for the files) + 34 (for the directory html pages).

[Anonymous]

Zipping (with no compression) 108 Mb in 225 in 34 folders on my PC took 12 seconds. I think this is not to bad.

This brings me to my initial comment on this page where I said...

If you are planning to share a complete folder, to someone who actually wants your complete folder, why don't you just zip it up first yourself then include it the virtual file system? Very easy and quick to do.

Do you think 12 seconds out of your life is to long to provide the service you apparently want to give your computer illiterate family or friends who want not only all the files in your folders but your HFS virtual file system page structure as well for some strange reason?  I don't think so.

Look, everybody has their own ideas of what they would like to see in HFS.  Don't forget that HFS is FREE.  Some are good ideas and have been implemented or are on the to-do list.  Other ideas simply aren't.  I think on-the-fly sever side zipping might be in that "other" category.

[Anonymous]

My, what a lively discussion. I can't seem to stop responding, even though this was initially intended as a minor feature request, with little priority to me. I just thought it would be neat.  But here goes, one last try to defend this idea:

Do you think 12 seconds out of your life is to long to provide the service you apparently want to give your computer illiterate family or friends who want not only all the files in your folders but your HFS virtual file system page structure as well for some strange reason? I don't think so.

This was merely to indicate how fast the archiving itself is as a rebuttal to the server load argument. This is timed without me finding my zipper, selecting the files, etc. But you are right, this does not cost that much time, even with my busy schedule   . However, this was not my point.

My point is:

... in many cases, one does not know which directory is wanted by the client.

Consider:

- My documents
-- My Music
--- Artist A
---- CD 1
---- CD 2
--- Artist B
---- CD 1
- My Images

It could very well be that this person either wants 1) all files 2) all music 3) music from one artist 4) a particular CD.

This means that, if you want to cover all options you would have to zip for each possible combination you could/should make a zipfile. For the tree above I count 7 possiblities, hence 7 zipfiles. This number will increase  exponentially with the number of branches in the directory tree.

Look, everybody has their own ideas of what they would like to see in HFS. Don't forget that HFS is FREE. Some are good ideas and have been implemented or are on the to-do list. Other ideas simply aren't. I think on-the-fly sever side zipping might be in that "other" category.

Absolutely! This is why I wrote...

Anyone can have his/her opinion about it and it is up to rejetto to see if he thinks it is worth implementing (or anybody else, for that matter).

I fully agree that if I want this badly enough, I should take the source code (not only is the software free, but even the source has been made available) and add the feature myself. Please recall that this discussion started as just a use-it-if-you-like-it feature request.

By the way, it seems that this option was already in Rejetto's todo list (although trailed by a questionmark), so it seems that a) I should read more thoroughly and b) perhaps it will not end up in the "other" category after all ;-)

[rejetto]

yes, it was already discussed, and i will work on this feature
what i cannot promise is the zip format
i probably will start with tar format, because it is much simpler
i had a look at zip format and found it has a loooong list of fields to be filled.
sad i can't use a library because i didn't found one able to archive on-the-fly like HFS needs to.

[maverick]

tar format?  Does anybody use that? I have been using computers for over 20 years and I don't ever remember seeing a tar archive.  Maybe it is more popular in other parts of the world.  What about the rar format?  Would the rar format be a hard one to do like zip?

Server side on-the-fly archiving.  Hmmm..... never thought about needing such a feature.  I guess my biggest concerns with this feature would be:
1. server load with mulitple users online at the same time browsing and downloading/uploading etc.
2. user abuse of current established "policy" by using this feature.  New "limits" would have to be established.

[rejetto]

i didn't consider rar
thought it was more complex than zip
but maybe it is worth a try

[MarkV]

Maybe even 7-Zip LZMA compression, it's LGPL software...

MarkV

[rejetto]

you didn't catch the point
we are talking about zero-compression archives
the real necessity is downloading the folder with single click
and compressing would likely take too much resources

[MarkV]

Then the term "compression" is misleading. You simply need a way to convert the folder into a data stream that can be downloaded as a whole.

MarkV