rejetto forum

Software => HFS ~ HTTP File Server => Topic started by: Novox on November 02, 2008, 03:07:12 PM

Title: too many folder.tar
Post by: Novox on November 02, 2008, 03:07:12 PM
Hello rejetto,

I have some problem about downloading whole folder. When the user download the folder as .tar. The client always sent many http request to the server, so this make the server overload. I don't know this problem was happen only in this version or not, but now I'm using the latest build (212). I have attached a picture for you to see what the problem is.

(http://img145.imageshack.us/img145/3581/111op8.th.jpg) (http://img145.imageshack.us/my.php?image=111op8.jpg)(http://img145.imageshack.us/images/thpix.gif) (http://g.imageshack.us/thpix.php)
Title: Re: too many folder.tar
Post by: rejetto on November 02, 2008, 03:28:27 PM
no, it's not related to this build. it's not even related to hfs... :)
it's the client who has this way of doing.
are you sure the server is overloaded? cpu or hard disk? for how many seconds?
Title: Re: too many folder.tar
Post by: SteveW on November 02, 2008, 05:56:58 PM
download accelerator ?
Title: Re: too many folder.tar
Post by: rejetto on November 02, 2008, 07:44:37 PM
yes, with client i meant a download accelerator/download manager
Title: Re: too many folder.tar
Post by: MarkV on November 02, 2008, 09:06:54 PM
Download Accelerator attempts to download the file in multiple pieces.
Title: Re: too many folder.tar
Post by: rejetto on November 02, 2008, 11:16:36 PM
...and the option "prevent leeching" is supposed to prevent this!
Title: Re: too many folder.tar
Post by: chthonic on November 03, 2008, 12:46:08 AM
I use getright or freshdownload for this and I disable anti-leech.. and it works fine
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 02:39:00 AM
I need to give a user to use download accelerator but can I limit these connection? Because it take a lot around 256 connections. This make HFS sometime not responding.
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 02:41:15 AM
he CAN use a download accelerator.
what that option does is to force it to work with a single connection (instead of multiple).
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:22:29 AM
umm, he can use a multiple connection but nothing can limit this to 5 or something?
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 03:31:48 AM
yes, there's a method.
but what's your goal?
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:51:26 AM
All I want is to limit the "sending connection" just like in the first reply screenshot. Because It make a lot of connection. Now I can limit the download connection but for "sending connection" I can't.
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 10:59:36 AM
ok, you want to limit.
what prevent leeching does is to limit to "1" the number of connection on the same file.
all those connections will disappear.

Now I can limit the download connection but for "sending connection" I can't.

There's no difference in this case: if those "sending"s are issued by the download manager as you say, then HFS is considering them as "download"s.
This if you didn't mark them by using the "don't consider as download".
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 11:50:14 AM
Umm, but if "sending" is as same as "downloading" why it doesn't have any progress bar? Sorry for asking annoying question.  :D
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 11:55:30 AM
because HFS is still preparing the bytes to be sent
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 11:56:05 AM
And another one question, I have limit download connection by using event script but why the client still can make a connection?

Here is my event script came from http://www.rejetto.com/forum/index.php?topic=6263.0

[download]
{.if|{. {.current downloads by user.} > 1.}|{:
  {.disconnection reason|only one download at time.}
:}.}
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 12:01:40 PM
1. that script is even more restrictive than "prevent leeching".
prev.leech. limits 1 download connection per file. a user can open more connections but on different files.
this scripts limits to 1 download for the whole user.

2. where did you put that script?
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 12:11:25 PM
I have inserted the script on "Other Option" -> "Edit Event script". I have change the number from 1 to 5 and the result is like this screenshot.

(http://img145.imageshack.us/img145/3581/111op8.th.jpg) (http://img145.imageshack.us/my.php?image=111op8.jpg)(http://img145.imageshack.us/images/thpix.gif) (http://g.imageshack.us/thpix.php)
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 12:33:32 PM
I just made several tests, and it worked for me.
I put the script, asked to download: accepted.
asked to make another download: refused.
then i changed the limit to 2.
re-asked to download: accepted.
asked another one: refused.
asked another one with another account: accepted.

Other people used this script, and didn't report any problem. i don't know what else to do.

while you can't get this script working for you, you can always use the "prevent leeching" option.
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 12:59:47 PM
Umm, thanks for your reply. I have tested by myself (my user) tried to download both single file and folder.tar. Everything works fine. But the problem is happen when some user trying to download folder.tar. The "sending connection" will increase in sometime and then stop for a while then come back again and gone again. It is not happen everytime but it's randomly.

In my test I have download the file by using FlashGet with 8 thread. Then HFS can limit it to 5, so it works.

I think the problem is on that user maybe he use the unique download accelerator program.
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 01:03:05 PM
My tests were made with folder.tar too!
What you mean by "unique..." ?
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 01:19:48 PM
I mean he doesn't use a familiar download accelerator program.

I want to know what the program that he use. It always sending a connection. It make HFS works hard sometime it not responding. I have view his http header, it said "Mozilla..."....  :-[
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 01:33:10 PM
there's no way i know a program could skip HFS limitations.
so, i suppose this program doesn't do anything special, but you can try.
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 01:44:27 PM
Well, where are the mysterious connections came from? nobody knows right? T_T
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 02:05:10 PM
from?
the ip and the username are shown.
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 02:09:42 PM
Yeah, IP and User are known but... It's impossible to limit, so it is mysterious.  :-\
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 02:29:47 PM
try using his same download manager
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:00:20 PM
Because of I don't know what is his download manager  :'(. Here the connection comes again. For me I have used FlashGet, it doesn't has any additional connection like this.

(http://img388.imageshack.us/img388/4481/11111hs1.th.jpg) (http://img388.imageshack.us/my.php?image=11111hs1.jpg)(http://img388.imageshack.us/images/thpix.gif) (http://g.imageshack.us/thpix.php)
Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 03:16:11 PM
is your "Prevent leeching" option enabled?
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:26:20 PM
OK, this is my summarize.

If I turn Prevent leeching ON. User can't create more than one connection per file. And the mysterious connection is gone.

If I turn Prevent leeching OFF. User can create more than one connection per file. And the mysterious connection is come.

So, I decided to choose ON because to keep reliability on HFS service but the negative is some user can't get their maximum download speed. However, this problem I can received.

THANKS FOR YOUR KINDLY ALWAYS REPLY MY ANNOYING QUESTION. YOU'RE ALWAYS THE BEST REJETTO!!!  ;D
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:30:50 PM
Ops! rejetto please wait I have some new issue.
Title: Re: too many folder.tar
Post by: Novox on November 11, 2008, 03:45:42 PM
WOW! rejetto! I have FOUND where is the BUG came from!!

Here it is. I have already enable "Prevent Leeching" but that user can make more connection to the file.

Here is the first screenshot,


(http://img60.imageshack.us/img60/7826/2222xh6.th.jpg) (http://img60.imageshack.us/my.php?image=2222xh6.jpg)(http://img60.imageshack.us/images/thpix.gif) (http://g.imageshack.us/thpix.php)

In this first screenshot, I have downloading single file named "heroes" and the user download folder.tar named "DVDS4END"
For single file, can't create more connection but folder.tar can.

second screenshot,
(http://img124.imageshack.us/img124/2659/333ax5.th.jpg) (http://img124.imageshack.us/my.php?image=333ax5.jpg)(http://img124.imageshack.us/images/thpix.gif) (http://g.imageshack.us/thpix.php)

I have downding folder.tar named "Vampires" and the user download folder.tar named "DVDS4END"
For both me and that user can make a new connection. This is avoid "Prevent leeching" option.

Conclude that "Prevent leeching" is not working on downloading folder.tar

This is a bug or a program function?

Title: Re: too many folder.tar
Post by: rejetto on November 11, 2008, 08:33:00 PM
see if you enabled "ignore limits" in those accounts.

prevent leeching is known to be working on folder.tar
my tests were made all on folder.tar
Title: Re: too many folder.tar
Post by: Novox on November 12, 2008, 12:31:15 AM
No "ignore limits" in the in account was enabled. I have used right-click set user/pass.

If you can see in the above screenshot. If the ignore limits was enable why user tkbcolo can't create more connection on a single file?
Title: Re: too many folder.tar
Post by: Novox on November 13, 2008, 08:45:07 AM
Hi rejetto,

Today, I have change my HFS to build 214. The problem is still. The many mysterious connections always connecting. I don't know where they come from. I have enabled "Prevent Leeching" option. The users can make only one TRANSFER connection per one file but they can make SENDING connection a lot. Sometime, SENDING connection exceed to 256 and make no one can request to HFS service.

Thanks!!
Title: Re: too many folder.tar
Post by: rejetto on November 29, 2008, 12:34:57 AM
MAYBE build 215 could solve this problem.
can you test it?
Title: Re: too many folder.tar
Post by: Novox on November 30, 2008, 07:28:12 AM
Hi rejetto,

Doesn't test with build 215 but tested on build 216

The problem is still, download accelerator can create more than one connection when downloading as folder.tar

Picture attached.
Title: Re: too many folder.tar
Post by: chthonic on November 30, 2008, 04:31:16 PM
I have been watching this thread.. and ironically.. I think the issue might be simpler than expected.. I had a similar issue.... and the solution was really a no brainer


goto MENU then OTHER OPTIONS ... then check to see if PERSISTENT CONNECTIONS is checked... if not.. you will see the strange IDLE conections showing... when its checked.. the connection doesnt show up.
Title: Re: too many folder.tar
Post by: rejetto on November 30, 2008, 06:19:45 PM
persistent connections is affecting "idle" connections, indeed.
but his problem is not about idle connections.
Title: Re: too many folder.tar
Post by: chthonic on November 30, 2008, 06:43:36 PM
hmmm I can think of only one DA program that makes that kind of connection.. that's why I stopped using it!


I went and did a search.. and I cant find it listed anywhere now.... hmmmm
Title: Re: too many folder.tar
Post by: Novox on December 01, 2008, 12:41:59 PM
But in my problem all incoming connections are "SENDING" not "IDLE".
Title: Re: too many folder.tar
Post by: chthonic on December 01, 2008, 01:27:57 PM
yes that is .... odd