Messages

1

HFS ~ HTTP File Server / New Version 2.3m download message

« on: March 20, 2019, 02:30:16 PM »

This is what I get when attempting to download the new version.

High Security Alert!!
You are not permitted to download the file "hfs.exe" because it is infected with the virus "W32/Generic_PUA_NA.A".
URL: http://www.rejetto.com/hfs/hfs.exe
 File quarantined.

http://www.fortinet.com/ve?vn=W32%2FGeneric_PUA_NA.A
Client IP: 
Server IP: 185.20.49.7
User name:
Group name:

2

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: July 31, 2018, 06:11:09 PM »

Any progress on this topic? I am currently running with a rate limited wget command "wget -r -nH -l1 --no-verbose --no-parent --limit-rate=8000k" to avoid the hang. Using FileZilla I get much better performance without the hang but using HFS would be my preference.

3

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 30, 2017, 11:10:42 PM »

Boston Area
I tried w g e t FTP option with FileZilla. After some tweaking of FileZilla I was able to get some good performance but not as good as with HTTP in HFS, assuming the hang can be resolved.

4

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 26, 2017, 04:38:56 PM »

Using *.* at the end.
-r -nH -l1 --no-verbose --no-parent --directory-prefix=/abc --reject "index.html*" --accept rpm http://192.168.0.14/HP/deliverables/AddtlRpms/*.*

Connecting to 192.168.0.14:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-09-26 12:32:43 ERROR 404: Not Found.

Warning: wildcards not supported in HTTP.

I'll see if I can find the HTTP verses other mode options to see if there are any alternatives that might work.

5

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 26, 2017, 12:08:15 PM »

Yes, targeting a directory with about 33GB of data. This is why I am interested in solving the unlimited speed issues. Ultimately there will be as many as 20 servers running in parallel that will all want to get files from HFS.

6

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 23, 2017, 12:26:03 PM »

Adding:
"The error you started to post vs the error you have now"
The issue has not changed "using unlimited rate results in hang".
I have tried many things including changes to "w get" and changes to the HFS configuration.
I have been able to make it worse, but I have not yet been able to make it any better.

Here is my current theory:
Linux send "w get" request.
Transfers begin, but at some point PC/HFS is so busy it is not able to respond to a request.
Linux "w get" timeout after request is 900 seconds. This is the "hang".
Recently I found that if I reset HFS, Linux realizes the connection drop and reissues the request and transfers begin again.

I realize that this is something that can happen. I also realize that this is something that can be managed like any other data transport method "start - fail - retry - loop until done".
In this case I just think that the failure is statistically common instead of statistically rare. In a statistically rare case the defaults work because a failure should not occur. This is why the rate limit case works flawlessly. In a statistically common case the defaults no longer work (waiting 900 seconds is completely unrealistic). If I knew about all of the controls and how they work, for both "w get" and HFS, I might be able to find a combination that works. This is what I am looking for, some assistance in finding a combination of controls that will retry immediately "making the assumption that HFS missed the request and will never respond so the request needs to terminate and retry". I would be willing to share a remote session with anyone willing to provide this level of assistance.

Some history: I ran into this issue more than a year ago using an older version of HFS. I made many similar attempts to get it working and after every attempt failed I was OK with the rate limit option because the total transfer size was small enough. This issue has been experienced in many locations using different PCs (Windows 7 Pro) running different versions of HFS and using different Linux servers running different versions of RHEL (6.5, 6.9. 7.2, 7.3).

7

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 23, 2017, 11:38:40 AM »

Already tried "--retry-connrefused", did not work.
Already posted complete command "-r -nH -l1 --no-verbose --no-parent --directory-prefix=/abc --reject "index.html*" --accept rpm http://192.168.0.14/HP/deliverables/AddtlRpms/".
Wireshark was a waste of time because it too had trouble keeping up with the high data rate.
My best troubleshooting tool was to set verbose and watch the activity on the Linux side.
The closest I have been to a working setup was when I figured out that using "F4" to stop then restart HFS allowed HFS to restart and eventually finish (many "F4"s).
If I insert the rate limit option in the command line "--limit-rate=8000k", I have no issues at all except the entire transfer takes ~45 minutes.

8

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 20, 2017, 06:45:15 PM »

Getting closer but still no solution.
What I find is that the server sends a request, but HFS never answers. If I use F4 to stop and start HFS, the transfer continues. However it is not long before I get another failure. I suspect that the default connection timeout is much longer than I ever wanted to wait so I never realized what was happening. Now that I have had a chance to experiment I can make it worse but I cannot seem to make it much better.
I also suspect that because the unlimited rate is very high that the Windows OS contributes to the fact that HFS may not be getting every single request.
So the question is:
How do I set the get command parameters such that I am resending requests to HFS in a way that is not confused or locked up due to connections not being terminated and restarted as both the get command and HFS would understand?

I have followed a few examples I found using Google but as I said, I can make it worse but I can't seem to make it better.

9

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 20, 2017, 04:27:40 PM »

This appears to be where it all starts in the log file:
Read error (Connection reset by peer) in headers.

After which I get a long list of "connection refused" messages.
"-F" made no difference.

Still looking for answers.

10

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 20, 2017, 03:23:04 PM »

Wireshark results:
Wireshark hangs with "not responding".
HFS hangs with no message, connections remain open
Linux server hangs in the middle of a file, issuing a CRLF results in a new blank line as the job has not completed
The state appears to be indefinite, or as long as I was willing to wait
If I switch off HFS, the server terminates the running job but Wireshark remains stuck and had to be terminated with the task manager. There was some erratic and intermittent behavior where it appeared to be trying to display more packets but I
could not put up with it any longer.
w g e t options example:
-r -nH -l1 --no-verbose --no-parent --directory-prefix=/abc --reject "index.html*" --accept rpm http://192.168.0.14/HP/deliverables/AddtlRpms/

The issue is still open and unexplained.

11

HFS ~ HTTP File Server / Re: Unlimited Speed Hang

« on: September 15, 2017, 08:07:14 PM »

RHEL 6.5, 6.9, 7.3 versions "w g e t"
I will give Wireshark a try to see if I can capture the failure.
Thanks

12

HFS ~ HTTP File Server / Unlimited Speed Hang

« on: September 07, 2017, 01:26:52 PM »

Setting HFS to default unlimited speed results in either a disconnect or hang. Server uses w g e t command and if I rate limit w g e t there is no issue (w g e t ... --limit-rate=8000k ...). However if I do not rate limit the w g e t command or rate limit HFS the transfer quits without any indication on how or why. The point at which it stops is also not consistent.
When unlimited HFS graph reports speeds like 2873980 kbps and
Log reports speeds anywhere from 3MB/s to 68MB/s

Can anyone help suggest what the problem might be and/or how to debug what the failure mechanism is?

Windows 7 Pro 64bit, SP1
Intel Core i5-2400 3.10 GHz
Ethernet Server Adapter X520-2 #2