rejetto forum

Client Download problem

gDamich · 13 · 8701

0 Members and 1 Guest are viewing this topic.

Offline gDamich

  • Occasional poster
  • *
    • Posts: 5
    • View Profile
Hi,

I'm new HFS user. I have my own android client which is just downloading file from HFS which I hosted in MS Azure cloud. Now, I'm not sure if my problem is in client, azure or HFS.

Thing is that after first DL my file is "cached" somehow in client system. eg. if I'm using PC Chrome app for DL, first time file is normally downloaded, but every time after that when I try to  DL it again I cannot see HTTP req in HFS. So I suppose that Chrome cached the file. But similar thing happens on my android client. I programmatically download file but it's almost instantly with client recorded throughput of 2-3 kbps. Ofc. on HFS I cannot see that DL (HTTP GET).

Anyone with similar experience.

PS: If you wonder who wants to dl same file multiple time to same pc. That's me... Do not ask why! :-)

BR,
G


Offline bmartino1

  • Tireless poster
  • ****
    • Posts: 910
  • I'm only trying to help i mean no offense.
    • View Profile
    • My HFS Google Drive Shared Link
i'm not sure what azure{client, azure} is it your pc device(another client?)  or a download manger???...

If hfs is not recieveing the conection request it will not let you download the file, so check you prot forwarding and web code to make sure you can download the filesfrom pc multiple times...

 I have used my andriod phone to download files from my hfs and it would go to storage(sd or phoen memroy dpends on setting in phone at time...), i haven't seen chrome cahed downloded files, and chrome wouldn't cash a file you have to downlaod, it will cash the website to lad faster ie, web info suchas text/HTML coded pictures/ etc...

i use live template with my hfs as it wil throw pictures music and other files into a online palyer, but also give you the option to download the file... it could be that your webcode or template is causing issues. If possible explaing to me what azure is, what the function is for it...

as for hfs you will probally have to download it agin and put it into a new folder and ()with everything its defaults() add the file that is failing to downlaod and test it...

Files I have snagged and share can be found on my google drive:

https://drive.google.com/drive/folders/1qb4INX2pzsjmMT06YEIQk9Nv5jMu33tC?usp=sharing


Offline bmartino1

  • Tireless poster
  • ****
    • Posts: 910
  • I'm only trying to help i mean no offense.
    • View Profile
    • My HFS Google Drive Shared Link
upon some searhing i belive when you say azure taht you mean microsft new iss webserver website systems...
http://azure.microsoft.com/en-us/

in which case you might be using azure for data space and haveing premision errors with links to your files. If you need space i recomed google drive as it will auto sync to a folder and you get 15 GB free!

I'm still not 100% sold on the new microsft producsts and it could be a bug problem with it on there end, if i have assumed coretly that this is azure...

---so lets start with some basics-- ie waht is where and how do you want things to run.. as i see it from your post:
HFS is an windows excutable runign a http file server
(your phone)andriod is a linux client machine coneting to hfs
azure(if asumed coretly is a server ferature)- live account login to acess "skydrive" and use for data sharing... which you have generated links and gave permisoins...

--- so you have files on azure that you need webhosting though hfs
ie as your andriod phone hits hfs webpage and clicks a azure http link for the file it will downlaod or bring a certain service...
----------------------------

Your expering an error in which hfs goes to download the same link/file and it doesn't work

this is due to (azure and it shraing permsion/ url path may have chaged/time downlaod and/or authentication)
still don't know much about azure and as it is still fearly new (atleast to me) i belvie the error you are experienc is with azure!
to my understaning,
Files I have snagged and share can be found on my google drive:

https://drive.google.com/drive/folders/1qb4INX2pzsjmMT06YEIQk9Nv5jMu33tC?usp=sharing


Offline gDamich

  • Occasional poster
  • *
    • Posts: 5
    • View Profile
Hi,

thanks for answers. just for your information, azure is MS cloud. They have various cloud services, and I'm using virtual machine with Windows Server 2012 on it. I'm accessing machine using RDP. So thing is that I have same android app client which is just performing http get and measuring download throughput.

From server point view (azure vm on win server 2012) I settup everything correct, endpoints for my HFS port 8081, rules for win firewall on vm.
So,if anything is wrong on server, it's probably in HFS. Can be other, but I do not have idea what.

From client perspective, I testes URL (file as http resource) with fiddler, chrome, IE9 and with my android app which is just using:
                        URL url = new URL(resource);
         URLConnection connection = url.openConnection();

So, for resource like http://myserver:8081/files/5MB one android phone with my client every time successfully dl that file, but same android app on another device use same code and on same nw (cellular data / WiFi internet connection) find resource but file size (int fileSize = connection.getContentLength();) is very low, like 80 bytes, and it dl it immediately with recorded throughput 2 kbps. Usually thr is above 2000kbps.

About other clients like fiddler, I can see that if file is 75MB then fiddler dl it everytime and I can see that on server HFS console, but if I try to dl 30MB or 5 MB etc, it download it immiditaly (less than 2s). That makes no sense because I know that my PC connection is like 10-20 Mbps.

I suspect that there is possibility that there are some nw nodes in cellular operator nw or intranet/internet nw which are caching files or similiar.

BR;
G


Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile
try adding "?3458357893593759" (random data) to the URL you download


Offline gDamich

  • Occasional poster
  • *
    • Posts: 5
    • View Profile
Ok,

that's working... Can you please explain a little? So it's caching? On HFS side?

BR,
G


Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile

Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile
the URL trick is a known way to avoid cache systems


Offline gDamich

  • Occasional poster
  • *
    • Posts: 5
    • View Profile
It seems that if I tried again with ?3458357893593759 it's not detectable from server part... This is fiddler session prop:

HTTP GET: http://{myserver}:8081/files/dl/5MB.bin?3458357893593759

SESSION STATE: Done.
The request was forwarded to the gateway.
Response Entity Size: 5242880 bytes.

== FLAGS ==================
BitFlags: [RequestGeneratedByFiddler, SentToGateway] 0x880
X-EGRESSPORT: 50833
X-RESPONSEBODYTRANSFERLENGTH: 5242880
X-CLIENTPORT: 0
X-CLIENTIP:
X-BUILDER-MAXREDIR: 10
X-FROM-BUILDER: Parsed
X-HOSTIP: 172.17.64.140
X-PROCESSINFO: Fiddler:3152

== TIMING INFO ============
ClientConnected:   12:11:16.540
ClientBeginRequest:   12:11:16.540
GotRequestHeaders:   12:11:16.540
ClientDoneRequest:   12:11:16.540
Determine Gateway:   0ms
DNS Lookup:       0ms
TCP/IP Connect:   1ms
HTTPS Handshake:   0ms
ServerConnected:   12:11:16.544
FiddlerBeginRequest:   12:11:16.544
ServerGotRequest:   12:11:16.545
ServerBeginResponse:   12:11:16.589
GotResponseHeaders:   12:11:16.589
ServerDoneResponse:   12:11:17.058
ClientBeginResponse:   12:11:17.061
ClientDoneResponse:   12:11:17.061

   Overall Elapsed:   0:00:00.521

The response was buffered before delivery to the client.

== WININET CACHE INFO ============
This URL is not present in the WinINET cache. [Code: 2]
* Note: Data above shows WinINET's current cache state, not the state at the time of the request.
* Note: Data above shows WinINET's Medium Integrity (non-Protected Mode) cache only.



Offline gDamich

  • Occasional poster
  • *
    • Posts: 5
    • View Profile
To add info more... if I dl 30MB file with random data URL part multiple times it goes through, but for 5 MB file doesn't work. Both files are in same folder.

EDIT: just figure this: If I add just "?" mark to URL then 5MB file is downloading from server...
« Last Edit: May 07, 2014, 10:29:07 AM by gDamich »


Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile

Offline bmartino1

  • Tireless poster
  • ****
    • Posts: 910
  • I'm only trying to help i mean no offense.
    • View Profile
    • My HFS Google Drive Shared Link
Thank you for suppling the infor on azure and further explaining your issue:

1st,
double check the vitural machines seting for the network(is it nat/bridged/etc...) what is truley running the vm (microsft vitrual pc or azure cloud service???) as i now understand azure is web space in which you have a vitural machien runing in the cloud and hfs is on the vm in the cloud...

--- if you are 100% sure it is windwos that is cahing the file:
http://www.catonmat.net/blog/clear-privacy-ie-firefox-opera-chrome-safari/
(scripts for clints to clear cashe...)

---as for windows inet cahs:
http://blogs.technet.com/b/askperf/archive/2007/08/21/under-the-hood-wininet.aspx

other wise you may need to set a scheduled task on the server to clear the cash:
http://windowsitpro.com/scripting/clear-out-cached-data

--- or delte /disable the cashing feature (only do this if you belive that windinet cahs is the issue - I don't think it is, as rejeto posted..and you confirmed it working....

Link on wininet cash...
http://msdn.microsoft.com/en-us/library/windows/desktop/aa383928(v=vs.85).aspx

---(adding a ?and random group of ###) will signal that it is a different website conecitng each time. meaning that azure is not forwarding and has net blocking to make sure that acess to it is limited and /or the vm/network setting on the server itself is acepting conection once...!

i belive it to be the network structure on how hfs is acesing teh net to become website worthy. not os/program

Files I have snagged and share can be found on my google drive:

https://drive.google.com/drive/folders/1qb4INX2pzsjmMT06YEIQk9Nv5jMu33tC?usp=sharing


Offline bmartino1

  • Tireless poster
  • ****
    • Posts: 910
  • I'm only trying to help i mean no offense.
    • View Profile
    • My HFS Google Drive Shared Link
to further explain what rejeto was saying:

------------
try adding "?3458357893593759" (random data) to the URL you download
-----------

I have a website with a file located at thiss adress:

http:mysite:8081\file

i can clink that link to downlaod the file

I ca also do the same by adding this at the end

http:mysite:8081\file?01

in yourcase after you have issued ?01 the file is no longer working and thus the next avilbe working downlaod is

http:mysite:8081\file?02

this (vcan be and may be your issue)is due to:

client webrowser that cash the file systems:
http://wizards.custhelp.com/app/answers/detail/a_id/1920/~/how-to-bypass-your-browser-cache.

windows not clearing the dns cashe:
---
Windows (regardless of browser):

1. Click the Start menu, then select Programs > Accessories > Command Prompt.

2. In the window that opens, type: ipconfig /flushdns

3. Hit the Return key.

4. Close the program. Restart your computer, then try the web address again.
---
make it a batch and run it once/twice a day if it fixes your issue...

double check you azure networking infracture:
http://azure.microsoft.com/en-us/documentation/articles/virtual-networks-add-virtual-machine/

Hope this helps :) :(
Files I have snagged and share can be found on my google drive:

https://drive.google.com/drive/folders/1qb4INX2pzsjmMT06YEIQk9Nv5jMu33tC?usp=sharing