rejetto forum
Software => HFS ~ HTTP File Server => Topic started by: raybob on August 17, 2011, 01:43:21 AM
-
Can you create a macro command to directly serve different HTTP codes to the user, for example something like {.if| I want to give this user a deny-403 code |{:{.serve|403.}:}.}
Reason I ask is because in certain cases I want to direct a user to the deny page but using {.redirect|/~deny.} will confuse search engines and browsers into thinking that the page exists and is accessible but got temporarily moved.... This would also be useful for serving a 307 code since using {.redirect.} only constitutes a temporary redirect, not a permanent one.
Thanks!
-
At the head section of your template:
<meta name="robots" content="noindex,nofollow,noarchive,nosnippet">
you should get the desired effect.
-
No, I'm looking for a way to actually use a macro to serve HTTP status codes of my choosing in the response headers; for users not just search engines, and for a variety of applications.
-
Like {.if| I want to hide this page |{:{.http status|404.}:}.} or {.if| I want to deny this user |{:{.http status|403.}:}.} or {.if| I permanently moved this file |{:{.http status|301.}{.add header|Location: http://website.tld/.}:}.}
Something to that effect would be very useful.
This can very easily be done in PHP... <?php header("HTTP/1.1 404 Not Found"); ?>
-
yes i mean to add something like that.
that was already in the to-do list, in this line
+ macros missing to cache a folder: {.reply|content=|var=|code=|filename=|mime=.}
i meant it as a way to provide cached content, but it would work for your purpose too.
keep an eye on future builds for new commands.
unfortunately i didn't solve my problems stopping me from developing hfs yet. :(
-
Rather than creating a new topic, I'll just ask here....
Have you ever thought about creating a command that functions similar to javascript setTimeout() or PHP sleep()
-
mmm, nope.
But sleep is out of question because HFS is not multi-threaded.
Would you need a setTimeout ?
-
But sleep is out of question because HFS is not multi-threaded.
Is that something you plan on doing in the future is making HFS multithreaded? Perhaps one thread per connection would be effective :)
HFS frequently maxes out my CPU on its allotted core (and this is a Core i7 here) and then the GUI freezes up....
Generally if I have >8 active downloads HFS lags to the point that it's basically knocked offline because I can't even access it from within my LAN in under a minute. That's on a VM server with an i7 and 3GB RAM.
EDIT:
Also I ask about setTimeout or something like that because capturing the output of a batch file only works if the batch takes under a certain amount of time to complete, like HFS doesn't wait for it.
-
Is that something you plan on doing in the future is making HFS multithreaded?
that won't come until a full rewrite of HFS.
would i have the time, i would go for a multiplatform version based on Python.
Generally if I have >8 active downloads HFS lags to the point that it's basically knocked offline because I can't even access it from within my LAN in under a minute. That's on a VM server with an i7 and 3GB RAM.
wow. What's the total upload speed at that moment?
Also I ask about setTimeout or something like that because capturing the output of a batch file only works if the batch takes under a certain amount of time to complete, like HFS doesn't wait for it.
but you don't know how long will the batch take, right?
you may wait too long, or arrive too early.
it's better if the "fetcher" of the result is in a timed event, isn't it?
-
Bandalit moved to http://www.rejetto.com/forum/index.php/topic,9941.0.html
-
wow. What's the total upload speed at that moment?
My internet doesn't go above 120KB/s (which is slightly problematic). HFS doesn't lag based on total upload speed but rather number of concurrent connections. I can easily get over 8MB/s with no lag on my local LAN if no one else is connected.
That being said there is a total of ~200 macros that get executed under [request] according to the macro-log and 542 items in my VFS ;D
Do you think the fact that it's under a VM has much effect? Every other application seems to run the same speed...
BTW what I said before wasn't totally accurate.... here's what I measured: >4 downloads makes the GUI slower, >10 downloads makes the GUI unusable by taking >10 seconds to show anything, >12 downloads makes a page take longer than 10 seconds to load and >15-20 downloads will make Firefox time out when loading a page. HFS has only been knocked offline once for me (completely unresponsive in GUI and over HTTP) and it had around 30 (concurrent) downloads.
I also tried disabling browsing compression but it didn't seem to have an effect.
-
your [request] script is executed only at the beginning of the connection.
in case the gui is slow also when no connections are established, it must be something else.
a quick test could be done by disabling your scripts for a while: you can just rename the hfs.events file
-
I'll try that next time I get heavy traffic.
-
Disabling events actually didn't seem to have an effect.
Today during some heavy traffic I tried temporarily disabling options and I managed to hit a record of 8 HTTP requests/second (something like 1000 HTTP packets per second) with literally zero lag. It was pretty cool :D Restoring the options pretty much immediately locked up the interface.
The following things I tried all had no effect:
Disabling Apache Log
Disabling Compressed Browsing
Disabling preventing download accelerators
clearing ban list
turning off high speed handling
-
i didn't understand, what's the result you just told, was it something different? and how did you get it?
-
No, I'm saying that going with my options, nothing I change seems to have an effect, yet clicking temporarily reset options fixes it immediately.
-
ah ok.
So, it is an option OR the template.
try to backup your template then "html template > restore default"
-
Next time I'm getting traffic I'll try replacing the template. That very well could be the problem since it's over 300KB in size ;)
Though how could the template be the problem if HFS also lags downloading files, not just pages?
I'll let you know the results.
---------------------------------------------------------
OK, without traffic, I tested how long it took HFS to generate a page with a bunch of {.exec.} commands in it..... usually it would take ~770 milliseconds but as high as 2 seconds to generate it, with my standard template. Changing the template to a nearly blank one had no effect.
Next I'll test with heavy concurrent traffic.
---------------------------------------------------------
Two more things.... Do you think it has an effect that my VFS has 634 items in it? I feel like this is the most likely cause.
Anyway, here's my customized options just as another thought:
# 2.3 beta (build 279)
window-max=yes
# default: no
easy=no
# default: yes
port=80
# default:
files-box-ratio=0.1609375
# default: 0
log-max-lines=0
# default: 2000
log-file-name=C:\Documents and Settings\admin\Desktop\Logs\HFS Main Log.log
# default:
log-font-name=Arial
# default:
log-font-size=8
# default: 0
log-date=yes
# default: no
log-only-served=no
# default: yes
log-server-start=yes
# default: no
log-server-stop=yes
# default: no
log-replies=yes
# default: no
log-file-tabbed=yes
# default: no
log-apache-format=%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"
# default:
tpl-file=C:\Documents and Settings\admin\Desktop\Global Template.tpl
# default:
confirm-exit=yes
# default: no
keep-bak-updating=no
# default: yes
custom-ip=www.filesplat.com
# default:
search-better-ip=no
# default: yes
start-minimized=yes
# default: no
auto-save-vfs=yes
# default: no
use-comment-as-realm=no
# default: yes
getright-template=no
# default: yes
dont-include-port-in-url=yes
# default: no
delete-partial-uploads=yes
# default: no
rename-partial-uploads=%name%.partialupload
# default:
tray-icon-for-each-download=no
# default: yes
enable-fingerprints=no
# default: yes
send-hfs-identifier=no
# default: yes
list-protected-items=yes
# default: no
browse-localhost=no
# default: yes
default-sorting=ext
# default: name
auto-save-vfs-every=60
# default: 0
last-update-check=40575.7608544907
# default: 0
tray-message=Uptime: %uptime%\nDownloads: %downloads%\nTotal Hits: %hits%
# default: %ip%\nUptime: %uptime%\nDownloads: %downloads%
max-ips=100
# default: 0
max-connections=400
# default: 0
max-contemporary-dls=25
# default: 0
max-contemporary-dls-ip=5
# default: 0
flash-on=
# default: download
graph-rate=1
# default: 10
no-reply-ban=yes
# default: no
(long ban list removed for privacy)
add-to-folder=/
# default:
last-file-open=C:\Documents and Settings\admin\Desktop\Server File System.vfs
# default:
encode-spaces=no
# default: yes
mime-types=*.js|text/javascript|*.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.css|text/css|*.txt|text/plain|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.tpl|text/plain|*.swf|application/x-shockwave-flash
# default: *.htm;*.html|text/html|*.jpg;*.jpeg;*.jpe|image/jpeg|*.gif|image/gif|*.png|image/png|*.bmp|image/bmp|*.ico|image/x-icon|*.mpeg;*.mpg;*.mpe|video/mpeg|*.avi|video/x-msvideo|*.txt|text/plain|*.css|text/css|*.js|text/javascript
in-browser-if-mime=yes
# default: no
icon-masks=.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||.jpg|10001||.mp3|10001||.pdf|10002||
# default:
(address2name removed for privacy)
recent-files=C:\Documents and Settings\admin\Desktop\Server File System.vfs|C:\Documents and Settings\admin\Desktop\Server File System.vfs|C:\Documents and Settings\admin\Desktop\test.vfs
# default:
trusted-files=C:\Documents and Settings\admin\Desktop\Server File System.vfs|C:\Documents and Settings\admin\Desktop\Server File System.vfs
# default:
tray-instead-of-quit=yes
# default: no
compressed-browsing=no
# default: yes
hints4newcomers=no
# default: yes
log-toolbar-expanded=yes
# default: no
do-not-log-address=192.168.1.1-192.168.1.80
# default:
min-disk-space=10
# default: 0
out-total=229392697625
# default: 0
in-total=27657785166
# default: 0
hits-total=404436
# default: 0
downloads-total=85054
# default: 0
upload-total=1922
# default: 0
connections-columns=IP address;174|File;417|Status;196|Speed;72|Time left;77|Progress;980|
# default: IP address;206|File;206|Status;206|Speed;50|Time left;50|Progress;50|
update-daily=no
# default: yes
ip-services=http://www.whatismyip.org/|;http://www.melauto.it/public/rejetto/ip.php|;http://checkip.dyndns.org|:;http://www.canyouseeme.org|td><b>
# default:
ip-services-time=40662.6237840394
# default: 0
-------------------------------------------------------------------
EDIT
I took my VFS, cleared it down to 2 items, and my personal page which typically took 0.7-0.8 seconds to load now took around 0.6 seconds to load. Not really solved yet.
I'll have to test that with high traffic though.
-
i didn't see anything strange in your options.
it must be the tpl.
-
After some testing, I can conclusively say that it IS my VFS that is causing the lag. My VFS is 168KB in size and has ~750 items. After renaming the VFS and restarting HFS, it was able to respond to all requests almost instantly, and without a frozen GUI. The problem has been getting worse lately and I guess my large VFS explains it. This also explains why the problem doesn't occur in FHFS :)
At one point today I had 11 simultaneous downloads and it took ~5 minutes to generate a single user page with ~20 files. Also it seems as if HFS puts files in priority over pages (good for HFS' original purpose but not for me lol) as starting a file download was almost instant, while generating a page was extraordinarily slow.
ALSO I'm unable to test this with old versions because apparently there's something in my events script that doesn't work before 279 ( {.cookie.}? ) and so no URLs load at all if using 274 :(
Another thought: I was wondering why the VFS in FHFS has paths stored as clear text, while the VFS in my Filesplat server seems to be completely encoded.... it was started with build 274 (or maybe even it was 2.2f). Was it changed since then, and would I have a performance benefit if I somehow converted to the new version?
This has gotten so bad that with 8 downloads it takes several minutes to load a page with macros.
Two more things I thought of: 1) Would HFS would better outside a VM? 2) Would HFS run better with a professional or server version of windows?
And yet another find.... with high traffic loading pages with few macros (preferences page etc.) takes only a few seconds, but loading macro-heavy pages (like user files pages) can take up to several MINUTES.
LASTLY: If you want to test the site yourself Rejetto go to www.filesplat.com and see how long it takes to load the main page... see the source at the top for the %build-time% and then go to /~currentdls to see the number of downloads at that moment.
-
it's strange that 750 items are "too much".
consider that the VFS can also contain scripts in it. That may be the problem.
HFS gives no priority to downloads over pages. That must be an apparent side effect of your speed problem.
the reason for encoded VFS is that above 10KB the file gets compressed.
1) VMs always introduce an overhead, i don't know if it's appreciable in your case
2) i have no idea, but i don't expect that
the build time for me was 0.024s
-
Hmmm.... as a result of me upgrading to Server 2008 the loading speed for pages is literally 10-20x faster....
The front page typically takes .016 seconds now whereas a file page with 20 files, 2 folders, 2 hash calculations that used to take 12 seconds now takes 0.5-2 seconds to load.
I haven't had any serious load lately but when I do I'll let you know if it's any faster.
EDIT: My problem seems to be solved... while 4 concurrent downloads would mostly lock it up before now it causes pretty much zero increase in page loading time....
I'm thinking it might be due to increased IO performance.... VirtualBox seems to like VDI files much more than VMDK, that might be one of the causes.
-
fucking strange
-
yes i mean to add something like that.
that was already in the to-do list, in this line
+ macros missing to cache a folder: {.reply|content=|var=|code=|filename=|mime=.}
i meant it as a way to provide cached content, but it would work for your purpose too.
keep an eye on future builds for new commands.
unfortunately i didn't solve my problems stopping me from developing hfs yet. :(
Has this yet come to reality? :)
-
nope :)