rejetto forum

Software => HFS ~ HTTP File Server => Beta => Topic started by: rejetto on June 08, 2020, 04:48:36 PM

Title: 2.4 template-making guide
Post by: rejetto on June 08, 2020, 04:48:36 PM
we should collect instructions on how to make a tpl for 2.4
let's see make a quick list
- the default tpl is always inherited. This means that unspecified/missing sections will be still taken from the default tpl. If you want to have some sections empty then you will have to have them as empty in your tpl.
- because of the point above you should probably have your [error-page]
- because of the point above, you don't need to copy libs/sections in yours, like the sha256.
- the 'private' section flag works no more. All sections are private unless you use the new 'public' flag.
- new templates (with new login system) will be recognized by the section [api level] section. At the moment the api level required is 2.
- because of the anti-DoS system, if you make XHR/fetch requests to folders, you may get an http error 429 that means you have to retry
- if your tpl works as a Single-Page-Application and doesn't use %list% in its main section, then use the 'no list' flag. That is, put this text at the top [|no list]

am i forgetting something?
Title: Re: 2.4 template-making guide
Post by: rejetto on June 10, 2020, 11:28:43 AM
in RC3 you will have some new features to make tpl and diff.tpl more compatible:

[section|ver=MASK|build=MIN-MAX|template=MASK]

ver and build apply to hfs versioning, while template will match over [template id] section. MASK supports ?*; operators as usual.

Of course HFS 2.3 will not support them, but if you do |ver=2.3 it will just ignore it and keep the section, while 2.4 will know it has to skip the section.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 10, 2020, 11:43:07 AM
to template makers,
with these new tools I think it would be better to restore the old 'selection' name instead of 'files', so that you can more easily make cross-version templates.
What you think?
I'm not totally sure that you can make a template that works both on 2.3 and 2.4 now, but maybe yes.
I may try to make a test with the default tpl itself.
Title: Re: 2.4 template-making guide
Post by: dj on June 10, 2020, 12:08:24 PM
What you think? it would be easier to restore the old 'selection' name.

My template (https://github.com/dj0001/hfs-template/releases/latest) works with both versions, but only if you add a diff.tpl.

This is because delphi checks, if a [unauthorized] section is in the user template.
If yes, the user template is rejected.

So you need to remove or rename it.

For compatibility reasons, it would be better, if
delphi checks, if a [unauth] section is in the user template and
if yes, the user template is allowed.

So you can write backwards compatible templates ([unauthorized=unauth]).
Title: Re: 2.4 template-making guide
Post by: rejetto on June 10, 2020, 01:21:00 PM
initially i thought that a tpl would have not been able to be compatible.

anyway, i was forgetting an important thing: 'files' and 'selection' are not the same. Selection has double %encoding because of past unicode problems (or bad design, not sure).
you may not notice any problem while you use just ascii for filenames.
Having no compatibility with 2.3 because of the login, I didn't have a reason to keep the double encoding. Even now I don't love the idea of keeping it, but not 100% sure. I may make extra tests to see if i find a better solution.

For compatibility reasons, it would be better, if delphi checks, if a [unauth] section is in the user template and if yes, the user template is allowed.

the reason for that was that you now always inherit the default tpl, so you automatically inherit the unauth even if you maybe don't write one because you want to keep the original one.
I need to think on this.
Title: Re: 2.4 template-making guide
Post by: danny on June 10, 2020, 10:10:26 PM
Can't delete.
All error pages redundant with extra scripts, extra jquery, extra css, extra footer.  Need a bypass for that problem.
I believe that the template's auto ban won't work; so, hfs.exe needs temp-auto-ban function added to the gui menu.
Login errors.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 10, 2020, 10:36:19 PM
All error pages redundant with extra scripts, extra jquery, extra css, extra footer.  Need a bypass for that problem.

ah, there is an unexpected behavior. A bug, i'd say.
I thought declaring an empty [error-page] would make the section empty, but it's still inheriting the default template.
Probably this was always the behavior with diff tpl but nobody noticed.
I think it should just be changed, hoping this is not breaking anything.
Consider it fixed in next release.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 10, 2020, 10:47:35 PM
I believe that the template's auto ban won't work;

you believe?
I would try first.
i think for improved security the event [unauthorized] should be used, because otherwise the tpl would catch only failures through the tpl's login form.
i want to introduce the ability to add events in separate file. Or maybe even in the tpl itself, why not, it would be cool and useful. I have to see what's easy to make. I'm putting this in my to-do.
Title: Re: 2.4 template-making guide
Post by: danny on June 10, 2020, 10:55:43 PM
you believe? I would try first.
  I tried it from /~login.  No auto-ban, because no [unauthorized] section in new templates.  Somehow, I never arrived at the /~unauth page.  It stayed at /~login.  The model-login did not have auto-ban. 

That function would be a lot better as temp-auto-ban in the hfs menu.
Perhaps the option for temp-auto-ban could be located at the bottom of the bans page? 
Title: Re: 2.4 template-making guide
Post by: NaitLee on June 11, 2020, 03:52:33 AM
ah, there is an unexpected behavior. A bug, i'd say.
I thought declaring an empty [error-page] would make the section empty, but it's still inheriting the default template.
Probably this was always the behavior with diff tpl but nobody noticed.
I think it should just be changed, hoping this is not breaking anything.

I once got the same problem around here (http://rejetto.com/forum/index.php?topic=13060.msg1065977#msg1065977), and there is a solution.
A single [error-page] is useless, but with just %content% below it works.

But, would this [error-page] slow down the template, or cause an slow-down attack possible/easy?
Title: Re: 2.4 template-making guide
Post by: LeoNeeson on June 11, 2020, 03:54:05 AM
I thought declaring an empty [error-page] would make the section empty, but it's still inheriting the default template.
Probably this was always the behavior with diff tpl but nobody noticed.
I noticed it some months ago (when I was doing the form-based login (http://rejetto.com/forum/index.php?topic=13054.0)), but I thought it was 'too much' to report it (I didn't wanted to bother you with such a small detail). I even thought it was not a mistake, but that it was done it on purpose to avoid having to write part of the 'head' section every time (so, I didn't report it). I can't remember other details now, but those were my 'breaking balls' small details that I was referring here (https://rejetto.com/forum/index.php?topic=13060.msg1066010#msg1066010). :D ;D (Now I do understand that I have to report if I've found something, because it could be an important detail).
Title: Re: 2.4 template-making guide
Post by: NaitLee on June 11, 2020, 03:58:22 AM
Can't delete.
In the del() function, replace the xhr.send(...) as:

Code: [Select]
xhr.send("action=delete&files=" + it);
Notice that "&files=", it's what "selections replaced by files" means.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 11, 2020, 09:04:13 AM
  I tried it from /~login.  No auto-ban, because no [unauthorized] section in new templates. 

it's beause it's an event, so it doesn't belong to the template.
i perfectly understand it's a complex system with few examples and it's hard to find the things you need.


Quote
That function would be a lot better as temp-auto-ban in the hfs menu.

that's what everybody think about the feature they want, but there's a reason why complex system are made with plugins.
This feature could make it to the GUI yes but not now, we have already too much in this, we need to stabilize.
I would not be eager if i was you, because when i'll make it you won't be able to customize it. What if you want to exclude an ip from the mechanism? The gui won't allow you that.

Quote
Perhaps the option for temp-auto-ban could be located at the bottom of the bans page? 

it's a good place i guess
Title: Re: 2.4 template-making guide
Post by: rejetto on June 11, 2020, 09:08:54 AM
I once got the same problem around here (http://rejetto.com/forum/index.php?topic=13060.msg1065977#msg1065977), and there is a solution.
A single [error-page] is useless, but with just %content% below it works.

that's a workaround, i've found there's a bug with it.

Quote
But, would this [error-page] slow down the template, or cause an slow-down attack possible/easy?
The slowing is very little.

Notice that "&files=", it's what "selections replaced by files" means.

i'm considering going back to 'selection' before official release. I have to make a test to understand if it's possible.
Title: Re: 2.4 template-making guide
Post by: danny on June 11, 2020, 09:51:24 AM
that's what everybody think about the feature they want, but there's a reason why complex system are made with plugins. This feature could make it to the GUI yes but not now, we have already too much in this, we need to stabilize. I would not be eager if i was you, because when i'll make it you won't be able to customize it. What if you want to exclude an ip from the mechanism? The gui won't allow you that. it's a good place i guess
Temporary auto ban wouldn't be such a problem.   5 fails = an hour wait.
Even if I couldn't customize it, I could wait the hour.  :)
In the del() function, replace the xhr.send(...) as:
Code: [Select]
xhr.send("action=delete&files=" + it);Notice that "&files=", it's what "selections replaced by files" means.
Thanks for that! 
Title: Re: 2.4 template-making guide
Post by: danny on June 11, 2020, 10:51:02 AM
Can't log in with Chrome browser
If it is tried at root, there's a message:  Login Required
If it is tried at a folder, there is a 404
No login prompt for Chrome
Except if you go to /~unauth then there is a basic-auth login prompt which doesn't log in. bug?

What is the correct way to reinstall the missing /~login functionality? 
Title: Re: 2.4 template-making guide
Post by: NaitLee on June 11, 2020, 12:51:12 PM
What is the correct way to reinstall the missing /~login functionality?

Now the password is dealt with sha256 on login for security, and templates should use in-page login (not browser popup).

dj's and my template are examples for in-page login. They includes how to sha256 a password and send login message.
As well as logout, now can be of use. ;)

My template repository on GitHub (https://github.com/NaitLee/Takeback-HFS-Template) have split parts of a whole template, "tkb.errorpages.tpl" also includes a full login system.
Title: Re: 2.4 template-making guide
Post by: danny on June 11, 2020, 02:45:41 PM
Is this a good way to make a replacement /~login section?

Edit: Here's the production example.
Code: [Select]
[login|public]
{.add header|Cache-Control: no-cache, max-age=0.}<!DOCTYPE html><html lang="en"><head><meta charset=UTF-8 /><meta name="viewport" content="width=device-width, initial-scale=1"><meta http-equiv=CACHE-CONTROL content=no-cache><title>HFS %version%</title></head>
<body bgcolor="black" text="white" alink="white" link="white" vlink="white">
<font size=4><center><br><br><br>
    <form method='post' onsubmit='return login()' >  <!-- return true   / -->
      <table border="0" cellspacing="20">
      <tr><td align='right'><label for="user">{.!Username.}</label><td><input name='user' size='25' required placeholder="%user%" id='user' />
      <tr ><td align='right'><label for="pw">{.!Password.}</label><td><input name='password' size='25' type='password' required id='pw' />
      <tr ><td><td><input type='submit' value='{.!Login.}' style='margin-top:13px'>
      </table>
    </form>
<font size="2">Keep me logged-in<input type="checkbox"></font><br><br></center>
<script>
const loc={}; loc.Logout='{.!Logout.}'  /*translate here*/
var sha256 = function(s) {return SHA256.hash(s)}
function logout() {fetch("/?mode=logout").then(res => location.reload()); return false;}
function login() {
    var sid = "{.cookie|HFS_SID_.}"  //getCookie('HFS_SID');
    if (!sid) return true;  //let the form act normally
    var usr = user.value;
    var pwd = pw.value;
var xhr = new XMLHttpRequest();
xhr.open("POST", "/?mode=login");  // /~login
var formData = new FormData();
formData.append("user",usr)
if (typeof SHA256 != 'undefined') formData.append("passwordSHA256",sha256(sha256(pwd).toLowerCase()+sid).toLowerCase()); else formData.append("password",pwd)
xhr.onload=function(){if(xhr.response=='ok') {
 if(document.querySelector("input[type=checkbox]").checked) localStorage.login=JSON.stringify([usr,pwd]); else localStorage.removeItem('login');
 location.replace(document.referrer)} else {alert("user or password don't match");document.querySelector("form").reset();location.reload(true)}}
xhr.send(formData)
    return false;
}
if(localStorage.login) document.querySelector("input[type=checkbox]").checked=true  //stop keep logged-in: call /~login and disable "Keep me logged-in"
document.querySelector("input[type=checkbox]").onchange=function(){if(!this.checked) localStorage.removeItem('login')}
if('%user%') {document.querySelector("input[type=submit]").value=loc.Logout; document.querySelector("input[type=submit]").onclick=function(){logout(); return false}; document.querySelector('button').hidden=false}
if(!'%user%' && localStorage.login) {
var tmp=JSON.parse(localStorage.login)
user.value=tmp[0]
pw.value=tmp[1]
var myform=document.querySelector("form"); if (myform.requestSubmit) myform.requestSubmit(); else myForm.submit()
} </script> <script src="/~sha256.js" onerror="setTimeout(()=> this.src='/~sha256.js',200)"></script>
It is good.  Many thanks to DJ for his login code. (https://rejetto.com/forum/index.php?topic=13060.msg1065861#msg1065861)

Edit2:  There was an extra step to prevent /~signin.  I had to put a link <a href="/~login"></a> first, as bait/bypass.  Only the first login link on the page gets changed to /~signin.  So, I made that one not-clickable.  Then I put the real login link after it.   Signin thing happened because of a diff.tpl, not HFS.

Edit3:  I installed a page refresh on bad password, so the error doesn't persist.



Title: Re: 2.4 template-making guide
Post by: NaitLee on June 11, 2020, 04:09:21 PM
Is this a good way to make a replacement /~login section?

Sure! Great job!
Also consider:
  Make the login page Throwback styled
  Logout, it's possible now
  Change password, it's mostly easy
It's already enough if you think Throwback should be lightweight & easy ;)
Title: Re: 2.4 template-making guide
Post by: danny on June 11, 2020, 04:31:30 PM
Thanks. 

I couldn't get the Thumbnails-OnDemand feature to work because this type of section is not supported: 
[file.jpg = file.jpeg = file.png = file.gif = file.tif = file.bmp = file.webp]

Title: Re: 2.4 template-making guide
Post by: rejetto on June 11, 2020, 09:35:32 PM
I couldn't get the Thumbnails-OnDemand feature to work because this type of section is not supported: 
[file.jpg = file.jpeg = file.png = file.gif = file.tif = file.bmp = file.webp]

that's another bug. Fixed in next release :)
Title: Re: 2.4 template-making guide
Post by: danny on June 12, 2020, 06:59:52 PM
Thanks!  Currently, it was possible to publish ThrowbackMP (all except thumbnails), and it is here. (https://rejetto.com/forum/index.php?topic=12055.msg1062733#msg1062733)

Documentation: 
update the delete (https://rejetto.com/forum/index.php?topic=13326.msg1066044#msg1066044) script
edit: or now revert the change

new location for [unauthorized] section macros is now the events file (https://rejetto.com/forum/index.php?topic=13334.msg1066130#msg1066130). 

[error-page]
%content%

[unauth]
{.redirect|/~login.}  Because /~unauth will bring up a basic-auth login box, which doesn't work.  So, it is good to redirect to a working login. (https://rejetto.com/forum/index.php?topic=13326.msg1066060#msg1066060)

[api level]
2

[mysection|public]
Add |public to sections that you link to.

Title: Re: 2.4 template-making guide
Post by: danny on June 12, 2020, 09:10:19 PM
https://github.com/rejetto/hfs2/releases/download/v2.4-rc03/hfs.exe
The file and file=type sections are working nicely. 
It is now possible to do thumbnails-ondemand, except that the script needs changed to add some small time-delay.
Code: [Select]
<script>document.addEventListener("DOMContentLoaded", function() {var lazyloadImages; if ("IntersectionObserver" in window) { lazyloadImages = document.querySelectorAll(".lazy"); var imageObserver = new IntersectionObserver(function(entries, observer) { entries.forEach(function(entry) { if (entry.isIntersecting) setTimeout(()=> { var image = entry.target; image.src = image.dataset.src; image.classList.remove("lazy"); imageObserver.unobserve(image); },500) }); }); lazyloadImages.forEach(function(image) { imageObserver.observe(image); }); } else {alert('Your browser needs an update');}});</script>
Is there a macro that can be used to shut off extras in case the server is too busy? 

This link has timed retry to prevent blanks: <a href="%item-url%"><img class="lazy" id="imgthumbs" data-src="%item-url%" alt="" onerror="setTimeout(()=> this.src='%item-url%',500)">%item-name%</a>
Title: Re: 2.4 template-making guide
Post by: NaitLee on June 13, 2020, 05:17:18 AM
How do we make it load only 4 photos at a time?  Or, a small time-delay between each?

Is there a macro or symbol that can be used to shut off extras in case the server is too busy? 

I used a setInterval() in Takeback to load a few images (append an img element) at a time.
Currently when server is busy, it returns a status code 429.
We can make an XHR request everytime an image-load period ends, see its status, then determine to continue or pause.
Title: Re: 2.4 template-making guide
Post by: danny on June 13, 2020, 09:21:02 AM
For supporting single-thread server, I found out that it is best to load images one at a time (all of one before starting another). 
Would img onload or object.onload be useful for this?

I wonder how to do one at a time + lazyload? 
Title: Re: 2.4 template-making guide
Post by: rejetto on June 13, 2020, 10:44:12 PM
I used a setInterval() in Takeback to load a few images (append an img element) at a time.
Currently when server is busy, it returns a status code 429.

How come this 429?
You should not  get a 429 for an image, unless you set some limits. 2.4 limits only folders. Right?

Guys, sorry but some changes are coming.
Check here https://rejetto.com/forum/index.php?topic=13060.325
Title: Re: 2.4 template-making guide
Post by: danny on June 13, 2020, 11:47:04 PM
With image thumbnails, it is necessary to limit the number of images loaded concurrently. 
One at a time is really best for single-thread server.
Also, many at a time won't work on dsl/3g, but few/one at a time is lively. 

Instead of vertical lazyload list with thumbnails auto-sized so that only 4-ish fit on screen, I suggest an update (which I can't figure out) to load one at a time so that we would have:
lowest cpu load
fast-action on slow connections
no more reliance on connections limiter
ability to do a tiles/gallery layout without server overload.

Title: Re: 2.4 template-making guide
Post by: rejetto on June 14, 2020, 08:06:45 AM
i'm just worried about the server giving 429 only for folders and not other cases, because it would be a bug.
I cannot tell exactly the case from what naitlee said.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 14, 2020, 09:21:23 AM
i just updated the first post for RC4.
Still, there is an important thing that would be good to address before 2.4 goes final.
At the moment sections are always directly requestable unless the 'private' flag is set on each of them.
I think this is not a good idea and we should invert the logic, because
- most sections don't need to be accessible via URL requests
- people can forget about declaring a section as private
- disclosing unnecessary information about your server is not good practice

This mean all the sections that needs it should set a 'public' flag (as opposite to the 'private'), and the 'private' would just be ignored.
I think this would still let you make a tpl that's backward compatible, right?

I know it's tiresome to have all these changes going on, but I think it's worth the price.
Comments?
Title: Re: 2.4 template-making guide
Post by: Mars on June 14, 2020, 09:46:59 AM
wouldn't it be simpler to name the accessible sections by url by preceding their name with ~( except for [] ) as [~login]

this would no longer pose a problem of interpretation and unnecessary option writing

the rest of the other sections could have their name coded to access them  url or form requests
Title: Re: 2.4 template-making guide
Post by: rejetto on June 14, 2020, 10:46:16 AM
wouldn't it be simpler to name the accessible sections by url by preceding their name with ~( except for [] ) as [~login]
this would no longer pose a problem of interpretation and unnecessary option writing

an interesting suggestion, but would mean to introduce a new syntax, while the section flags already exist, and for this kind of purpose
Title: Re: 2.4 template-making guide
Post by: danny on June 14, 2020, 05:36:11 PM
i'm just worried about the server giving 429 only for folders and not other cases, because it would be a bug.
I wouldn't worry about it except for this:
<script src="/~sha256.js">
should be changed to
<script src="/~sha256.js" onerror="this.src='/~sha256.js'">

No matter which limit, connections/429/downloads... the important externals need onerror added.
Problem:  onerror retries too fast.  Is there a way to slow down the retries by 200ms each?

...At the moment sections are always directly requestable unless the 'private' flag is set on each of them.  I think this is not a good idea and we should invert the logic......This mean all the sections that needs it should set a 'public' flag (as opposite to the 'private'), and the 'private' would just be ignored....
I don't mind adding |public to sections that are linked-to. 

Have linked to these and cannot add public to them:
~folder.tar
/~sha256.js
%encoded-folder%
%list%

Here is a template with the |public section changes added (attached)
You could test with it
Title: Re: 2.4 template-making guide
Post by: rejetto on June 14, 2020, 08:06:32 PM
I wouldn't worry about it except for this:
<script src="/~sha256.js">
should be changed to
<script src="/~sha256.js" onerror="this.src='/~sha256.js'">
No matter which limit, connections/429/downloads... the important externals need onerror added.

i can tell you that a request like that won't be affected by the anti-dos. Just double checked.
I know the problems that can produced by some user configured limits. I'm not currently trying to avoid them, never did in 15 years.
Maybe in the future, but the onerror would work only for a second attempt, not the best that can be done (using js).

Quote
Problem:  onerror retries too fast.  Is there a way to slow down the retries by 200ms each?

try this
onerror="setTimeout(()=> this.src='/~sha256.js', 200)"
Title: Re: 2.4 template-making guide
Post by: danny on June 14, 2020, 10:15:48 PM
onerror="setTimeout(()=> this.src='/~sha256.js', 200)"
Thanks!!!
That really helped. 
Now the retries don't jam the server.   
I used 500 for images retry on my photo version. 
Title: Re: 2.4 template-making guide
Post by: LeoNeeson on June 15, 2020, 12:32:43 AM
This mean all the sections that needs it should set a 'public' flag (as opposite to the 'private'), and the 'private' would just be ignored. [...] Comments?
Since v2.4 won't be compatible with 2.3 templates, I think the idea is good.

wouldn't it be simpler to name the accessible sections by url by preceding their name with ~( except for [] ) as [~login]. this would no longer pose a problem of interpretation and unnecessary option writing
I also like Mars's comment, of doing a "white-list" of allowed sections (with the name of the 'most common' and 'most used' sections). Sections on the white-list doesn't need to have a 'public' flag (the could have it, but it would not be mandatory). And all the sections that are NOT in the white-list would be automatically 'private'. So, the logic would be: if a section doesn't have a 'public' flag, and is neither on the 'white-list', then that section is private.

Since we talk about changes, how about changing the way to call a section (and file sections) "~" to "?" (since that's the most common way of doing it). For example: "/?login" (instead of "/~login"), or "/?lib.js" instead of "/~lib.js". This would bring consistency with the current: "/?mode=jquery". If I'm not mistaken, this idea was on the to-do list. To keep compatibility, you could make both ("~" and "?") as valid options.

Cheers,
Leo.-
Title: Re: 2.4 template-making guide
Post by: rejetto on June 15, 2020, 08:35:40 AM
I also like Mars's comment, of doing a "white-list" of allowed sections

i don't think mars suggested that, so i will consider it your suggestion ;D

Quote
if a section doesn't have a 'public' flag, and is neither on the 'white-list', then that section is private.

i see, but i don't think we need two methods on this

Quote
"~" to "?" (since that's the most common way of doing it).

the ? is a standard, yes, and we have it the standard way, with ?mode=section&id=
having ?whatever is not the common way of using the ? and is a very dangerous choice, because sections can have any name, and you can pass other parameters a lot of confusion can arise. If still have ~ to stay short.
Title: Re: 2.4 template-making guide
Post by: danny on June 16, 2020, 04:33:16 AM
Sometimes we would link to items which are located inside of the .exe.  I can't edit to add public to those. 
So, if they should be linked to; then, that would be a good use for a white-list.
Although, an actual white-list function is not really required for that.
Title: Re: 2.4 template-making guide
Post by: LeoNeeson on June 16, 2020, 07:26:55 AM
i don't think mars suggested that, so i will consider it your suggestion ;D
I must have misunderstood his message then (or he inspired me that idea). :P ;D

My white-list idea is having on the template a place where we declare all the public sections at once. For example, something like this (see the line marked in red color):

Quote
[+special:strings]
public.sections=[common-head];[];[style.css];[upload];
option.newfolder=1
option.move=1
option.comment=1
option.rename=1

This way, it's much easier to visualize all the public sections at once (and it's less work when adapting the old templates, since it's just adding one line of text). This is only an idea (it's currently not implemented). What the rest of you think about it?...

Sometimes we would link to items which are located inside of the .exe. I can't edit to add public to those.
I guess if Rejetto implements this, it would not be required to declare 'public' a link like "/?mode=jquery".

Cheers,
Leo.-
Title: Re: 2.4 template-making guide
Post by: danny on June 16, 2020, 08:27:13 PM
Here is a bad idea with gallery view, but it might be useful for testing, RC4

The template attached can appear to work for localhost/lan, but probably not do for multiple users with internet connections.

To make it more informative for testing, you can find onerror="this.src='%item-url%'" and change it to onerror add to log file. 
Then you can point it at a folder with hundreds of images (or search *.jpg), scroll, and see what the limiting factors may be. 

This file is not suitable for regular use.
Title: Re: 2.4 template-making guide
Post by: danny on June 17, 2020, 02:51:15 PM
RC5 (https://github.com/rejetto/hfs2/releases/tag/v2.4-rc05) is ready.

For this, add |public to sections that you link to. 
<a href="/~mysection">mysection</a>
goes with [mysection|public]
Title: Re: 2.4 template-making guide
Post by: Mars on June 17, 2020, 05:49:27 PM
some sections do not need to be with public attribute because they are already included by a macro call in a public section ~style.css, ~lib.js

[icons.css|public|no log|cache]  --> {.$icons.css.}
[normalize.css|public|no log|cache]  --> {.$normalize.css.}
[sha256.js|public]  --> {.$sha256.js.}

another thing needs probably to be extended

public sections are called mainly with the reference / ~ sectionname, however in a diff template this type of section may have to be completely redefined.
Due to the inheritance principle this section is available from anywhere in the vfs using the macro {.section | sectionname.} But should also be available by adding ~ sectionname to the url of any folder element vfs as example http://127.0.0.1/New%20folder%20(2)/~test




Title: Re: 2.4 template-making guide
Post by: rejetto on June 17, 2020, 08:16:35 PM
some sections do not need to be with public attribute because they are already included by a macro call in a public section ~style.css, ~lib.js

you are right, but there's a reason for it: i marked them public in case other templates want to use them by url

Quote
But should also be available by adding ~ sectionname to the url of any folder element vfs as example http://127.0.0.1/New%20folder%20(2)/~test

~ is a valid character for files, so introducing this feature was a debatable decision, i prefer to not extend it to folders.
it is also not necessary because you can use use ?mode=section with folders.
Title: Re: 2.4 template-making guide
Post by: Mars on June 18, 2020, 12:39:28 AM
you are right, but there's a reason for it: i marked them public in case other templates want to use them by url

~ is a valid character for files, so introducing this feature was a debatable decision, i prefer to not extend it to folders.
it is also not necessary because you can use use ?mode=section with folders.

add attached file ( which contain "this is the file" ) to root of vfs

add this section in the tpl

[test|public]
this is a section

go to this url
127.0.0.1/~test 

pay attention to the result: a recording is proposed, record in any directory and then examine the content

the file has priority over the section  whatever the chosen location.

among others:
"because we must  use use ?mode=section with folders", the name must appear in the url like all names starting with ajax.
but  all ajax sections can be made non-public for the benefit of a single section by using some changes

[ajax|public|no log]
{.check session.}
{.$ajax.{.postvar|method.}.}


[ajax.mkdir|public|no log]

[ajax.rename|public|no log]

[ajax.move|public|no log]

[ajax.comment|public|no log]

[ajax.changepwd|public|no log]

function ajax(method, data, cb) {
    if (!data)
        data = {};
    data.method = method;   
    data.token = HFS.sid; // avoid CSRF attacks
    showLoading()
    // calling this section 'under' the current folder will affect permissions commands like {.get|can delete.}
   return $.post("?mode=section&id=ajax", data).then(function(){
        if (cb)
            showLoading(false)
        ;(cb||getStdAjaxCB()).apply(this,arguments)
    }, ajaxError);
}//ajax




also added an example of a new method for changing the password, including the need for the old password and confirmation of the new one to validate the action, naturally the old password is encrypted when sent and compared by hfs to validate the new one.
it is not possible to use complex encryption for the new one but it does not appear in plain text when sent

the big advantage is that it avoids accidentally changing the current password
moving the mouse over the eyes to the right of the input fields allows you to view their content
Title: Re: 2.4 template-making guide
Post by: rejetto on June 18, 2020, 09:48:06 PM
i tried your tpl, but i see no "this is the file" to be added to the root. I only see the tpl attached to your post.
Anyway, i tried with another file, renamed it ~test and accessed it. The file has priority over the section, it was downloaded, the content was that of the file.
Everything is fine, that's the expected behavior.

The single [ajax] section is not bad, i may use it for the default tpl.

Consider that asking twice the password makes the "showText" unnecessary.

You should not use {.cookie|HFS_SID_.} in a cached section. If there's a logout/login in the meantime it will still have the old sid.  That's why i use "HFS.sid".

Using the base64 just to avoid the plain text is good, i had the same suggestion from LeoNesson.

Title: Re: 2.4 template-making guide
Post by: danny on June 22, 2020, 07:55:02 AM
How can I change the login section to avoid the "white screen" (bare server response) (https://rejetto.com/forum/index.php?topic=13060.msg1066148#msg1066148) problem? 

The login section that I have been using, works right most of the time, but sometimes, it doesn't catch the server response (sometimes the user gets white screen error).
Code: [Select]
[login|public]
{.add header|Cache-Control: no-cache, max-age=0.}<!DOCTYPE html><html lang="en"><head><meta charset=UTF-8 /><meta name="viewport" content="width=device-width, initial-scale=1"><meta http-equiv=CACHE-CONTROL content=no-cache><title>HFS %version%</title></head>
<body bgcolor="black" text="white" alink="white" link="white" vlink="white">
<font size=4><center><br><br><br>
    <form method='post' onsubmit='return login()' action="/?mode=login">  <!-- return true   / -->
      <table border="0" cellspacing="20">
      <tr><td align='right'><label for="user">{.!Username.}</label><td><input name='user' size='25' required placeholder="%user%" id='user' />
      <tr ><td align='right'><label for="pw">{.!Password.}</label><td><input name='password' size='25' type='password' required id='pw' />
      <tr ><td><td><input type='submit' value='{.!Login.}' style='margin-top:13px'>
      </table>
    </form>
<font size="2">Keep me logged-in<input type="checkbox"></font><br><br>
<button onclick="var tmp=prompt('new password'); if(tmp) {var fd=new FormData();fd.append('new',tmp);fd.append('token','{.cookie|HFS_SID_.}');fetch('/~ajax.changepwd',{method:'POST',body:fd})}" hidden>{.!Change password.}</button>
<br><br>
</center>
<script>
const loc={}; loc.Logout='{.!Logout.}'  /*translate here*/
var sha256 = function(s) {return SHA256.hash(s)}
function logout() {fetch("/?mode=logout").then(res => location.reload()); return false;}
function login() {
    var sid = "{.cookie|HFS_SID_.}"  //getCookie('HFS_SID');
    if (!sid) return true;  //let the form act normally
    var usr = user.value;
    var pwd = pw.value;
var xhr = new XMLHttpRequest();
xhr.open("POST", "/?mode=login");  // /~login
var formData = new FormData();
formData.append("user",usr)
if (typeof SHA256 != 'undefined') formData.append("passwordSHA256",sha256(sha256(pwd).toLowerCase()+sid).toLowerCase()); else formData.append("password",pwd)
xhr.onload=function(){if(xhr.response=='ok') {
 if(document.querySelector("input[type=checkbox]").checked) localStorage.login=JSON.stringify([usr,pwd]); else localStorage.removeItem('login');
 location.replace(document.referrer)} else {alert("user or password don't match");document.querySelector("form").reset();location.reload(true)}}
xhr.send(formData)
    return false;
}
if(localStorage.login) document.querySelector("input[type=checkbox]").checked=true  //stop keep logged-in: call /~login and disable "Keep me logged-in"
document.querySelector("input[type=checkbox]").onchange=function(){if(!this.checked) localStorage.removeItem('login')}
if('%user%') {document.querySelector("input[type=submit]").value=loc.Logout; document.querySelector("input[type=submit]").onclick=function(){logout(); return false}; document.querySelector('button').hidden=false}
if(!'%user%' && localStorage.login) {
var tmp=JSON.parse(localStorage.login)
user.value=tmp[0]
pw.value=tmp[1]
var myform=document.querySelector("form"); if (myform.requestSubmit) myform.requestSubmit(); else myForm.submit()
} </script> <script src="/~sha256.js" onerror="setTimeout(()=> this.src='/~sha256.js',509)"></script>

[ajax.changepwd|public|no log]
{.check session.}
{.break|if={.not|{.can change pwd.}.} |result={.!Forbidden.} (0).}
{.if|{.length|{.set account||password={.postvar|new.}.}/length.}|{.!OK.} (1)|{.!Failed.} (2).}

How do I correct the white screen bug? 
It is more frequent on mobile. 

P.S.  This login section is DJ's from  https://rejetto.com/forum/index.php?topic=13060.msg1065861#msg1065861
I have made 3 changes:  I copied the change password script from release 2, I copied the ajax section from takeback, and I added page reload on bad password so that bad password can't persist.  Anyhow, the one big problem is that sometimes a successful login is met with this peculiar form of disaster: 
(https://rejetto.com/forum/index.php?action=dlattach;topic=13060.0;attach=10126;image)
Looks blank, but actually says ok in dust-speck-sized print.  It is more likely to happen to the phone than the PC.
It doesn't happen so often, but I need to remove all chances of this happening. 
What is the recommended fix?
This looks exactly like blacklist (anything else) versus whitelist (nothing else) logic error.  It is currently operating like blacklist, which always needs fallback, such as timer (range 1.2s to 3s) then redirect ../

Edit, solution was to delete
action="/?mode=login
Title: Re: 2.4 template-making guide
Post by: dj on June 22, 2020, 08:41:19 AM
@danny
I can't reproduce your problem.

I found "username not found" neither in your template nor in the default template.
So perhaps it can be a problem with the login section.
Try to rename the 'login' section to 'signin' and also the links.
Try to test 'unauth' without redirect.
Title: Re: 2.4 template-making guide
Post by: danny on June 22, 2020, 09:34:13 AM
@danny
I can't reproduce your problem.

I found "username not found" neither in your template nor in the default template.
So perhaps it can be a problem with the login section.
Try to rename the 'login' section to 'signin' and also the links.
Try to test 'unauth' without redirect.
Thanks!!!
And, yes, there is an intermittent/rare problem with that.  The 'good problem' is that it mostly works.  The function is more likely to fail with an older android phone (although an updated pc can do that sometimes/rarely).  The problem is least likely to happen if you have menu/users/specify directory redirect after login (still possible--less frequently).  So, there is a testing hardship with both vfs options variety and browser variety. 
Trouble is:  The result of login is not confined to useable output (some of the output is not clickable) (https://rejetto.com/forum/index.php?topic=13060.msg1066148#msg1066148) and may appear broken.
Goal is: Confine the outcome to only what is clickable/usable (nothing else).

Seems useful to add a fallback-redirect so the responses/error doesn't linger on-screen?
Title: Re: 2.4 template-making guide
Post by: NaitLee on June 22, 2020, 10:40:21 AM
How can I change the login section to avoid the "white screen" (bare server response) (https://rejetto.com/forum/index.php?topic=13060.msg1066148#msg1066148) problem? 

dj's code instantly sent ("submit") changepwd data to server side, parsed by the macro from takeback.

In your code the form instantly submitted password to server, and instantly got the changepwd result, which is simply a plain string "OK (1)".
Seems page XHR response and page refresh happens at same time, that's may why your problem appears with a chance.
Additionally, dj's code should get a responce "ok" (not "OK (1)"), then recognizes changepwd success, otherwise fails with only "user or password don't match".

In takeback I did not use a form, (this is not standard, I'm going to correct it with a form and override its submit() function, just like HFS default tpl)
 by clicking "Okay" the password data is sent to server via XHR, returning that "OK (1)", and dealt by an alert().

You can have a look of my code, then adapt it to password form's submit():

Code: [Select]
function changePwd(newpass) {
        var xhr = new XMLHttpRequest();
        xhr.open('POST', '?mode=section&id=ajax.changepwd');
        xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
        xhr.onreadystatechange = function() {
        if (xhr.readyState === 4 && xhr.status === 200) {
            console.log(xhr.responseText);
            var code = ( xhr.responseText.split('(')[1] == undefined ? -1 : xhr.responseText.split('(')[1].split(')')[0] );
            if (code == "1") {
                alert('{.!Complete! Use your new password next time!.}');
                beforeRedirect();
            } else {
                if (code == "0") {
                    alert("{.!You cannot change your password!.}");
                } else if (code == "3") {
                    alert("{.!Failed: Old password you input is wrong!.}");
                } else if (code == "4") {
                    alert("{.!Macro is detected in your input. Please do not attack..}");
                } else if (xhr.responseText.trim() == "bad session") {
                    alert("{.!Bad session. Try to refresh the page..}");
                } else {
                    alert('{.!Unknown error.}: \n'+xhr.responseText.trim());
                }
            }
            }
        };
        xhr.send("token={.cookie|HFS_SID_.}" + "&old=" + sha256(oldpwd.value) + "&new="+btoa(unescape(encodeURIComponent(newpass))));
    }

Or adapt dj's changepwd ajax.
Takeback's changepwd ajax is updated too, have a look as well.
Title: Re: 2.4 template-making guide
Post by: danny on June 22, 2020, 11:18:48 AM
Should the usage of HFS now be confined to either those degreed in javascript or only-default?  I need to know.  For all other cases then [login] section should not have been redacted from HFS2.4.  This is a very good time to decide what happens when the users try to log-in. 
The varieties need to be restricted to what is useful (and clickable--phone/mobile needs clickable).  Anyway there is need of a restricted set of fixed parameters.

P.S.  It usually does work well.  But, we need to control/define/specify what happens when it doesn't. It is a goal.
Title: Re: 2.4 template-making guide
Post by: rejetto on June 22, 2020, 09:58:09 PM
I found "username not found" neither in your template nor in the default template.
So perhaps it can be a problem with the login section.

"username not found" is  in the EXE, one of the errors the login can return.

Should the usage of HFS now be confined to either those degreed in javascript or only-default?  I need to know. 

at the moment javascript is needed, yes.
This doesn't mean you need to master it, you can just copy and adapt from the default tpl.
For example, adding these lines to your tpl will let you execute showLogin() which in turn will show the dialog.
Like for example  <input type=button onclick="showlogin()" value="Login please" />

Code: [Select]
<script>
var HFS = { user: '{.js encode|%user%.}', sid: '{.cookie|HFS_SID_.}', }
</script>
<link rel="stylesheet" href="/~style.css" type="text/css">
<script type="text/javascript" src="/?mode=jquery"></script>
<script type="text/javascript" src="/~lib.js"></script>

This is not the only way, i just wrote it now.
Probably there is space for improv to let other tpl take just as little as needed for the login.
I know this stuff is much more complicated, guys, but the result is much better, the experience is more what people expect. This is how every bloody website has been loggin you in in the last 10 years. We were stuck in the past.
Maybe with more time I'll be able to give you an alternative that won't force you into javascript, but I fear final 2.4.0 will be like this.
Title: Re: 2.4 template-making guide
Post by: danny on June 25, 2020, 08:00:45 AM
...Probably there is space for improv to let other tpl take just as little as needed for the login...
That could be good.  Currently, login is working nicely with no external calls (no jquery, no css, no extras).  So, maybe a simplified/minimized example could be made? 
Title: Re: 2.4 template-making guide
Post by: dj on June 25, 2020, 09:37:45 AM
Maybe with more time I'll be able to give you an alternative

additional manual add [api level] to template
tested with Throwback
Title: Re: 2.4 template-making guide
Post by: danny on June 26, 2020, 12:20:45 PM
Instead of <a href="/~login>log-in/out</a> with [login|public],
Is it possible to link to [login] (private)? 
Title: Re: 2.4 template-making guide
Post by: rejetto on June 28, 2020, 09:16:03 AM
(Going to make multi-file select to Takeback. Almost done.)
A question: in a folder, after getting (only) selected filenames, how to archive them without jQuery?

something like this
[...document.getElementsByClassName('selector')].filter(x=> x.checked).map(x=> x.parentNode.getAttribute('href'))
Title: Re: 2.4 template-making guide
Post by: rejetto on June 28, 2020, 09:16:44 AM
Instead of <a href="/~login>log-in/out</a> with [login|public],
Is it possible to link to [login] (private)? 

nope, that's the whole sense of public/private, nothing else
Title: Re: 2.4 template-making guide
Post by: danny on July 05, 2020, 04:57:19 AM
nope, that's the whole sense of public/private, nothing else
Thanks!!!
I was just double-checking to see if I got the concept. 
Title: Re: 2.4 template-making guide
Post by: rejetto on July 07, 2020, 09:53:02 AM
natilee OT moved to http://rejetto.com/forum/index.php?topic=13341.0
Title: Re: 2.4 template-making guide
Post by: danny on July 14, 2020, 08:10:43 AM
How do I do a streamlined macro, such as:
if this and this and this all true, then do task, else do other

This could be useful for validation. 
It seems best to validate for at least 2 favorable conditions before doing upload:
%number-addresses-downloading%*%speed-out% < 7500 AND %connections% < 40
then do upload, else javascript 5 sec recheck. 

Throwback14 initial and current check only %number-addresses-downloading%*%speed-out% < 7500 (somewhat less than max workload) then do upload, else recheck in 5s.  That came from  Throwback13 gigabit support. 
Trouble is that 97 ongoing connections on slow-rate internet (workload undetected) could still stop an upload. 
So, it is necessary to test for more-than-one favorable condition before making an upload.
It is straightforward to do if this if then else, but I'm not clear how to macro for if these (several), if then else.

P.S.
Do we have %number-addresses-uploading% or other clue if there were ongoing uploads?  Because I'd like to do, if ongoing uploads, hide archive button. 
Title: Re: 2.4 template-making guide
Post by: dj on July 14, 2020, 08:23:02 AM
{ if | {and | A | B} | C  | D }
wiki (https://www.rejetto.com/wiki/index.php?title=HFS:_scripting_commands#Logic_and_flow)
Title: Re: 2.4 template-making guide
Post by: danny on July 15, 2020, 11:22:11 PM
Thanks DJ!