Routing system, or several roots for several websites ?

[Rom_1983]

Edit (2022-04-29) : /!\ DO NOT USE THE PYTHON SCRIPT PROVIDED BY @NAITLEE, UNTIL MORE ARE INVESTIGATED IN THE SECURITY OF THE SCRIPT AND THIS WARNING HAS BEEN REMOVED /!\


Hello,

I have set several alias in DUCKDNS.ORG, like :

• project1.duckdns.org
• project2.duckdns.org

all pointing to my home IP address. They are intented to be public websites.

In HFS, I have REAL folders for each of them :

/
|-- project1
|-- project2

PROBLEM

When I share the URLs, they are like :

Code: [Select]

project1.duckdns.org/project1
project2.duckdns.org/project2

and this is ugly.

This is even worse if I store the REAL folders in a parent "empty" folder :

/
|-- public_websites /
                                   |-- project1
                                   |-- project2

wich leads to URLs like :

Code: [Select]

project1.duckdns.org/public_websites/project1
project2.duckdns.org/public_websites/project2

REQUEST

I would like them to be just project1.duckdns.org and project2.duckdns.org, pointing to the REAL FOLDERS whatever the position in the VFS tree.
For this, I see two solutions.

SOLUTION 1 : A ROUTING SYSTEM

This is basically what's called "URL rewritting".

HFS should provide a way to detect the URL typed, and LINK IT (not redirect !) to REAL FOLDERS, as junctions (in Windows) or hard links (in LINUX) redirect resources of a hard disk.
One way to achieve this would be to right-click on folders, set the URL, allowing us to type "/" in order for HFS to "reroute" the "/" root to those resources, depending on a certain hostname detected.
There's already a macro to detect the important part of the URL :

Code: [Select]

{.header|host.}
But I don't see what to do with it in the DIFF TEMPLATE of the "/" node. 

SOLUTION 2 : SEVERAL ROOTS

HFS should provide several roots ("/") in several UI, like if it was hosting different websites.


If there is already a solution, I would be excited to hear it because I'm actually desperate. DYNDNS don't offer the way to set an IP + a route as a suffix, like "/public_websites/project1", wich would help to translate project1.duckdns.org into something like 70.56.33.81/public_websites/project1 and transmit the request, thus allowing HFS to directly receive and interprete the "/public_websites/project1" route but still letting it hidden to the user in his browser.

[NaitLee]

Hi,

I've tried event script but didn't find a way either...

Once I'm thinking about a "middleware" for HFS 2 to enhance it in a flexible way.

Though I didn't do it earlier, today I took some time to investigate such thing, and by the way did what you may want: "virtual host"

I put my work in attachment, if you can do command line & installed Python >= 3.8, it's very easy to get started.

- Install Python, if not yet there
- Extract to somewhere, if on Windows, right-click while pressing Shift, open CMD or PowerShell.
- Inside invoke middle.py, let's say you want the middleware be at port 80, and let HFS 2 listen 8080:
  py middle.py -p 80 -q 8080 -v project1.duckdns.org,/public_websites/project1 project2.duckdns.org,/public_websites/project2
  (you can put this content in file start.bat and start it by double-clicking later)
- Now go to http://127.0.0.1:80/ and see if it works. (Note: now visiters should use port 80 rather than 8080)

There are also parameter -b and -d to listen other addresses. For more help use -h.
That means you can middleware HFS on another computer.

There are also some drawbacks, for example (currently) slower, and HFS sometimes exposes an absolute link that have the "true" path (though still can't access "true" root directory).

This is not yet considered a "project". Further development not guaranteed...
But fixes in short term is accepted

[Rom_1983]

Hello @NaitLee,

Firstable, if you programmed this right after having read my topic, I want to thank you for your kindness and professionalism 😦. I like seing developers devoted in improving things, that's a great quality in a world often ruled by ignorance and contempt where users are always seen like importunate beggars. A big congratulation 👍

Now, I'm a bit distrustful concerning a program shared on Internet, that could be malicious, so I want to check the code before executing it (it will take me some hours, although the script doesn't seem too long).  That said, it seems interesting since it gives users the full control about the redirection, independently from the DYNDNS providers : no extra accounts connections nor profile tweaking, and that's great !

I've read carefully what you wrote and I have a few questions :

1 - Does the use of the :80 port will affect other programs running like CADDY SERVER, wich could use this port ? What I suspect is : I have several CADDY server binded to various local URL with the :80 port, and if your script doesn't filter the entering URL requesting through port :80, I don't see how your script and servers like CADDY could coexist if your script blindly redirect anything entering through the :80 port.
2 - Is the -b parameter dedicated to that purpose ?
3 - I don't understand the difference between -b and -d : does the first one gives the possibility to analyze the IP requesting, or to route to another computer ? In the second case, HFS should be on that computer, and I don't see why we should precise the HFS location with -d.
4 - What do you mean by "and HFS sometimes exposes an absolute link that have the "true" path" ?

Also, we need to be sure that :
5 - Is the IP "mask" (ex : project1.duckdns.org) chunked with what is added after it by the webusers requests ? Ex : will the request "project1.duckdns.org/stuff/thing.html" be recomposed in "project1.duckdns.org/public_websites/project1/stuff/thing.html" locally ?

[rejetto]

just  a side comment,
I suspect that I can make quite easily such feature in HFS 3, to serve multiple domains each with its folder. Maybe even as a plugin. I'm taking note and will try soon. That would be cool.
I'm in a rush now and cannot study the solution made by NaitLee for HFS 2, but kudos anyway.
that said, HFS is not designed for serving static websites, it's just a secondary feature, so it's good only for simple cases.

[Rom_1983]

I'm taking note and will try soon. That would be cool.

Woohooooo 

@NaitLee : I don't succeed in using your commandline. I got these error messages in my CMD :

Code: [Select]

192.168.1.1 - - [22/Apr/2022 18:58:39] "GET / HTTP/1.1" 400 -
192.168.1.1 - - [22/Apr/2022 18:58:48] "GET / HTTP/1.1" 400 -
162.221.192.90 - - [22/Apr/2022 19:00:52] "GET / HTTP/1.1" 200 -
192.168.1.1 - - [22/Apr/2022 19:02:42] "GET / HTTP/1.1" 400 -
The 19:00:52 strange IP address correspond to this in HFS (the only entry appearing in it) :

Code: [Select]

19:00:52 192.168.1.1:55200 Requested GET /
192.168.1.1 is my box/router.

EDIT :
162.221.192.90 is an IP from Dallax, Texas, attributed to Zenlayer : https://www.lookip.net/ip/162.221.192.90
The port 55200 has been known for being used by XSAN : https://www.adminsub.net/tcp-udp-port-finder/55200 , https://en.wikipedia.org/wiki/Xsan
This is not duckdns, located in Canada : https://www.ip-lookup.org/location/duckdns.org
Many abuse reported from this IP : https://www.abuseipdb.com/check/162.221.192.90

WTF ?

Edit (02.24AM) :

I'm still attacked, with PHP injections now (see attachment). I find suspicious that the day someone gives me a Python script, someone attempts to hack me. 🚨
I have several enemies due to my Twitch channel, and was aware since I announced the creation of my website two weeks ago, but anyway, I find this surprising.
@rejetto : can you check the script please ?

[hanshenrik]

i suppose you could use different ports for it, running project1 on port 1001 project2 on port 1002 project3 on port 1003 etc



buuut maybe your needs exceeds HFS2's capabilities

[NaitLee]

First that sorry for the inconvenience, I didn't expect that it's a big trouble for big-scale users.
One thing to clarify is, I wrote the Python script all by sincere and no malicious thing. I'll never fault anyone.
Some common points were concerned but one is missing: the "Server" HTTP header.
The script did nothing to it, and Python BaseHTTP keeps it's default: Python/<version> BaseHTTP/<version>
This way bad bots could identify that and trying to exploit it with their "common sense" in brute. But of course it will never work except taking network traffic.
To change it is easy. But in your case it's still not so well: the big-scale use case nature is somewhat out of my scope
To filter these spikes is also possible, though needs extra work.

For now under class MiddlewareHandler put these:

Code: [Select]

    def send_response(self, code, message=None):
        self.log_request(code)
        self.send_response_only(code, message)
This removes Server header exposure, by overriding parent class method.
I'll update the code attachment. Please update as it fixes another thing.

You can of course use other port.
The -b is for middleware to "bind" an network address, (ex. "-b 127.0.0.1" for only listening to localhost (that is, only your computer could access it), and any other.)
The "opposite" -d is to tell "where is HFS". Default to localhost.
Same applies to "-p" for middleware to use a port and "-q" to tell what is HFS port.

Answers:
1. Maybe yes. But you can use other ports freely with -p parameter
2. See above. You can know more about "address binding" from internet
3. Also see above. Don't take pressure, the difference is just "letter direction" and "whether middleware or HFS"
4. Ex. with HFS 2.4 default template you can see the directory "breadcrumbs" still contains "project1" etc, even though proxied.
5. Yes!

[NaitLee]

For the attack:
- Not me! My local network/Internet service (provider) structure don't even allow such kind of public access.
- Nothing except the IP address is meanful for analysis.
  So just block/ban the address, in HFS or System (Windows) Firewall.
  An "if" condition in this middleware can also do it.

Code: [Select]

    def do_whatever(self):
        'Just proxy through'
        if self.client_address[0] in ('162.221.192.90', ): # you can add more
            return
        host = self.headers.get('Host')
        # ...

EDIT: I see crack/attack prevention a good/interesting point to discover. I would like to try to do it and see if it would work well.
Middleware could see what's going on, and we can analyze things inside.

[Rom_1983]

One thing to clarify is, I wrote the Python script all by sincere and no malicious thing. I'll never fault anyone.

I want to believe you, I'm thinking about a hasardous coincidence. I have banned the IPs through my Windows firewall.

The -b is for middleware to "bind" an network address, (ex. "-b 127.0.0.1" for only listening to localhost

Ok but how could our computer "listen to" to another computer's requets like if the script was running on it ? That seems impossible to me. That's why I find this option strange.
AH YES ! I know : maybe to bind to another NIC on the same computer ? That would be useful.

My thought is different, because what a domain name "hosts" (virtually) which "directory" is totally up to you.

That doesn't answer to my question (or I don't understand). Anyway I can't test it and observe the redirection, because your script doesn't work and I have ERRORS as I said.
My HFS is set to port 8000, I've opened that port on my router (NAT) binded to my PC LAN IP, and I've set "sandwichtv.duckdns.org=sites_publics/sandwichTV" to work with your script (with 127.0.0.1 and 80 / 8000 ports). I have Python v3.9 installed.
I type "http://sandwichtv.duckdns.org" in my browser and this gives me the ERRORS.

Code: [Select]

@echo off
start cmd /k py middle.py -p 80 -d 127.0.0.1 -q 8000 -v sandwichtv.duckdns.org,sites_publics/sandwichTV
exit /b

[NaitLee]

Sorry that I've changed something before but forgot to say. 
You need absolute path in middleware -v parameter: (put slash at beginning)

-v folder1,/folder1 folder2,/another/folder2

Paths are still corresponding to HFS virtual dir structure.

I just implemented some banning mechanisms. The script structure is not that good, and I should clean it later.
It prevents some common attack attempts while banning that bad client. While this may be too "violent" for casual curious guys.
Things happened are available in file banlist.txt
You can test it anyway. See attachment.

[Rom_1983]

FINALLY ! It works !  Indeed, it was the "/" missing at the begining.
Congrats for the banning system, it will help to secure things.  I'm actually working on a MS-DOS batch to facilitate adding IPs to a desired rule of the Windows' firewall dedicated to block them.
I'm also working on a batch to automate the use of your Python script, with an INI file and the possibility to set several HFS servers and/or hosts+routes.

I'll keep in touch.

[Rom_1983]

@naitlee : I sent you a PM regarding a new security problem. Did you received it ?

[rejetto]

what if i told you i just finished a 15-lines plugin that gives you this.

...HFS 3 of course

[vladimirov70]

what if i told you i just finished a 15-lines plugin that gives you this.

...HFS 3 of course

Thanks!

[vladimirov70]

Two different sites, using only HFS settings.
Menu -Start/Exit -Uncheck "1 instance only".
Launch the first instance of HFS. Menu - IP adress - Put a checkmark in the "Dont include port in URL" box. This will be the first site with an address like http://191.ХХХ.ХХХ.ХХХ.
Launch second HFS instance. Menu - IP adress - Uncheck "Dont include port in URL".
This will be the second site with an address like http://191.ХХХ.ХХХ.ХХХ:8080
I checked it today and it works.