rejetto forum

Software => HFS ~ HTTP File Server => Topic started by: Konde on October 18, 2007, 04:40:39 PM

Title: Search Engine Builder Professional
Post by: Konde on October 18, 2007, 04:40:39 PM
Hi Guys,

I'm using HFS on my Student Residence as a sharing machine and I'm very happy with it.
But now I was trying to build a search engine for the HFS and I bought the Search Engine Builder and I don't know how to work with it.

Can anyone of you guys do like a little tutorial of how should we install it?

Thanks in advance

Konde
Title: Re: Search Engine Builder Professional
Post by: bacter on October 18, 2007, 10:11:10 PM
I don't know the search engine builder, but eventually two considerations may be useful for you:

- Search shoud be done on server-side, so that only selected fileentries are served. But HFS has no search support and there is no server-side scripting available.

- You could implement search on client side, accessing hfs with the ?recursive option. Instead of displaying the delivered list, you dont display the served list and use a java funtion to select the items and display only the matched file entries. But this method should not be used on a server, because HFS has to create a webpage with perhaps some thousend filentries, thats to much load for the server and needs some seconds to prepare the page. So this method only permits testing your skills in java-script and DOM.

So with hfs, wait for the search function until hfs 3.5 ... ;)  ... except if rejetto moves ?search=(filemask) to a higher position in th todo-list!
Title: Re: Search Engine Builder Professional
Post by: Mars on October 18, 2007, 11:02:24 PM
In the server hfs, there is for every file (in advanced -->> files filter) a mask defined by hfs admin.

 Would be it not possible to associate simply with this filter, a filter of overload defined in the url by the user as why not:

 ?search=(...)   
 ?mask=(...)          associate with ?recursive
 ?filter=(...)
 
with include it in the filesfilter list, and apply it

 
Quote
main.pas

// filtered out
    if not fileMatch( ifThen(f.isFolder(), foldersfilter, filesfilter), f.name)
    then continue;
 


Shall see we it in the version 2.2a build138. ??

Rejetto, is it possible to add at same time the option ?page=uu   working like

?offset=nn                                list files from offset (nn) to end of list
?offset=nn&page=pp                   page is ignored, must working as  ?offset=nn
?offset=nn&limit=mm                   list files from offset (nn) to (offset+limit)
?page=pp&limit=mm                    list files from (after calculate) offset=page*limit to (offset+limit)
?offset=nn&page=pp&limit=mm     offset is ignored and recalculate as above in according with ?page=pp&limit=mm


and why not   ?recursive&nofiles    and    ?recursive&nofolder   to having only folders or files in the list

An idea would be to add the following variables

%recursive%    values TRUE or FALSE
%offset% %page% %limit% values are this passed in the last url line

?offset=0 define the first indice file
?page=0 must define the first page
?limit=0  define no files , will be redefine as 'no limits'

With this, We can use the values tramsmises by variables %var% into template, more easy to use in javascript. thanks
Title: Re: Search Engine Builder Professional
Post by: Foggy on October 19, 2007, 12:27:34 AM
You can create a client side search that uses flash/javascript and searches the files.lst file.

The bonus of that is hfs will only show the files/folders the user who is searching has access to in files.lst.
Title: Re: Search Engine Builder Professional
Post by: Mars on October 19, 2007, 12:34:18 AM
foggy make us a little template that we can see what you want to explain. thanks :o

I can make all the management of filter and pages in javascript but its will more easy for me with rejetto adaptations. ;D
Title: Re: Search Engine Builder Professional
Post by: Foggy on October 19, 2007, 01:02:32 AM
basically I'm saying you can get javascript/flash/some other client side language to download the files.lst file and then search through it.

I have been working on one for the past couple of weeks and it is no where near finished but the basic functionality works.

In the zip file is what I've made so far, just add them to the root of hfs, set hfs to use the files.tpl file and open the html page in hfs.

warning, if you have an old computer with a slow cpu or a very large file system use at your own risk :P
Title: Re: Search Engine Builder Professional
Post by: rejetto on October 23, 2007, 07:30:32 PM
?search=(...)   
 ?mask=(...)          associate with ?recursive
 ?filter=(...)

ok, experimentally i introduce filter, that is applied to both files and folder, or files-filter and folders-filter if you want to specify them separately.
this parameters are used only if no filter is defined in the VFS itself (server side), otherwise they are ignored.

Quote
Rejetto, is it possible to add at same time the option ?page=uu   working like

ok
'offset' will have precedence over 'page'

Quote
and why not   ?recursive&nofiles    and    ?recursive&nofolder   to having only folders or files in the list

just use ?files-filter=\* to get nofiles ;)

Quote
%recursive%    values TRUE or FALSE
%offset% %page% %limit% values are this passed in the last url line

i guess it is better to have a common way to get every URL parameter.
symbols will be called %urlvar-NAME%
Title: Re: Search Engine Builder Professional
Post by: Giant Eagle on October 23, 2007, 08:02:56 PM
<input huge post here...>

omg..

This would allow us to create an advanced folder tree structure on the webpages :|

Posibilities are endless again! thnx :P
Title: Re: Search Engine Builder Professional
Post by: Mars on October 23, 2007, 08:44:54 PM
Dear rejetto thank you for all these positive reponses.

There is a board for the versions beta, one for templates, possibility of opening one board specialized for the propositions of modifications of hfs in the kind of the one that I proposed higher? So we would not be obliged to post almost everywhere in the forum and quickly to see again(to revise) what was already proposed.
Title: Re: Search Engine Builder Professional
Post by: rejetto on October 23, 2007, 09:30:53 PM
the point is that suggestions are often made as replies, and not as topics.
see this topic, it is not a suggestion, it's a request for help. then you made the suggestion.
a board like the one you ask would not solve the problem, because suggestions would still be spread over the boards.
Title: Re: Search Engine Builder Professional
Post by: Mars on October 23, 2007, 09:40:04 PM
i would say a board Specially for hfs  delphi code not same as html or java for templates

have you receiving my mail about delphi code for ?sort=n+ .. ?sort=n-   
Title: Re: Search Engine Builder Professional
Post by: rejetto on October 23, 2007, 09:44:41 PM
a board is needed when there are lots of posts. delphi posts are very rare.

yes i got your private message, but it's still to be read.
Title: Re: Search Engine Builder Professional
Post by: ELEVENNNN on November 01, 2007, 06:32:59 AM
If anyone has the SEBP software I need it, send it to me in a PM or something, I will host it with my bandwidth (real server, not home-based) for anyone who needs it in addition to me. I'll take a look at sites to "purchase" it.

-Kurt
Title: Re: Search Engine Builder Professional
Post by: Garphy on November 07, 2007, 12:08:42 PM
is it possible to search files that in the subfolders?

Title: Re: Search Engine Builder Professional
Post by: TSG on November 07, 2007, 01:41:20 PM
yes, ?filter=*query*&recursive < will show all files and folders.

or ?files-filter=*query*&recursive&folders-filter=\  < will show all files no folders.

or ?folders-filter=*query*&recursive&files-filter=\  < will show all folders no files.

Removing &files-filter=\, or &folders-filter=\ will show all folders regardless of the filter and the filtered files, i am trying to get this removed if the terminology changes.
Title: Re: Search Engine Builder Professional
Post by: SamePaul on November 19, 2007, 03:01:09 PM
url https://samepaul.myftp.org/?search=optokoppler (https://samepaul.myftp.org/?search=optokoppler) lists along with correct files other that are not related to the search mask.
Also on search for "mp3" (https://samepaul.myftp.org/?search=mp3) mask, some files are listed as folders (e.g. 01 - Bittersweet (feat. Ville Valo & Lauri Ylonen).mp3 on my site).
Title: Re: Search Engine Builder Professional
Post by: ELEVENNNN on November 20, 2007, 03:15:06 AM
Quote
Content for my private use. You have no right to download anything.

This isnt related, but if you expect for that to happen, you need to password protect a few things. (Using "List protected items only for allowed users")  I downloaded a song, don't hurt me.
As far as search fixes go, Rejetto, the author of HFS is in Prague for another few days, I'm sure he will look at this when he gets back.

Pretty cool drawings btw.
Title: Re: Search Engine Builder Professional
Post by: SamePaul on November 20, 2007, 09:53:30 AM
Quote
Content for my private use. You have no right to download anything.

This isnt related, but if you expect for that to happen, you need to password protect a few things. (Using "List protected items only for allowed users")  I downloaded a song, don't hurt me.
As far as search fixes go, Rejetto, the author of HFS is in Prague for another few days, I'm sure he will look at this when he gets back.
No hard feelings, believe me  ;) I put disclaimer to .. you know, just for case someone would claim his rights or whatsoever. Things that I really want to protect are SSL-ed and passworded.
Search engine is cool thing and I really appreciate Rejetto's efforts on it. Till today I used google indexing for this, which is not so convenient as internal searching.

Quote
Pretty cool drawings btw.
Thanks  :D
Title: Re: Search Engine Builder Professional
Post by: rejetto on November 20, 2007, 11:06:37 PM
samepaul, the problem is not about the search, as you can see the file/folder is visible even if you just browse the folder /Music/More/Rammstein/Singles/Engel/

if you think this is a bug, as it seems to be :), open a new topic in the bug reports board.
sending me privately your vfs file may help.

if you want people to connect to your hfs only through stunnel (as i guess you are using it), you need to say HFS to accept connections only over 127.0.0.1
i connected to you in simple http.
Title: Re: Search Engine Builder Professional
Post by: ELEVENNNN on November 21, 2007, 01:46:45 AM
Rejetto for stunnel and his situation can you explain the logic behind why people are suppose to connect to him at his loopback address (remotely it would appear?)  How is this translated?  This is just for my edification but I dont understand what 127.001 has to do with ssl connects.

As i know someone will ask my work has a policy that prevents private message of any kind (not physical boundaries) or I would have sent you a pm.

Edit: Thanks for the explanation Unknown, that makes sense.
Title: Re: Search Engine Builder Professional
Post by: Unknown8063 on November 21, 2007, 05:17:19 AM
STunnel handles all external connections and in turn, connects users to HFS.  So from HFS's point of view all STunnel users are coming from 127.0.0.1.  Rejetto suggests binding to 127.0.0.1 to prevent anyone from accessing HFS without going through STunnel.

This actually produced interesting results on one version of the ToG template that assumed 127.0.0.1 was the administrator. :S
Title: Re: Search Engine Builder Professional
Post by: SamePaul on November 21, 2007, 09:12:44 AM
samepaul, the problem is not about the search, as you can see the file/folder is visible even if you just browse the folder /Music/More/Rammstein/Singles/Engel/

if you think this is a bug, as it seems to be :), open a new topic in the bug reports board.
sending me privately your vfs file may help.

if you want people to connect to your hfs only through stunnel (as i guess you are using it), you need to say HFS to accept connections only over 127.0.0.1
i connected to you in simple http.
The problem is not an accessibility of something. I do not want to continue (or even start) discussion on SSL, STunnel or whatever related to this.
I want to focus ONLY on search. And this topic is exactly dedicated to this subject.
Again, the problem is that some files in search results list displayed as folders. For example 02 - Feuerrader.Mp3 is  file and listed as file during regular browsing, but it is listed as folder in search results!
Another bug is that in search results I see files that are not related to search mask. For example I do search by "mp3" mask, but for some reason I see in results files like "suspend.exe" or "timeout.exe" etc. Until someone can explain me how mask "mp3" matches these files, I consider this behavior as bug.
Title: Re: Search Engine Builder Professional
Post by: rejetto on November 21, 2007, 03:25:03 PM
The problem is not an accessibility of something. I do not want to continue (or even start) discussion on SSL, STunnel or whatever related to this.

it was only an additional suggestion and had nothing to do with the rest.

Quote
For example 02 - Feuerrader.Mp3 is  file and listed as file during regular browsing, but it is listed as folder in search results!

what you say does not correspond to my experience.
i accessed your HFS, opened that folder (no searching) and that file is listed as folder.
so the problem is not related to search. that's what i was saying above.

Quote
Another bug is that in search results I see files that are not related to search mask. For example I do search by "mp3" mask, but for some reason I see in results files like "suspend.exe" or "timeout.exe" etc.

at the moment search is not working well because of a bug. if you have file/folder filters set on your VFS, they are conflicting with the search system. it will be fixed ASAP.
Title: Re: Search Engine Builder Professional
Post by: rejetto on November 21, 2007, 06:54:59 PM
From build #145 that file won't be mistaken as a folder.
But it's a partial solution: it won't work anyway because that unicode support is to be completed.