rejetto forum
Software => HFS ~ HTTP File Server => Topic started by: Rarst on April 28, 2008, 09:25:55 PM
-
First huge thanks for this great program. :) I use it alot and like it alot. :)
Only thing it lacks for me - RSS functionality (even forum search only gives few dusty topics). So I made small program that generates RSS feed with links and such for folders shared in my HFS. Programming language used - AutoIt v3 ( http://www.autoitscript.com/ ). It's bit messy since I made it for myself but it works and produces valid RSS 2.0 feeds.
So I decided to ask:
1. If someone interested in such feed generator?
2. If HFS author ok with me posting source code for my prog here (in case someone wants it which depends on first question :) )?
-
1. If someone interested in such feed generator?
I have no need for one currently but someone else might. :)
2. If HFS author ok with me posting source code for my prog here (in case someone wants it which depends on first question :) )?
I highly doubt rejetto would have any problems with that.
What is contained in the rss feed?
-
What is contained in the rss feed?
Each RSS item (message) has file name, file size, file creation date and http link to file.
Can scan few directories into one feed, include files by mask, exclude files by mask, exclude files by date (only include created in last X days). Can create feed and exit or sleep in tray, updating feed in specified intervals.
That's current state (which of course might change for better).
Main downside - it's hardly one-click setup. User has to setup at least few paths (local and website) and there is bunch of optional RSS tags as well.
I am currently rewriting it for reading options from ini file - much friendlier than hardcoded values I used at first. :)
-
Each RSS item (message) has file name, file size, file creation date and http link to file.
Can scan few directories into one feed, include files by mask, exclude files by mask, exclude files by date (only include created in last X days). Can create feed and exit or sleep in tray, updating feed in specified intervals.
You can code most of those features into the template of the latest hfs beta versions with the help of template macro's. That way you could always get a live rss feed but it would take a fair bit of work to code.
if that interests you take a look in the hfs wiki in the macro section to get an idea of the power hfs has.
-
Macros are beta feature... And they look even less human language than average programming language... :) I looked through section and probably understood nothing except that HFS has macro... :)
-
Yep, I understood that much (that macroses are... well macroses :) ).
But I don't have slightest clue how could I use them to produce RSS feed.
Roughly my algorithm is:
1. Generate header section (big, plenty of tags)
2. Generate footer section (tiny, just closing xml tags)
3. Scan directory, generate <item>s for each file that matches criteria (included, not excluded, not expired). Replace characters that are invalid in xml (spaces and such). Repeat for few directories if needed.
4. Glue everything together into complete feed, write feed to file.
Is it possible with macro? What amount of pain would it be to write? Would it be to give final result to another user? I want to make few friends use it and they are much less tech-savvy.
-
Yeah, it is possible.
do you understand how the template system works, with the different sections?([file],[files],etc)
for the end user it could be as simple as loading a template into hfs.
except instead of saving it to file it would be created each time it is requested.
-
I hadn't tried changing template (had no need) but looked at it and I understand concept (I know html/css).
Main template purpose seems to be controlling existing page elements, what should I read in docs on how to generate files with it?
One more important thing - RSS requires time stamps in RFC822 format and it's huge pain since it uses words for day and month (Tue, 29 Apr 2008 12:04:25 +0200). HFS seem to operate with time in OS format, would I be able to convert in RFC822?
-
at first I was thinking you could modify the main template but I remembered about files.lst which is a seperate template for when you append ~files.lst to the url.
ill have a look at the RFC822 format.
http://www.rejetto.com/wiki/index.php?title=HFS:_Hidden_things
the example in the url is the default list. try accessing it in your hfs to understand how it works.
edit: I cant see a way of finding out the day of the week but it should be possible to convert the rest of the time/date.
-
Reference link for RFC822 (RSS allows 4 digit year, but the rest must be strictly by format)
http://asg.web.cmu.edu/rfc/rfc822.html#sec-5
I'll poke that template bit later today.
-
Ok, I have made rough filelist template (can't test it properly at work):
<?xml version="1.0" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>%host%%folder% directory feed</title>
<description>Files in %host%%folder% directory</description>
<link>http://%host%</link>
<atom:link href="http://%host%%folder%~files.lst?recursive" rel="self" type="application/rss+xml" />
%files%
</channel>
</rss>
[files]
%list%
[file]
<item>
<title><![CDATA[%item-name%]]></title>
<description><![CDATA[%item-name% (%item-size%)]]></description>
<guid><![CDATA[http://%host%%folder%%item-url%]]></guid>
</item>
It seems that's as far as symbols can get it. HFS takes care of inclusion/exclusion by filetype so that question gone.
What remains:
- feed generated - timestamp in RFC822
- file created - timestamps in RFC822 (btw I only found symbol for file modification date in wiki, no symbol for creation date?)
- exclusion by file creation date (otherwise feed is going to be huge for big folder)
- symbols seems to work with current folder, if I need few - I stack them in one and use ?recursive, right?
- %item-url% takes care of spaces, converting them to %20, but it is also needed to change square brackets [ ] to %5B %5D (maybe other symbols, but these three are only one that gave me trouble with validation so far)
PS template and symbols seem so much nicer and fluffier than those scary macros :)
-
Ok, I have made rough filelist template (can't test it properly at work):
it seems to work
<?xml version="1.0" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>foggy.dnip.net/Media/the chasers/ directory feed</title>
<description>Files in foggy.dnip.net/Media/the chasers/ directory</description>
<link>http://foggy.dnip.net</link>
<atom:link href="http://foggy.dnip.net/Media/the chasers/~files.lst?recursive" rel="self" type="application/rss+xml" />
<item>
<title><![CDATA[chaser_2007_ep14.wmv]]></title>
<description><![CDATA[chaser_2007_ep14.wmv (89.28 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep14.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep15.wmv]]></title>
<description><![CDATA[chaser_2007_ep15.wmv (87.97 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep15.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep16.wmv]]></title>
<description><![CDATA[chaser_2007_ep16.wmv (88.01 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep16.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep17.wmv]]></title>
<description><![CDATA[chaser_2007_ep17.wmv (88.31 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep17.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep18.wmv]]></title>
<description><![CDATA[chaser_2007_ep18.wmv (91.12 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep18.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep19.wmv]]></title>
<description><![CDATA[chaser_2007_ep19.wmv (89.27 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep19.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep20.wmv]]></title>
<description><![CDATA[chaser_2007_ep20.wmv (88.17 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep20.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep21.wmv]]></title>
<description><![CDATA[chaser_2007_ep21.wmv (88.12 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep21.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep22.wmv]]></title>
<description><![CDATA[chaser_2007_ep22.wmv (90.96 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep22.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep23.wmv]]></title>
<description><![CDATA[chaser_2007_ep23.wmv (91.10 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep23.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep24.wmv]]></title>
<description><![CDATA[chaser_2007_ep24.wmv (93.05 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep24.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep25.wmv]]></title>
<description><![CDATA[chaser_2007_ep25.wmv (94.43 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep25.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep26.wmv]]></title>
<description><![CDATA[chaser_2007_ep26.wmv (91.90 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep26.wmv]]></guid>
</item><item>
<title><![CDATA[chaser_2007_ep27.wmv]]></title>
<description><![CDATA[chaser_2007_ep27.wmv (187.18 MB)]]></description>
<guid><![CDATA[http://foggy.dnip.net/Media/the chasers/chaser_2007_ep27.wmv]]></guid>
</item>
</channel>
</rss>
It seems that's as far as symbols can get it. HFS takes care of inclusion/exclusion by filetype so that question gone.
you can also specify filters in the url to only show certain files.
- exclusion by file creation date (otherwise feed is going to be huge for big folder)
you can sort by the creation/modified date and can combine that with a limit of X files.
- symbols seems to work with current folder, if I need few - I stack them in one and use ?recursive, right?
correct
[/quote]
- %item-url% takes care of spaces, converting them to %20, but it is also needed to change square brackets [ ] to %5B %5D (maybe other symbols, but these three are only one that gave me trouble with validation so far)
[/quote]
{.replace|[|%5B|]|%5D|%item-url%.}
PS template and symbols seem so much nicer and fluffier than those scary macros :)
yeah, they are a lot simpler but you cant do anything dynamic with them :(
-
Ok, at home at can properly test it...
New issue - how do I make public feed for closed folder? Filelist seems to be treated as included in folder.
Another new one - it seems I can't set mime type for file list, even if I add *.lst application/rss+xml - it is still served as text/plain.
you can sort by the creation/modified date and can combine that with a limit of X files.
Sort as with sort=[ n | e | s | t | d ] Sort by [ Name | Ext | Size | Date | Hits ] ?
Sorting by date... Well it sorts by something but I can't find it related to dates in any way. Is it supposed to work on stable (wiki mentions that some of these are beta) ?
-
Sort as with sort=[ n | e | s | t | d ] Sort by [ Name | Ext | Size | Date | Hits ] ?
Sorting by date... Well it sorts by something but I can't find it related to dates in any way. Is it supposed to work on stable (wiki mentions that some of these are beta) ?
It should work in all versions since it was introduced.
sort=t should sort by date. combine that with limit=X and you could display a feed of the 10 newest files, etc
-
Crap, my bad. :) I assumed D stands for date without paying close attention.
But now it seems I can't use sort and limit at same time. They work separately but not when applied together.
Upd: and in file list limit is always ignored
Also I played with public/private and it seems that by HFS mechanics there will be no way to make public file list for private folder. File list is generating according to user rights, so even if user (@anonymous) gets file list - all private files he can't see in html won't appear in file list either.
It's interesting playing with file list template and it allows lots of customization but seems some hard-coded limitations prevent from using it as feed. :(
-
But now it seems I can't use sort and limit at same time. They work separately but not when applied together.
?sort=t&limit=X
Upd: and in file list limit is always ignored
I didnt know that.
-
?sort=t&limit=X
Works but not the way needed... It first limits and then sorts.
btw I have some troubles with limit... It is supposed to limit to first nn files from name-sorted list? Well it doesn't in some folders. I think it has trouble with files that start with bracket ( or [. Seems to work fine in folder where there are no such files. When there are - limit gives mix of files started with bracket and other files, while it should only give first ones - with bracket and never get to bracketless which are further down the list.
Upd: and in file list limit is always ignored
I didnt know that.
Sorry, my mistake again. I think that was because I tried ?limit=x?recursive insted of joining with &.
PS second time I am missing something in wiki... that color scheme is weird example of sufficient contrast, nice colors but barely readable (at least for my eyes) result :)
PPS if I bug too much we can close filelist question for now, it seems public/private, mime type and RFC822 timestamps won't be solved by small tweaks anyway
Can some one confirm is this correct limit behaviour or I should post it to bug reports?
http://localhost/temp2/~files.lst
[a.txt
[b.txt
[c.txt
[d.txt
[e.txt
[f.txt
[g.txt
[h.txt
[i.txt
[j.txt
a.txt
b.txt
c.txt
d.txt
e.txt
f.txt
g.txt
h.txt
i.txt
j.txt
http://localhost/temp2/~files.lst?limit=15
[a.txt
[b.txt
[c.txt
[d.txt
[e.txt
a.txt
b.txt
c.txt
d.txt
e.txt
f.txt
g.txt
h.txt
i.txt
j.txt
-
i'm a bit late, but you can post your program if you still want to (yes, i read the rest of the topic).
It's up to users to trust you about running an executable. ;)
?sort=t&limit=X
Works but not the way needed... It first limits and then sorts.
the opposite method is in the to-do-list (other discussion (http://www.rejetto.com/forum/index.php?topic=5196.msg1029734#msg1029734))
-
i'm a bit late, but you can post your program if you still want to (yes, i read the rest of the topic).
Thousands of potential happy users for my program are even more late as you see... It seems interest for RSS in userbase hovers around 1 person per year. :)
If you are ever converted to RSS fondness and decide to implement it properly I would be happy to cook up list of stuff for to-do. :)
-
well, what i want to add to HFS is to be able to serve a folder by using several templates.
because the template system should be enough for RSS generating,
you would just have a template named "rss".
-
well, what i want to add to HFS is to be able to serve a folder by using several templates.
because the template system should be enough for RSS generating,
you would just have a template named "rss".
Yep, template system itself is fitting but there are number of features (abovementioned: mime type, RFC822 timestamps, access to private files, etc) lacking for proper RSS implementation at moment.
-
Well, since I got PM from a willing
test subject person... :)
http://strel28-24.skif.com.ua/folder2rss.zip
.au3files - are source code
.exe - compiled program
.ini - setup stuff (important)
1. Unpack somewhere
2. Read ini-file
3. Make sure you understand what you read (important)
4. Setup ini file the way you need
5. Run exe (make it autorun if you want)
6. Put xml that appeared into HFS
6a. Validate link to your RSS with http://feedvalidator.org/
6b. If something doesn't work repeat 3 or ask me
PS 7 Some feedback... I get nervous when I see download followed by long silence :)
PPS it's not exactly noob friendly at moment... For people who hadn't ever seen contents of ini file http://en.wikipedia.org/wiki/INI_file might be good place to start.
-
Merci Rarst !
Sorry for PS 7 may be not today ... but sure will input feeback ;)
TCube
-
I'm interested ...
-
Forgot one thing, this must be set in HFS:
Menu -> Other options -> MIME types
*.xml application/rss+xml
-
Salut !
Just getting the grasp for various browsers to handle RSS - to be be sure RSS from HFS works fine for users.
A question "top of my head" : RSS can be hot linked/embed into video streaming object from an html page ... can this be avoided ?
:P
merci
TCube
-
Yep, template system itself is fitting but there are number of features (abovementioned: mime type, RFC822 timestamps, access to private files, etc) lacking for proper RSS implementation at moment.
i will work on this.
setting the mime type could be a macro.
timestamps formatting should be already be possible through macros, i'll see.
what you mean by access to private files?
-
what you mean by access to private files?
yes, ? , seems no problem - I've tried moving the RSS folder 'round VFS - i.e :
- 1 private files with "user access" within a bunch of "open" files into RSS : this peculiar file is not seen.
- Login required when the RSS is within a private folder
Looking for something more complicated Rarst ? ;D
-
RSS can be hot linked/embed into video streaming object from an html page ... can this be avoided ?
Lost me, can you please rephrase? Also is it on topic of my program or RSS in general?
setting the mime type could be a macro.
Anything goes, it's just for better compatibility. Most RSS readers are able to digest RSS feed even without proper mime. Still it's better to follow rules. :)
timestamps formatting should be already be possible through macros, i'll see.
Main problem is stupid RFC822 uses words for day of week (Mon, Tue, Wed...). If there is a way to do it with macro it's totally beyond my abilities (HFS macro syntax puzzles me to the point of brain damage :( ).
what you mean by access to private files?
Bit hard to explain, I'll make example with some screenshots later today.
Looking for something more complicated Rarst ?
Like what? :)
-
Ok, on public\private. See screenshot for example FS.
(http://www.rejetto.com/forum/index.php?action=dlattach;topic=5846.0;attach=2673;image)
Without login to private folder:
http://localhost/~files.lst?recursive
http://localhost/public/public.txt
http://localhost/public/private/~files.lst?recursive
Unauthorized
This is a protected resource.
Your username/password doesn't match.
With login to private folder:
http://localhost/~files.lst?recursive
http://localhost/public/private/private.txt
http://localhost/public/public.txt
http://localhost/public/private/~files.lst?recursive
http://localhost/public/private/private.txt
So if folder is private it is not possible to access file list for it without login. Assuming RSS stream gets same behaviour - it would be necessary to include login info into link to get RSS for private folder. And I am not even sure that RSS readers can digest it properly. And it sucks from usability viewpoint. And hidden files are ignored btw.
So in theory RSS stream going to need something like "include hidden files in RSS" and "include restricted files in RSS" options.
-
Hello Rarst,
1st point : my comment is general .
But that's allright ! If I don't want people embedding videos within RSS info, I'll have to specify [include mask, defaults to *.*] include=*.mp3 *.jpeg *.bmp and so on ... in your program.
-
I'll have to specify [include mask, defaults to *.*] include=*.mp3 *.jpeg *.bmp and so on ... in your program.
It doesn't support multiply include masks (and exclude masks) at moment. :( I use it for neatly organized stuff so this wasn't priority. :) btw there is also hardcoded exclusion for *.!ut files (torrent temps).
Oh, just realized that it is possible to simulate multiply include masks by scanning folder few times - few [scanX] sections with same folder and different "include=" shoud work.
May need to implement masks properly, depends on how soon we may get native HFS support for RSS. ;)