rejetto forum
Software => HFS ~ HTTP File Server => Programmers corner => Topic started by: bacter on November 04, 2009, 04:20:37 PM
-
When sending from a form with method 'get' or 'post', values that contain curlybraces with his points or the macro expression separator '|', the values seem to be sent to the server.. but hfs clears the whole content of this values that contain {., {: and .}, :} on arriving.!
I see this when the method is 'get' at least, and experience the same when the method is 'post'.
This is really annoying when you type or modify a chunk of text and loose it this way. I suppose that for security reasons no macros (or elements of) can be accepted in this type of data... but perhaps it would be possible to 'escape' in some way the macro delimiters. Using html char values like { do not solve the problem.
This escape could be replacing the signs in the incoming postvar for further use in some way like this:
{. --> {+.,
.} --> .+},
{: --> {+:,
:} --> :+} , etc.
| --> ¦
or this way:
{ --> ({)
} --> (})
| --> (or)
This would allow to save {.any macro|expr.} as {+.any macro¦expr.+} or ({).any macro(or)expr.(}), something that can never be reverted to a executable macro. This would allow to use the macro signs {:{.another macro.}:} inside documents used in a wiki, forum or any cms.
When loading such a file, then we can then replace the codes with the original signs and no macro formed this way will be executed.
Thanks for a possible solution.
-
can't you "escape" such values with javascript?
something like
string.replace("{.", "{+.")
-
I suppose it would be possible to use javascript to do this .. but if anyone use forms with method post and don't use such a script to do the conversion, any 'error' typing those sequences by accident (its sometimes nice to use | to separate some items) clearing the postvar may be loss of data.
So anyway, also if a workaround with javascript would be possible, i think it would be better to 'escape' the offending caracter(sequences) or to substitute them with any allowed sign (so at least, data is not lost and the user may correct his sintax).
to simplify it could be:
| --> ¦
{. --> ¦. , {: --> ¦:
.} --> .¦ , :} --> :¦
or this way:
| --> ¦
{. --> ¿. , {: --> ¿:
.} --> .? , :} --> :?
As we can assume that ¦ is never entered directly from keyboard and has no specific use, such a substitution would eliminate the risk of data loss and would also preserve the security of hfs against introducction of malicious code.
-
i think we should avoid inventing an encoding, giving headache to someone.
i would go for an html encoding.
&#XXX;
do you want to test it?
http://www.sendspace.com/file/8dkbh5
-
Thanks Rejetto!
Your solutions to the problems i find, are always far better than my proposals to avoid them! :D :D :D
Also i don't understand why you 'escape' the points of the end macro to #46 and #58, instead of the ending brace } to #125, your solution works perfectly!
Thank you very much! ;) ;)
-
happy to know it's working. i will keep it for the next release, hoping there's no unwanted side effect :P
Also i don't understand why you 'escape' the points of the end macro to #46 and #58, instead of the ending brace } to #125, your solution works perfectly!
the method is generic, and will just encode the first character.
i find pointless to distinguish between symbols.
-
Hi Rejetto
I have found another problem with caracter in postvar and urlvar.
When a postvar contains some template simbols like %user% , %total-in% they are immediatly substituted, if it contains names of sections like %upload% nothing happens and the symbol remains remains intact, with other section names like %files%, they are substituded by some partial execution: some html from the files section plus spaces and CR-LF generated by macro executions is included.
I see two possible solutions to the problem:
A) The percent sign % should be replaced by % in postvar and urlvar.
B) Assure that a postvar dont suffer any implicit interpretation, so that we can apply {.set|data|{.replace|%|%|{.postvar|fieldname.}.}.} before writing data to a file.
PD: Don't forget the breadcrumbs-from bug (still buggy) ;)
-
Don't think %XXX% is for getting sections text (that's demanded to {.section.} command).
This is true only for SOME symbols, others have different meaning.
You will find a good list at http://www.rejetto.com/wiki/index.php/HFS:_Template_symbols
and you'll see there's no %upload%.
Symbols substitution is at the moment made in an initial step, before macros.
It cannot be done AFTER, because many macros uses %symbols% value to work, like {.if.}
To do it WHILE, i should change the parser deeply.
At the moment there's no way to stop %symbols% like for {:quoting:}.
I may even introduce a special way to do it, but i think that for such special case a javascript encoding should be enough.
Many of these cases, anyway, would be solved by just not having the text "expanded", but managed in a variable.
I'll try to be clearer: when you call {.postvar.} or {.urlvar.} you actually get its value made part of the output (the html), even just for a while.
If you do nothing more, you will get the value in your face, in the browser.
If you instead put such value as parameter of another command, like {.set.} or {.save.}, then it will not be displayed, but instead its used for further elaboration. That's often the case for an application like the one you are developing.
By having this value directly put in a variable, instead of being replaced in the output, we should nicely workaround the problem. And it would be even more efficient.
In the example you made, you would not need to {.set.} and maybe not even {.replace.}
Some commands may just be adapted to be able to work with variables, like i already did in the last year.
i'm sorry for the breadcrumbs, but i had truly no time lately.
this was long, but i hope it was clear enough.
next release.
-
Thanks Rejetto for yout long answer.
I understand that the % symbol is a rather complex question, as it implies different modes of treatment.
A) when applied to sections, %files% or {.$files.} for example are equivalent.
B) when applied to symbols like %user%, there is (at the moment) no alternative way to access this 'variable'.
Perhaps in a future - i know this may lead to compatibility problems with 'old' templates - this variables (from type B) could be substituted by pre-initialized variables: %user% ---> {.^user.}
You are right that the problem can be solved with a workaround in javascript. But as i want to get the wiki and forum things working initially without javascript, i will do some additional 'escaping' at the moment: .%. will be displayed as % . As the %-problem is not very destructive , this should be sufficient. I would reconsider the javascript in future versions that uses a javascript editor instead of simple forms.
---
By the way, i don't know if there is a bug with numeric macros and real numbers, or if i don't use proper sintax: calculations (calc, add ..) dont work with real numbers ({.calc|8,7-6,4.} does not return anything {.calc|1+(5/2).} returns a real number!
-
i guess you should try by using a dot instead of a comma, as decimal separator.
about the javascript, you should consider having the js script degrade gracefully, having all others miss the encoding, and being unable to use such character.
you may even show a warning, "please enable javascript, or you won't be able to use the % sign in your writing."
javascript should not be disabled for every site. extra-security-minded people should be able to enable on a per-site basis.
-
I guess you should try by using a dot instead of a comma, as decimal separator.
Problem with real numbers seems more complex than this:
hfs is normally started with windows. Then an integer calc operation {.calc|1+(5/2).} returns 3,5 and all operations that involve real numbers - no matter if coma or point is used - return 0 or nothing.
When i stop hfs, then restart it with running windows, the calc operation {.calc|1+(5/2).} returns 3.5 !!, and calc with real numbers with decimal point dot works, but add,sub and mul dont either.
-
i didn't understand.
can you please tell me how to reproduce the problem?
next version, anyway, will support real numbers with add/sub/mul/div
-
Hi rejetto:
When i had problems using math functions, doubting to use decimal separator comma or point, i added the following code at the bottom of my template to test:
<br /> {.calc|1+(5/2).}
<br /> 8.7- 6.4 point:{.calc|8.7-6.4.} coma:{.calc|8,7-6,4.}
<br /> add 8.7 6.444 point:{.add|8.7|6.444.} coma:{.add|8,7|6,444.}
<br /> sub 8.7 6.444 point:{.sub|8.7|6.444.} coma:{.sub|8,7|6,444.}
<br /> mul 8.7 6.444 point:{.mul|8.7|6.444.} coma:{.mul|8,7|6,444.}
<br /> round 8.7345, 2 point: {.round|8.7345|2.} coma: {.round|8,7345|2.}
So i observed the following results:
3,5
8.7- 6.4 point: coma:
add 8.7 6.444 point:0 coma:0
sub 8.7 6.444 point:0 coma:0
mul 8.7 6.444 point:0 coma:0
round 8.7345, 2 point: coma: 8,73
This is the output when hfs starts with windows after booting.
When i was doubting if it could be a problem of the special build i used, i tried with previous version and got the following output:
3.5
8.7- 6.4 point:2.3 coma:
add 8.7 6.444 point:0 coma:0
sub 8.7 6.444 point:0 coma:0
mul 8.7 6.444 point:0 coma:0
round 8.7345, 2 point: 8.73 coma:
So i first thought it would be a problem of the build, and returned to the newest special build. And it worked too!
Result is: After a new boot or reboot windows (shutdown and restart the computer), hfs uses the comma separator. Then i exit hfs, rerun it, and it works using the point separator!
I have checked that there is no other hfs.exe in any directory to be sure that its the same .exe is used on startup of the machine :)
EDIT
This is so with windows XPp SP3 , can't test it with W7 because HDD died this morning
-
how is your hfs started at first?
is it in the "startup" folder? is it a service?
-
Rejetto:
there is a link to hfs.exe in the startup folder.
i have activated the menu-> start/exit -> run hfs when windows starts option.
I thought that perhaps it has to do with the country information that could be not ready when hfs starts (if it retrieves it ?), but the country information for spain says to use the point for thousands grouping and comma as decimal separator.
But this does not seem to be the reason, as coma is used just after booting, and is changed to point after exit and re-run.
-
Hi rejetto:
With new build 249 i get the following results of the above macros.
first run with windows booting:
3,5
8.7- 6.4 point: coma:
add 8.7 6.444 point:0 coma:15,144
sub 8.7 6.444 point:0 coma:2,256
mul 8.7 6.444 point:0 coma:56,0628
round 8.7345, 2 point: coma: 8,73
after re-run:
3.5
8.7- 6.4 point:2.3 coma:
add 8.7 6.444 point:15.144 coma:0
sub 8.7 6.444 point:2.256 coma:0
mul 8.7 6.444 point:56.0628 coma:0
round 8.7345, 2 point: 8.73 coma:
To avoid problems if somebody uses some floating point constants in the template, i suggest to use fixed point notation.
-
thank you for your precious testing bacter.
i will think of a solution.
-
By looking at the calls of functions system, it is evident that it is the local format of the server that is used for all the calculations
result:=floatToStr(parF(0) [color=brown]operator[/color] parF(1));
function FloatToStr(Value: Extended): string;
function FloatToStr(Value: Extended; const FormatSettings: TFormatSettings): string;
result:=floatToStr(parF(0) operator parF(1)), FormatSetting(par('format'));
adding a param as 'format' can solve the bacter' problem , we can take Microsoft Excel as example to define the format by using chars as '#' '0' ',' '.'
## ##0,00
0.#
etc ....
TFormatSettings = record
CurrencyFormat: Byte;
NegCurrFormat: Byte;
ThousandSeparator: Char;
DecimalSeparator: Char;
CurrencyDecimals: Byte;
DateSeparator: Char;
TimeSeparator: Char;
ListSeparator: Char;
CurrencyString: string;
ShortDateFormat: string;
LongDateFormat: string;
TimeAMString: string;
TimePMString: string;
ShortTimeFormat: string;
LongTimeFormat: string;
ShortMonthNames: array[1..12] of string;
LongMonthNames: array[1..12] of string;
ShortDayNames: array[1..7] of string;
LongDayNames: array[1..7] of string;
TwoDigitYearCenturyWindow: Word;
end;
{ Currency and date/time formatting options
The initial values of these variables are fetched from the system registry
using the GetLocaleInfo function in the Win32 API. The description of each
variable specifies the LOCALE_XXXX constant used to fetch the initial
value.
CurrencyString - Defines the currency symbol used in floating-point to
decimal conversions. The initial value is fetched from LOCALE_SCURRENCY.
CurrencyFormat - Defines the currency symbol placement and separation
used in floating-point to decimal conversions. Possible values are:
0 = '$1'
1 = '1$'
2 = '$ 1'
3 = '1 $'
The initial value is fetched from LOCALE_ICURRENCY.
NegCurrFormat - Defines the currency format for used in floating-point to
decimal conversions of negative numbers. Possible values are:
0 = '($1)' 4 = '(1$)' 8 = '-1 $' 12 = '$ -1'
1 = '-$1' 5 = '-1$' 9 = '-$ 1' 13 = '1- $'
2 = '$-1' 6 = '1-$' 10 = '1 $-' 14 = '($ 1)'
3 = '$1-' 7 = '1$-' 11 = '$ 1-' 15 = '(1 $)'
The initial value is fetched from LOCALE_INEGCURR.
ThousandSeparator - The character used to separate thousands in numbers
with more than three digits to the left of the decimal separator. The
initial value is fetched from LOCALE_STHOUSAND. A value of #0 indicates
no thousand separator character should be output even if the format string
specifies thousand separators.
DecimalSeparator - The character used to separate the integer part from
the fractional part of a number. The initial value is fetched from
LOCALE_SDECIMAL. DecimalSeparator must be a non-zero value.
CurrencyDecimals - The number of digits to the right of the decimal point
in a currency amount. The initial value is fetched from LOCALE_ICURRDIGITS.
DateSeparator - The character used to separate the year, month, and day
parts of a date value. The initial value is fetched from LOCATE_SDATE.
ShortDateFormat - The format string used to convert a date value to a
short string suitable for editing. For a complete description of date and
time format strings, refer to the documentation for the FormatDate
function. The short date format should only use the date separator
character and the m, mm, d, dd, yy, and yyyy format specifiers. The
initial value is fetched from LOCALE_SSHORTDATE.
LongDateFormat - The format string used to convert a date value to a long
string suitable for display but not for editing. For a complete description
of date and time format strings, refer to the documentation for the
FormatDate function. The initial value is fetched from LOCALE_SLONGDATE.
TimeSeparator - The character used to separate the hour, minute, and
second parts of a time value. The initial value is fetched from
LOCALE_STIME.
TimeAMString - The suffix string used for time values between 00:00 and
11:59 in 12-hour clock format. The initial value is fetched from
LOCALE_S1159.
TimePMString - The suffix string used for time values between 12:00 and
23:59 in 12-hour clock format. The initial value is fetched from
LOCALE_S2359.
ShortTimeFormat - The format string used to convert a time value to a
short string with only hours and minutes. The default value is computed
from LOCALE_ITIME and LOCALE_ITLZERO.
LongTimeFormat - The format string used to convert a time value to a long
string with hours, minutes, and seconds. The default value is computed
from LOCALE_ITIME and LOCALE_ITLZERO.
ShortMonthNames - Array of strings containing short month names. The mmm
format specifier in a format string passed to FormatDate causes a short
month name to be substituted. The default values are fecthed from the
LOCALE_SABBREVMONTHNAME system locale entries.
LongMonthNames - Array of strings containing long month names. The mmmm
format specifier in a format string passed to FormatDate causes a long
month name to be substituted. The default values are fecthed from the
LOCALE_SMONTHNAME system locale entries.
ShortDayNames - Array of strings containing short day names. The ddd
format specifier in a format string passed to FormatDate causes a short
day name to be substituted. The default values are fecthed from the
LOCALE_SABBREVDAYNAME system locale entries.
LongDayNames - Array of strings containing long day names. The dddd
format specifier in a format string passed to FormatDate causes a long
day name to be substituted. The default values are fecthed from the
LOCALE_SDAYNAME system locale entries.
ListSeparator - The character used to separate items in a list. The
initial value is fetched from LOCALE_SLIST.
TwoDigitYearCenturyWindow - Determines what century is added to two
digit years when converting string dates to numeric dates. This value
is subtracted from the current year before extracting the century.
This can be used to extend the lifetime of existing applications that
are inextricably tied to 2 digit year data entry. The best solution
to Year 2000 (Y2k) issues is not to accept 2 digit years at all - require
4 digit years in data entry to eliminate century ambiguities.
Examples:
Current TwoDigitCenturyWindow Century StrToDate() of:
Year Value Pivot '01/01/03' '01/01/68' '01/01/50'
-------------------------------------------------------------------------
1998 0 1900 1903 1968 1950
2002 0 2000 2003 2068 2050
1998 50 (default) 1948 2003 1968 1950
2002 50 (default) 1952 2003 1968 2050
2020 50 (default) 1970 2003 2068 2050
}
-
The problem is not the output (floatToStr). It might be ok that local settings of the server is used to format floating point values.
The problem is in the input, as macro does not use typed variables:
Suppose if you save in a #var the time: 40142,12345 with {.set|#t1|%now%.}
Then you want to calculate a time t2 32 hours or 1,5 days later.
In some systems you then should write: {.set|#t2|{.add|#t1|1,5.}.} and in another {.set|#t2|{.add|#t1|1.5.}.}
So it seems to me, the only solution is to make a full check if a varible is a number:
If first caracter is a digit, repeat for each of the following caracters if its a digit or a decimal separator ,(comma) OR .(point).
If a decimal separator is found, set decimal POINT and a flag and continue until a non digit value or end of string is found.
So: 1,234 -> 1.234
1.234,547 -> 1.234
1,23 GB -> 1.23
Then calculations are done and the result can be output as now, using local settings.
-
HFS sets comma as separator at start, ignoring local settings.
so, that's strange.
anway, i applied a workaround for next release.
let me know if it's enough.