rejetto forum
Software => HFS ~ HTTP File Server => Bug reports => Topic started by: LeoNeeson on March 13, 2016, 09:42:24 AM
-
I think I've found a bug, since HFS says 'uploading a MD5 file is forbidden'. After doing an extensive search on this forum about "MD5" implementation on HFS, just to be sure this wasn't' posted before, I think there is a bug on HFS that prevents MD5 files to be uploaded...
> How this happened?...
The other day I was uploading a bunch of files to my server, and it was unable to upload a MD5 file. This doesn't have anything to do with the 'fingerprints' feature of HFS, since I have that option disabled (or at least it should not interfere with it). I've tried renaming the .md5 file to .txt, and HFS uploaded the file successfully. But having the .md5 extension, give the following error: "File name or extension forbidden.". This doesn't happen with any other checksum files (like .sha1, for example).
> How to reproduce the problem?...
1) Enable the 'Upload' feature to some real folder.
2) Using any browser (using the web interface), try to upload a ".md5" file to the server.
3) Bang! The file cannot be uploaded...
Here is a log...
00:28:13 192.168.0.101:1760 Requested GET /MyFolder/
00:28:13 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:15 192.168.0.101:1761 Requested GET /MyFolder/New/
00:28:16 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:29 192.168.0.101:1761 Upload failed for Test.md5: File name or extension forbidden.
00:28:29 192.168.0.101:1761 Upload failed Test.md5
00:28:29 192.168.0.101:1760 Requested POST /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /?mode=jquery
00:32:51 192.168.0.101:1770 Uploading Test.txt
00:32:51 192.168.0.101:1770 Fully uploaded Test.txt - 44 @ 0B/s
00:32:51 192.168.0.101:1770 Requested POST /MyFolder/New/
Here is an screenshot (cropped)...
(http://i.imgur.com/OYvJdu2.png?1)
I'm almost sure this bug/error has to be related to the 'fingerprints' feature. I can provide more details if you need them. To me, uploading .md5 files is important.
> EDIT: The "solution" for this, it's here (http://www.rejetto.com/forum/bug-reports/uploading-a-md5-file-is-forbidden/msg1061109/#msg1061109). Thank you Rejetto.
-
to test:
upload a file text.txt
upload the the md5 named as textmd5.txt
rename the file as text.txt.md5 to see if it's possible
-
to test:
upload a file text.txt
upload the the md5 named as textmd5.txt
rename the file as text.txt.md5 to see if it's possible
I did this, and it works, but that's not the way it should work. ::)
The last step, when you say "rename the file as text.txt.md5", I did that on the server (with the Windows Explorer), not through Firefox/Chrome (since I'm not using a custom template with rename option, and the default template doesn't have any rename option). I bet if I use a custom template (with the rename function), it will work fine, since the problem is only when you upload the file (for example, if I already have a md5 file on the server, I can download it normally).
Programmatically speaking, HFS checks the file extension when you upload the file, and if it's a MD5, it rejects the file. But if I have the 'fingerprints' feature disabled, it should allow uploading .md5 files normally (check the file "main.pas" in the source code)
These are the references, on the "main.pas" file, about "MD5"...
begin
result:=validFilename(data.uploadSrc)
and not sameText(data.uploadSrc, DIFF_TPL_FILE) // never allow this
and fileMatch(getMask(), data.uploadSrc);
if not result then
data.uploadFailed:='File name or extension forbidden.';
end; // complyUploadFilter
PROTECTED_FILES_MASK = 'hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted';
procedure TmainFrm.saveNewFingerprintsChkClick(Sender: TObject);
const
MSG = 'This option creates an .md5 file for every new calculated fingerprint.'
+#13'Use with care to get not your disk invaded by these files.';
begin
if saveNewFingerprintsChk.Checked then
msgDlg(MSG, MB_ICONWARNING);
end;
Either way, if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally (out-of-the-box, I mean, without any further configuration from the end-user). That's the way it should work, don't you think?... (http://www.rejetto.com/forum/Themes/default/images/post/question.gif)
-
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,
it will be possible to upload a md5 or rename a file as such that only if the calculation of the internal md5 is validated
-
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,
@Mars: If you read my post, I've said I have the 'fingerprints' feature disabled, so, this file should be treated by HFS like any other file. If the internal MD5 feature of HSF is off, it should allow normal upload of MD5 files. English is not my native language, but it's not so hard to understand what I'm saying, IMHO... ::)
OK, I'll translate in french, just for you, lol: :P
"Si vous avez lu mon poste, je me suis dit que je dois la fonctionnalité 'empreintes digitales' désactivé, donc, ce fichier doit être traité par HFS comme tout autre fichier. Si la fonction MD5 interne du HSF est éteint, il devrait permettre le charger normal des fichiers MD5."
@rejetto: you are the main developer, what do you think about this?... will you fix it in next version?... It's easy to fix: if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally, like any other file. Thank you. :)
-
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted
You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted
you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these
-
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted
You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted
you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these
@Rejetto: Great!, thank you. It's working as expected now. :D
Two questions:
A) Is there any setting in "hfs.ini" to make this configuration permanent, to all new folders I share? (I know I can configure several folders at once, but it would be great to have a default setting for uploads in "hfs.ini", or at least allow normal upload of md5, if 'fingerprints' feature is disabled).
B) Just wondering: Is there any technical reason to forbid uploading md5 files by default, even if the 'fingerprints' feature is disabled? (I mean, if the 'fingerprints' feature disabled, it should allow uploading .md5 files normally, I guess). That's why I thought this was a bug.
@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)
-
@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)
:'( I almost died .... laughing ;D ;D
-
i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.
-
MD5 files are as are the individual COMMENTS files: hidden files, this is why it is not possible to upload them.
when a uploaded file is not conform to the source file, the md5 cannot to match the one you would upload
For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match
-
:'( I almost died .... laughing ;D ;D
LOL! ;D
For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match
I agree. It's better for the end-user to have the md5 file generated by HFS. In my case, I can bypass this using the "custom upload mask", as Rejetto suggested.
i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.
Don't worry, it's OK. Using the "custom upload mask" on each folder, works fine, so, it's better to leave all it as it is. Sorry for all the mess. At least for me, problem resolved. Thank you.
-
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.
Besides, MD5 is broken for a long time. HFS should change to SHA512.
-
it may take time but leave the door open to allow this.
If necessary it is possible to put a verification procedure when renaming or uploading a file md5 comparing it with the signature from hfs.
In all cases, it will be the internal signature that would be saved as valid information
-
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.
I never thought in that "evil" possibility (of uploading of fake MD5 files). Now I understand why it's better to leave it, the way it is. Anyway, when someone uploads a file, and even if the MD5 file is generated by HFS, that doesn't guarantee the file is clean (A checksum doesn't replace an antivirus and the common sense).
In my own particular case, I was needing to upload a MD5 file (along to a big ISO file) to check if the file was uploaded OK (when I were back at home). And since it's a 100% private server (for my own use only), uploading fake MD5 files doesn't apply here. And since I already had the MD5 file of that ISO, it was faster uploading the MD5 (than enabling the internal 'fingerprints' feature of HFS and letting the server calculate the checksum).
I've enabled the fingerprints feature today, just for testing. It would be great if HFS only generates MD5 files only when a new file is uploaded, and not all the files I already have on my server (I will see how this work, when I have more time).
For me (at least for personal & private use), MD5 is more than enough. Anyway, I'm open to new checksum systems, like SHA512.
-
i think that when ftp would come in to effect to add a md5 file
-
I've enabled the fingerprints feature today, just for testing. It would be great if HFS only generates MD5 files only when a new file is uploaded, and not all the files I already have on my server (I will see how this work, when I have more time).
For me (at least for personal & private use), MD5 is more than enough. Anyway, I'm open to new checksum systems, like SHA512.
Have you look at menu>fingerprints> Create fingerprint on addition under (uploaded size in KB) to create md5 for new files only
MD5 information is only visible when account and password are added to url based on this model
account:password@http://localhost/folder/file.txt#MD5
it would be more appropriate to create a MD5 or SHA512 by using macros since diff template only at each loading when it is finished
-
Have you look at menu>fingerprints> Create fingerprint on addition under (uploaded size in KB) to create md5 for new files only
MD5 information is only visible when account and password are added to url based on this model
account:password@http://localhost/folder/file.txt#MD5
it would be more appropriate to create a MD5 or SHA512 by using macros since diff template only at each loading when it is finished
I don't get it working. I'm not getting the #MD5 part (marked in red) included in the URL, and neither the .md5 file (only "hfs.comments.txt" is generated, besides the uploaded file).
For example, I get this URL:
http://TestUser:TestPass@localhost/FileTest.pdf
I'm not getting this (with the #MD5 part):
http://TestUser:TestPass@localhost/FileTest.pdf#12345678901234567890123456789012
I've enabled all the needed options. Have a look...
(http://i.imgur.com/yvmrCyI.png)
I never tried this option before. May be I need to configure something else... :-[
-
if your example FileTest.pdf is at root of VFS
go to disk and add a file new named FileTest.pdf.md5,
edit it as text and put only this next tex, and save
-LES_LAPINS_AIMENT_LES_CAROTTES-
save the file and browse the root folder -->move your mouse on FileTest.pdf and you will see on statusbar thomething as
http://localhost/FileTest.pdf#!md5!-LES_LAPINS_AIMENT_LES_CAROTTES-
;D ;D
Without using hfs options, you can easily create the md5 only for new uploaded files with size >0
[upload-success]
{.inc|ok.}
{.comment|Reset md5 var.}
{.set|md5.}
<li> <a href="%item-url%">%item-name%</a>
<br />%item-size% @ %smart-speed%B/s
{.if|{.>|%item-size%|0.}|{:
{.comment|Calculate md5.}
{.set|md5|{.md5 file|%item-resource%.}.}
{.comment|Create a new md5 file.}
{.save|%item-resource%.md5|{.^md5.}.}
{.comment|View md5 on Upload Result.}
<br />MD5 : {.^md5.}
:}.}
{.if| {.length|%user%.} |{:
{.comment|.append|%folder-resource%\hfs.comments.txt |{.filename|%item-resource%.}=uploaded by %user%
/append.}
{.set item|%item-url%|comment=uploaded by %user%.}
:}/if.}
by replacing previously {.md5 file|%item-resource%.} with an exec macro , it's possible to generate another type of Hash
Create md5, after each download is finished, might take a long time, the ideal would be to calculate during the transfer phase as and when the data arrives
in both cases the total transfer time would increased but what would be the fastest, it remains to be seen.
-
if your example FileTest.pdf is at root of VFS
go to disk and add a file new named FileTest.pdf.md5,
edit it as text and put only this next tex, and save
-LES_LAPINS_AIMENT_LES_CAROTTES-
save the file and browse the root folder -->move your mouse on FileTest.pdf and you will see on statusbar thomething as
http://localhost/FileTest.pdf#!md5!-LES_LAPINS_AIMENT_LES_CAROTTES-
;D ;D
LOL :P
Yes, it shows the MD5 in the URL (FileTest.pdf#!md5!-xxxxxxxxxxxxxxxxxxx) if I manually copy "FileTest.pdf" and "FileTest.pdf.md5" into a shared folder. But if I upload the file "FileTest.pdf" alone, the "FileTest.pdf.md5" file is still not being generated by HFS.
I will try this on a few days...
-
@Mars: (http://www.rejetto.com/forum/profile/?u=2913) Finally, I've got the time to test your change in the template code.
I've changed the code a little bit (marked in red) to generate a "normal" MD5 file (compatible with any external MD5 reader, like HashCheck v2.1.11 (http://code.kliu.org/hashcheck/))
Here is...
[upload-success]
{.inc|ok.}
{.comment|Reset md5 var.}
{.set|md5.}
<li> <a href="%item-url%">%item-name%</a>
<br />%item-size% @ %smart-speed%B/s
{.if|{.>|%item-size%|0.}|{:
{.comment|Calculate md5.}
{.set|md5|{.md5 file|%item-resource%.}.}
{.comment|Create a new md5 file.}
{.save|%item-resource%.md5|{.^md5.} *%item-name%.}
{.comment|View md5 on Upload Result.}
<br />MD5 : {.^md5.}
:}.}
{.if| {.length|%user%.} |{: {.comment|.append|%folder-resource%\hfs.comments.txt |{.filename|%item-resource%.}=uploaded by %user%
/append.}
{.set item|%item-url%|comment=uploaded by %user%.}
:}/if.}
Everything seems to work fine, but I have a problem in the generated MD5 file. Imagine I have a file called: "Rock & Roll.zip" (or any file containing the "&" character). Using the code above (with the change marked in red), that generates an invalid MD5 file (I mean, a md5 file which can't be read by an external MD5 reader, like HashCheck v2.1.11). The problem is the "&" character in the filename. I've tried several options:
{.save|%item-resource%.md5|{.^md5.} *%item-name%.}
That generates the following MD5 File (it contains "&" which makes the problem):
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF *Rock & Roll.zip
{.save|%item-resource%.md5|{.^md5.} *%item-resource%.}
That generates the following MD5 File (the file name now is OK, but an MD5 must not have the full path):
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF *C:\SharedFolder\Music\Rock & Roll.zip
{.save|%item-resource%.md5|{.^md5.} *%filename%.extension.}
That generates the following MD5 File (non valid file name):
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF *%filename%.extension
In my example, a correct MD5 file should look like this:
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF *Rock & Roll.zip
What I need to change to make it work?...
-
{.save|%item-resource%.md5|{.^md5.} *%item-name%.}
That generates the following MD5 File (it contains "&" which makes the problem):
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF *Rock & Roll.zip
it is strange that the "&" character is replaced with its equivalent web, but if there is only him, it is simple to use macro "replace"
{.save|%item-resource%.md5|{.^md5.} {.replace |&|&|*%item-name%.} /save.}
do not hesitate to use HFS log to evaluate expressions that are trying to use as "%filename%.extension", either using the macro {.add to log | ....}, either enabling debug macros to html
-
@Mars: Thank you. :) Thanks to your "replace" macro trick, I've removed the "full path" and used %item-resource% instead of %item-name%. Now works beautifully with any file I throw in (even works with another unicode symbols I've tested). This is the final result (with some cosmetic changes marked in green):
[upload-success]
{.inc|ok.}
{.comment|Reset md5 var.}
{.set|md5.}
<li> <a href="%item-url%">%item-name%</a>
<br /><b>Size & Speed:</b> %item-size% @ %smart-speed%B/s
{.if|{.>|%item-size%|0.}|{:
{.comment|Calculate md5.}
{.set|md5|{.md5 file|%item-resource%.}.}
{.comment|Create a new md5 file.}
{.save|%item-resource%.md5|{.^md5.} {.replace |%folder-resource%\|*|%item-resource%.} /save.}
{.comment|View md5 on Upload Result.}
<br /><b>MD5 Checksum:</b> {.^md5.}
:}.}
{.if| {.length|%user%.} |{: {.comment|.append|%folder-resource%\hfs.comments.txt |{.filename|%item-resource%.}=uploaded by %user%
/append.}
{.set item|%item-url%|comment=uploaded by %user%.}
:}/if.}
-
Svitch svitch bitsh, battle against gods 8) woff woff woff