rejetto forum

[SOLVED] Uploading a MD5 file is forbidden?...

LeoNeeson · 23 · 19811

0 Members and 1 Guest are viewing this topic.

Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
I think I've found a bug, since HFS says 'uploading a MD5 file is forbidden'. After doing an extensive search on this forum about "MD5" implementation on HFS, just to be sure this wasn't' posted before, I think there is a bug on HFS that prevents MD5 files to be uploaded...

> How this happened?...
The other day I was uploading a bunch of files to my server, and it was unable to upload a MD5 file. This doesn't have anything to do with the 'fingerprints' feature of HFS, since I have that option disabled (or at least it should not interfere with it). I've tried renaming the .md5 file to .txt, and HFS uploaded the file successfully. But having the .md5 extension, give the following error: "File name or extension forbidden.". This doesn't happen with any other checksum files (like .sha1, for example).

> How to reproduce the problem?...
1) Enable the 'Upload' feature to some real folder.
2) Using any browser (using the web interface), try to upload a ".md5" file to the server.
3) Bang! The file cannot be uploaded...

Here is a log...
Code: [Select]
00:28:13 192.168.0.101:1760 Requested GET /MyFolder/
00:28:13 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:15 192.168.0.101:1761 Requested GET /MyFolder/New/
00:28:16 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:29 192.168.0.101:1761 Upload failed for Test.md5: File name or extension forbidden.
00:28:29 192.168.0.101:1761 Upload failed Test.md5
00:28:29 192.168.0.101:1760 Requested POST /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /?mode=jquery
00:32:51 192.168.0.101:1770 Uploading Test.txt
00:32:51 192.168.0.101:1770 Fully uploaded Test.txt - 44 @ 0B/s
00:32:51 192.168.0.101:1770 Requested POST /MyFolder/New/

Here is an screenshot (cropped)...


I'm almost sure this bug/error has to be related to the 'fingerprints' feature. I can provide more details if you need them. To me, uploading .md5 files is important.

> EDIT: The "solution" for this, it's here. Thank you Rejetto.
« Last Edit: March 18, 2016, 05:28:08 AM by LeoNeeson »
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline Mars

  • Operator
  • Tireless poster
  • *****
    • Posts: 2059
    • View Profile
to test:

upload a file text.txt

upload the the md5 named as textmd5.txt

rename the file as text.txt.md5 to see if it's possible


Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
to test:

upload a file text.txt

upload the the md5 named as textmd5.txt

rename the file as text.txt.md5 to see if it's possible
I did this, and it works, but that's not the way it should work. ::)

The last step, when you say "rename the file as text.txt.md5", I did that on the server (with the Windows Explorer), not through Firefox/Chrome (since I'm not using a custom template with rename option, and the default template doesn't have any rename option). I bet if I use a custom template (with the rename function), it will work fine, since the problem is only when you upload the file (for example, if I already have a md5 file on the server, I can download it normally).

Programmatically speaking, HFS checks the file extension when you upload the file, and if it's a MD5, it rejects the file. But if I have the 'fingerprints' feature disabled, it should allow uploading .md5 files normally (check the file "main.pas" in the source code)

These are the references, on the "main.pas" file, about "MD5"...
Code: [Select]
  begin
  result:=validFilename(data.uploadSrc)
    and not sameText(data.uploadSrc, DIFF_TPL_FILE) // never allow this
    and fileMatch(getMask(), data.uploadSrc);
  if not result then
    data.uploadFailed:='File name or extension forbidden.';
  end; // complyUploadFilter

Code: [Select]
  PROTECTED_FILES_MASK = 'hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted';
Code: [Select]
procedure TmainFrm.saveNewFingerprintsChkClick(Sender: TObject);
const
  MSG = 'This option creates an .md5 file for every new calculated fingerprint.'
    +#13'Use with care to get not your disk invaded by these files.';
begin
if saveNewFingerprintsChk.Checked then
  msgDlg(MSG, MB_ICONWARNING);
end;

Either way, if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally (out-of-the-box, I mean, without any further configuration from the end-user). That's the way it should work, don't you think?...
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline Mars

  • Operator
  • Tireless poster
  • *****
    • Posts: 2059
    • View Profile
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,

it will be possible to upload a md5 or rename a file as such that only if the calculation of the internal md5 is validated
« Last Edit: March 15, 2016, 12:37:34 AM by Mars »


Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,
@Mars: If you read my post, I've said I have the 'fingerprints' feature disabled, so, this file should be treated by HFS like any other file. If the internal MD5 feature of HSF is off, it should allow normal upload of MD5 files. English is not my native language, but it's not so hard to understand what I'm saying, IMHO... ::)

OK, I'll translate in french, just for you, lol: :P
"Si vous avez lu mon poste, je me suis dit que je dois la fonctionnalité 'empreintes digitales' désactivé, donc, ce fichier doit être traité par HFS comme tout autre fichier. Si la fonction MD5 interne du HSF est éteint, il devrait permettre le charger normal des fichiers MD5."




@rejetto: you are the main developer, what do you think about this?... will you fix it in next version?... It's easy to fix: if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally, like any other file. Thank you. :)
« Last Edit: March 15, 2016, 03:50:04 AM by LeoNeeson »
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted

You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted

you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these


Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted

You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted

you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these
@Rejetto: Great!, thank you. It's working as expected now. :D

Two questions:
A) Is there any setting in "hfs.ini" to make this configuration permanent, to all new folders I share? (I know I can configure several folders at once, but it would be great to have a default setting for uploads in "hfs.ini", or at least allow normal upload of md5, if 'fingerprints' feature is disabled).

B) Just wondering: Is there any technical reason to forbid uploading md5 files by default, even if the 'fingerprints' feature is disabled? (I mean, if the 'fingerprints' feature disabled, it should allow uploading .md5 files normally, I guess). That's why I thought this was a bug.

@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)
« Last Edit: March 17, 2016, 07:06:13 AM by LeoNeeson »
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline Mars

  • Operator
  • Tireless poster
  • *****
    • Posts: 2059
    • View Profile

@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)

 :'(  I almost died .... laughing   ;D ;D


Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13510
    • View Profile
i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.


Offline Mars

  • Operator
  • Tireless poster
  • *****
    • Posts: 2059
    • View Profile
MD5 files are as are the individual COMMENTS files: hidden files, this is why it is not possible to upload them.

when a uploaded file is not conform to the source file, the md5 cannot  to match the one you would upload

For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match


Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
:'(  I almost died .... laughing   ;D ;D
LOL!  ;D

For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match
I agree. It's better for the end-user to have the md5 file generated by HFS. In my case, I can bypass this using the "custom upload mask", as Rejetto suggested.

i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.
Don't worry, it's OK. Using the "custom upload mask" on each folder, works fine, so, it's better to leave all it as it is. Sorry for all the mess. At least for me, problem resolved. Thank you.
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline MarkV

  • Tireless poster
  • ****
    • Posts: 764
    • View Profile
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.

Besides, MD5 is broken for a long time. HFS should change to SHA512.
http://worldipv6launch.org - The world is different now.


Offline Mars

  • Operator
  • Tireless poster
  • *****
    • Posts: 2059
    • View Profile
it may take time but leave the door open to allow this.

If necessary it is possible to put a verification procedure when renaming or uploading a file md5 comparing it with the signature from hfs.
 In all cases, it will be the internal signature that would be saved as valid information



Offline LeoNeeson

  • Tireless poster
  • ****
    • Posts: 842
  • Status: On hiatus (sporadically here)
    • View Profile
    • twitter.com/LeoNeeson
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.
I never thought in that "evil" possibility (of uploading of fake MD5 files). Now I understand why it's better to leave it, the way it is. Anyway, when someone uploads a file, and even if the MD5 file is generated by HFS, that doesn't guarantee the file is clean (A checksum doesn't replace an antivirus and the common sense).

In my own particular case, I was needing to upload a MD5 file (along to a big ISO file) to check if the file was uploaded OK (when I were back at home). And since it's a 100% private server (for my own use only), uploading fake MD5 files doesn't apply here. And since I already had the MD5 file of that ISO, it was faster uploading the MD5 (than enabling the internal 'fingerprints' feature of HFS and letting the server calculate the checksum).

I've enabled the fingerprints feature today, just for testing. It would be great if HFS only generates MD5 files only when a new file is uploaded, and not all the files I already have on my server (I will see how this work, when I have more time).

For me (at least for personal & private use), MD5 is more than enough. Anyway, I'm open to new checksum systems, like SHA512.
HFS in Spanish (HFS en Español) / How to compile HFS (Tutorial)
» Currently taking a break, until HFS v2.4 get his stable version.


Offline bmartino1

  • Tireless poster
  • ****
    • Posts: 910
  • I'm only trying to help i mean no offense.
    • View Profile
    • My HFS Google Drive Shared Link
i think that when ftp would come in to effect to add a md5 file
Files I have snagged and share can be found on my google drive:

https://drive.google.com/drive/folders/1qb4INX2pzsjmMT06YEIQk9Nv5jMu33tC?usp=sharing