Author Topic: [SOLVED] Uploading a MD5 file is forbidden?...  (Read 8827 times)

0 Members and 1 Guest are viewing this topic.

Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
[SOLVED] Uploading a MD5 file is forbidden?...
« on: March 13, 2016, 09:42:24 AM »
I think I've found a bug, since HFS says 'uploading a MD5 file is forbidden'. After doing an extensive search on this forum about "MD5" implementation on HFS, just to be sure this wasn't' posted before, I think there is a bug on HFS that prevents MD5 files to be uploaded...

> How this happened?...
The other day I was uploading a bunch of files to my server, and it was unable to upload a MD5 file. This doesn't have anything to do with the 'fingerprints' feature of HFS, since I have that option disabled (or at least it should not interfere with it). I've tried renaming the .md5 file to .txt, and HFS uploaded the file successfully. But having the .md5 extension, give the following error: "File name or extension forbidden.". This doesn't happen with any other checksum files (like .sha1, for example).

> How to reproduce the problem?...
1) Enable the 'Upload' feature to some real folder.
2) Using any browser (using the web interface), try to upload a ".md5" file to the server.
3) Bang! The file cannot be uploaded...

Here is a log...
Code: [Select]
00:28:13 192.168.0.101:1760 Requested GET /MyFolder/
00:28:13 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:15 192.168.0.101:1761 Requested GET /MyFolder/New/
00:28:16 192.168.0.101:1761 Requested GET /?mode=jquery
00:28:29 192.168.0.101:1761 Upload failed for Test.md5: File name or extension forbidden.
00:28:29 192.168.0.101:1761 Upload failed Test.md5
00:28:29 192.168.0.101:1760 Requested POST /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /MyFolder/New/
00:32:45 192.168.0.101:1760 Requested GET /?mode=jquery
00:32:51 192.168.0.101:1770 Uploading Test.txt
00:32:51 192.168.0.101:1770 Fully uploaded Test.txt - 44 @ 0B/s
00:32:51 192.168.0.101:1770 Requested POST /MyFolder/New/

Here is an screenshot (cropped)...


I'm almost sure this bug/error has to be related to the 'fingerprints' feature. I can provide more details if you need them. To me, uploading .md5 files is important.

> EDIT: The "solution" for this, it's here. Thank you Rejetto.
« Last Edit: March 18, 2016, 05:28:08 AM by LeoNeeson »
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline Mars

  • Operator
  • Insane poster
  • *****
  • Posts: 1830
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #1 on: March 13, 2016, 10:17:15 PM »
to test:

upload a file text.txt

upload the the md5 named as textmd5.txt

rename the file as text.txt.md5 to see if it's possible

Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
Re: Uploading a MD5 file is forbidden?...
« Reply #2 on: March 14, 2016, 07:24:58 AM »
to test:

upload a file text.txt

upload the the md5 named as textmd5.txt

rename the file as text.txt.md5 to see if it's possible
I did this, and it works, but that's not the way it should work. ::)

The last step, when you say "rename the file as text.txt.md5", I did that on the server (with the Windows Explorer), not through Firefox/Chrome (since I'm not using a custom template with rename option, and the default template doesn't have any rename option). I bet if I use a custom template (with the rename function), it will work fine, since the problem is only when you upload the file (for example, if I already have a md5 file on the server, I can download it normally).

Programmatically speaking, HFS checks the file extension when you upload the file, and if it's a MD5, it rejects the file. But if I have the 'fingerprints' feature disabled, it should allow uploading .md5 files normally (check the file "main.pas" in the source code)

These are the references, on the "main.pas" file, about "MD5"...
Code: [Select]
  begin
  result:=validFilename(data.uploadSrc)
    and not sameText(data.uploadSrc, DIFF_TPL_FILE) // never allow this
    and fileMatch(getMask(), data.uploadSrc);
  if not result then
    data.uploadFailed:='File name or extension forbidden.';
  end; // complyUploadFilter

Code: [Select]
  PROTECTED_FILES_MASK = 'hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted';
Code: [Select]
procedure TmainFrm.saveNewFingerprintsChkClick(Sender: TObject);
const
  MSG = 'This option creates an .md5 file for every new calculated fingerprint.'
    +#13'Use with care to get not your disk invaded by these files.';
begin
if saveNewFingerprintsChk.Checked then
  msgDlg(MSG, MB_ICONWARNING);
end;

Either way, if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally (out-of-the-box, I mean, without any further configuration from the end-user). That's the way it should work, don't you think?...
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline Mars

  • Operator
  • Insane poster
  • *****
  • Posts: 1830
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #3 on: March 14, 2016, 06:23:48 PM »
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,

it will be possible to upload a md5 or rename a file as such that only if the calculation of the internal md5 is validated
« Last Edit: March 15, 2016, 12:37:34 AM by Mars »

Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
Re: Uploading a MD5 file is forbidden?...
« Reply #4 on: March 15, 2016, 03:38:31 AM »
I think that the creation of the md5 file should be possible only from the server and not uploadable. if an attempt was still possible directly or by renaming, then missing a comparaison system to control the actual value of md5, with the file someone tries to impose,
@Mars: If you read my post, I've said I have the 'fingerprints' feature disabled, so, this file should be treated by HFS like any other file. If the internal MD5 feature of HSF is off, it should allow normal upload of MD5 files. English is not my native language, but it's not so hard to understand what I'm saying, IMHO... ::)

OK, I'll translate in french, just for you, lol: :P
"Si vous avez lu mon poste, je me suis dit que je dois la fonctionnalité 'empreintes digitales' désactivé, donc, ce fichier doit être traité par HFS comme tout autre fichier. Si la fonction MD5 interne du HSF est éteint, il devrait permettre le charger normal des fichiers MD5."




@rejetto: you are the main developer, what do you think about this?... will you fix it in next version?... It's easy to fix: if the 'fingerprints' feature is disabled, it should be possible to upload .md5 files normally, like any other file. Thank you. :)
« Last Edit: March 15, 2016, 03:50:04 AM by LeoNeeson »
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline rejetto

  • Administrator
  • Insane programmer
  • *
  • Posts: 12859
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #5 on: March 15, 2016, 10:10:55 AM »
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted

You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted

you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these

Follow members gave a thank to your post:


Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
Re: Uploading a MD5 file is forbidden?...
« Reply #6 on: March 17, 2016, 07:04:17 AM »
it's configurable.
By default these files are NOT allowed:
hfs.*;*.htm*;descript.ion;*.comment;*.md5;*.corrupted

You can right click on the upload folder, go to "upload mask" and enter a custom mask, like
\hfs.*;*.htm*;descript.ion;*.comment;*.corrupted

you see i removed the md5 part.
Please, mind the initial slash, it means: DON'T allow these
@Rejetto: Great!, thank you. It's working as expected now. :D

Two questions:
A) Is there any setting in "hfs.ini" to make this configuration permanent, to all new folders I share? (I know I can configure several folders at once, but it would be great to have a default setting for uploads in "hfs.ini", or at least allow normal upload of md5, if 'fingerprints' feature is disabled).

B) Just wondering: Is there any technical reason to forbid uploading md5 files by default, even if the 'fingerprints' feature is disabled? (I mean, if the 'fingerprints' feature disabled, it should allow uploading .md5 files normally, I guess). That's why I thought this was a bug.

@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)
« Last Edit: March 17, 2016, 07:06:13 AM by LeoNeeson »
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline Mars

  • Operator
  • Insane poster
  • *****
  • Posts: 1830
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #7 on: March 17, 2016, 04:45:01 PM »

@Mars: J'espère que vous n'êtes pas fâché contre moi, pour mon dernier commentaire... / I hope you're not upset with me, for my last comment... :)

 :'(  I almost died .... laughing   ;D ;D

Offline rejetto

  • Administrator
  • Insane programmer
  • *
  • Posts: 12859
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #8 on: March 17, 2016, 10:59:43 PM »
i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.

Offline Mars

  • Operator
  • Insane poster
  • *****
  • Posts: 1830
    • View Profile
Re: Uploading a MD5 file is forbidden?...
« Reply #9 on: March 18, 2016, 12:28:19 AM »
MD5 files are as are the individual COMMENTS files: hidden files, this is why it is not possible to upload them.

when a uploaded file is not conform to the source file, the md5 cannot  to match the one you would upload

For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match

Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
Re: Uploading a MD5 file is forbidden?...
« Reply #10 on: March 18, 2016, 05:23:47 AM »
:'(  I almost died .... laughing   ;D ;D
LOL!  ;D

For this reason, it is better to generate md5 from HFS rather than downloaded in one that may not match
I agree. It's better for the end-user to have the md5 file generated by HFS. In my case, I can bypass this using the "custom upload mask", as Rejetto suggested.

i'm not sure,
i cannot tell why uploading the md5 could be a problem.
I may have had a good reason that i don't remember, or maybe i just included md5 because is one of the file "handled" by HFS.
In the latter case i would just remove it. But it's not easy to know.
Don't worry, it's OK. Using the "custom upload mask" on each folder, works fine, so, it's better to leave all it as it is. Sorry for all the mess. At least for me, problem resolved. Thank you.
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline MarkV

  • Insane poster
  • *****
  • Posts: 745
    • View Profile
Re: [SOLVED] Uploading a MD5 file is forbidden?...
« Reply #11 on: March 18, 2016, 09:54:50 AM »
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.

Besides, MD5 is broken for a long time. HFS should change to SHA512.
http://worldipv6launch.org - The world is different now.

Offline Mars

  • Operator
  • Insane poster
  • *****
  • Posts: 1830
    • View Profile
Re: [SOLVED] Uploading a MD5 file is forbidden?...
« Reply #12 on: March 18, 2016, 05:26:00 PM »
it may take time but leave the door open to allow this.

If necessary it is possible to put a verification procedure when renaming or uploading a file md5 comparing it with the signature from hfs.
 In all cases, it will be the internal signature that would be saved as valid information


Offline LeoNeeson

  • Insane poster
  • *****
  • Posts: 467
  • Solitario...
    • View Profile
    • twitter.com/LeoNeeson
Re: [SOLVED] Uploading a MD5 file is forbidden?...
« Reply #13 on: March 19, 2016, 10:23:24 AM »
I'm reasonably sure it's in order to not allow uploading of fake MD5 files that would confuse users and might even hide dangerous malware.
I never thought in that "evil" possibility (of uploading of fake MD5 files). Now I understand why it's better to leave it, the way it is. Anyway, when someone uploads a file, and even if the MD5 file is generated by HFS, that doesn't guarantee the file is clean (A checksum doesn't replace an antivirus and the common sense).

In my own particular case, I was needing to upload a MD5 file (along to a big ISO file) to check if the file was uploaded OK (when I were back at home). And since it's a 100% private server (for my own use only), uploading fake MD5 files doesn't apply here. And since I already had the MD5 file of that ISO, it was faster uploading the MD5 (than enabling the internal 'fingerprints' feature of HFS and letting the server calculate the checksum).

I've enabled the fingerprints feature today, just for testing. It would be great if HFS only generates MD5 files only when a new file is uploaded, and not all the files I already have on my server (I will see how this work, when I have more time).

For me (at least for personal & private use), MD5 is more than enough. Anyway, I'm open to new checksum systems, like SHA512.
• HFS ahora también disponible en Español! (Clic aqui) :)
• HFS is now also available in Spanish! (Click here)

Offline bmartino1

  • Insane poster
  • *****
  • Posts: 785
  • I'm only trying to help i mean no offense.
    • View Profile
    • none - google translate
Re: [SOLVED] Uploading a MD5 file is forbidden?...
« Reply #14 on: March 19, 2016, 02:32:52 PM »
i think that when ftp would come in to effect to add a md5 file
I'm only trying to help i mean no offense.
thank you for your time and patience,
Bmartino1