Messages

1

HFS ~ HTTP File Server / Redux: .tar doesn't support files of size above 8gb

« on: August 09, 2021, 03:07:30 PM »

Re: http://rejetto.com/forum/index.php?topic=11682.0

I am seeing this issue as well in hfs2-2.4-rc07. Did the fix posted in the above forum post reviewed/actioned into a fix?

Currently, I can download a 8+GB file successfully....but I have a use case where I need to download a folder w/ lots of tiny files and 1 or 2 large files (that could be <20GB).

Thank you. I appreciate any help or pointers re: this matter.

2

HFS ~ HTTP File Server / Re: passing utf-8 characters in hfs.tpl function

« on: March 26, 2021, 08:48:28 PM »

I compiled 2.4.0rc07 and I can confirm that w/ the below change, I'm seeing utf-8 values in my arguments inside the Powershell script.

2.4.0 rc07 src:
utillib.pas Line 1238:

Code: [Select]

result:=(cmd > '') and (32 < shellexecuteW(0, 'open', pwidechar(cmd), pwidechar(pars), NIL, showCmd)

3

Programmers corner / Re: [Tutorial] How to compile the HFS's Source code

« on: March 26, 2021, 05:06:35 PM »

@LeoNeeson. Thanks for those pointers.

I was able to figure out how to build HFS 2.4.0 rc07.
But it reqd a few lines of code changes. Not sure if its the right changes.

Specifically:
In utilLib.pas:  Line 363-365:
RAD IDE's drop-down helper suggested 2 possibilities: MountPointReparseBuffer or SymbolicLinkReparseBuffer. I chose the former.

Code: [Select]

      if BytesReturned < DWORD(ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength + SizeOf(WideChar)) then exit;
      SetLength(Destination, (ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength div SizeOf(WideChar)));
      Move(ReparseData.Reparse.MountPointReparseBuffer.PathBuffer[0], Destination[1],
     ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength);

Anyway, here's my full Readme.txt that got me up and running w/ 2.4.0 rc07


README for building 2.4.0 rc07

Code: [Select]

Assumption: Installed everything to E:\Portable\TDLite.
If you move it elsewhere, modify include paths in RAD (Step 4a) appropriately

1. Install Embarcadero 10.3.3 RAD IDE Community Edition to E:\Portable\TDLite
https://www.embarcadero.com/products/delphi/starter


1. Install HFS Source
   https://github.com/rejetto/hfs2#libs-used
   2.4.0rc07: https://github.com/rejetto/hfs2/releases/tag/v2.4-rc07
   https://github.com/rejetto/hfs2/archive/refs/tags/v2.4-rc07.zip
   Extract to: E:\Portable\TDLite\Projects\hfs2-2.4-rc07

   Build HFS Data resources:
   Execute: E:\Portable\TDLite\Projects\hfs2-2.4-rc07\recompile_data.bat

3. Setting up HFS Library pack
HFS incl. all all its reqd 3rdparty libs will be installed under E:\Portable\Projects\hfs-lib

Ref: https://github.com/rejetto/hfs2#libs-used
a. Overbyte 8.64: http://www.overbyte.eu/frame_index.html
   Daily archive: http://wiki.overbyte.eu/arch/icsv8w.zip
   Extract to: E:\Portable\Projects\hfs-lib\OverbyteIcsV8_64
b. TRegExpr 0.952b: https://github.com/andgineer/TRegExpr/releases
   https://github.com/andgineer/TRegExpr/archive/refs/tags/0.952b.zip
   Extract to: E:\Portable\TDLite\Projects\hfs-lib\TRegExpr0_952b
c. Jedi CL 2.7: https://github.com/project-jedi/jcl
   Install this git repo to a temp folder somewhere.
   git clone --recurse-submodules -j8 https://github.com/project-jedi/jcl.git
   cd jcl/jcl
   ./Install.bat
   copy *.* E:\Portable\TDLite\hfs-lib\JCL2_7
d. KDL 4.1: https://sites.google.com/site/kryvich/localizer
   kdl41.zip: https://drive.google.com/open?id=0BwH77FLlRxs2RnFRTnMxcnZNbUE
   Extract to E:\Portable\Projects\hfs-lib\kdl41

4. Building in Embarcadero RAD:
   Open E:\Portable\TDLite\Projects\hfs2-2.4-rc07\hfs.bdsproj

   a. Modify include paths:
      In Projects window, right-click hfs.exe and choose "Options"
      Click on "Delphi Compiler" > Search Path, and replace w/:
      $(DELPHI)\Lib\win32\release;$(DELPHI)\Projects\hfs-lib;$(DELPHI)\Imports;$(DELPHI)\Projects\hfs-lib\JCL2_7\source\windows;$(DELPHI)\Projects\hfs-lib\JCL2_7\source\common;$(DELPHI)\Projects\hfs-lib\JCL2_7\source\include;$(DELPHI)\Projects\hfs-lib\Kdl41;$(DELPHI)\Projects\hfs-lib\OverbyteIcsV8_64\Source;$(DELPHI)\Projects\hfs-lib\OverbyteIcsV8_64\Source\Include;$(DELPHI)\Projects\hfs-lib\TRegExpr0_952b\Source;$(DELPHI)\Projects\hfs2-2.4-rc07;
      Save and close.
     
   b. SourceModifications reqd to compile:
      utilLib.pas
      Line 363-365:
      if BytesReturned < DWORD(ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength + SizeOf(WideChar)) then exit;
      SetLength(Destination, (ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength div SizeOf(WideChar)));
      Move(ReparseData.Reparse.MountPointReparseBuffer.PathBuffer[0], Destination[1], ReparseData.Reparse.MountPointReparseBuffer.SubstituteNameLength);
   
   c. In Projects window, right-click hfs.exe and choose "Build"
 
DONE
   

4

Programmers corner / Re: [Tutorial] How to compile the HFS's Source code

« on: March 25, 2021, 03:24:05 PM »

Hi,
I'm trying to build 2.4.0 rc07 with Embarcadero 10.3.3 Community Edition.

1. Do I use the same Library pack as mentioned by LeoNeeson at the top of this thread? Or is there a new one somewhere?
2. Delphi fails to build with this error: "Cannot resolve unit name 'uFreeLocalizer' at line 28 of hfs.dpr" Got past this.
3. I now get an error in uFreeLocalizer.pas

Code: [Select]

WideToAnsiString(ws, s, GetACP);. Modified definition to

Code: [Select]

s: AnsiString; and it got past.
4. Next compile error in OverbytelcsMimeUtils.pas Undeclared identifier errors: TBufferedFileStream(Stream).Mode & TBufferedFileStream(Stream).FastSize

I'm playing whack-a-mole here so it kinda appears that there is a new Library Pack, I need to use.

Thanks

5

HFS ~ HTTP File Server / Re: passing utf-8 characters in hfs.tpl function

« on: March 25, 2021, 03:11:02 PM »

Looking at the 2.4.0 rc07 src:
utillib.pas Line 1238:

Code: [Select]

result:=(cmd > '') and (32 < shellexecute(0, 'open', pchar(cmd), pchar(pars), NIL, showCmd)
Should this be:

Code: [Select]

result:=(cmd > '') and (32 < shellexecuteW(0, 'open', pwidechar(cmd), pwidechar(pars), NIL, showCmd)

6

HFS ~ HTTP File Server / passing utf-8 characters in hfs.tpl function

« on: March 23, 2021, 05:18:36 PM »

Hi,
I created a simple function like below which exec's an external application.
It takes 2 utf-8 arguments. I've noticed that the args show up with ? chars for the utf-8 characters in the Powershell script.
I've tried changing the active code page in the PS1 script by adding chcp 65001 and Input/Output encoding to UTF-8, but makes no difference.

However, if I change the active code page for the machine by changing this registry entry and then RESTARTING windows..

Code: [Select]

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage\OEMCP 65001it sends utf-8 characters into the PS1 script.

Is there anyway to do this, without changing the active page of the machine & restarting?

Code: [Select]

[ajax.foo|public|no log]
{.check session.}
{.set|x|{.postvar|var1.}.}
{.set|y|{.postvar|var2.}.}
{.exec|powershell.exe -F foo.ps1 "{.^x.}" "{.^y.}"|out=#output.}
{.^#output.}


7

Programmers corner / Re: [Tutorial] How to compile the HFS's Source code

« on: March 09, 2021, 11:04:04 PM »

Hi,
I'm trying to compile the HFS 2.4.0 RC07 codebase with instructions from this thread, and I'm encountering the following errors.
[Pascal Fatal Error] hslib.pas(33): F1026 File not found: 'system.contnrs.dcu'
If I comment out the use of the system.contnrs lib I get other errors:
[Pascal Error] hslib.pas(209): E2003 Undeclared identifier: 'TobjectList'
[Pascal Error] hslib.pas(309): E2004 Identifier redeclared: 'chop'
[Pascal Error] hslib.pas(314): E2004 Identifier redeclared: 'chop'
[Pascal Fatal Error] hslib.pas(338): F1026 File not found: 'ansistrings.dcu'

Note that the instructions worked just fine for 2.3m, and the current build is attempted on the same machine.

Any suggestions would be appreciated.

Thanks

8

HFS ~ HTTP File Server / Re: Is utf-8 encoding supported? Chinese, Korean, Czech characters?

« on: March 04, 2021, 03:14:21 PM »

Curl cmds I've tried (and failed to get it to work)

curl.exe -v -u user:password "http://localhost:280/data/Blah/?mode=section&id=ajax.mkdir" --data "name=안녕22"

curl.exe -v -u user:password "http://localhost:280/data/Blah/?mode=section&id=ajax.mkdir" --data "name=안녕22" -H "Content-type: text/plain; charset=utf-8"

curl.exe -v -u user:password "http://localhost:280/data/Blah/?mode=section&id=ajax.mkdir" --data "name=안녕22" -H "Content-type: application/x-www-form-urlencoded; charset=utf-8"

curl.exe -v -u user:password "http://localhost:280/data/Blah/?mode=section&id=ajax.mkdir" --data-urlencode "name=안녕22" -H "Content-type: application/x-www-form-urlencoded; charset=utf-8"

I found that curl supports a trace option (--trace -) and I tried that

curl.exe -v -u user:password "http://localhost:280/data/Blah/?mode=section&id=ajax.mkdir" --data "name=안녕22" -H "Content-type: application/x-www-form-urlencoded; charset=utf-8" --trace -

And it does appear to be a curl issue. The data sent to HFS is bogus. See attached pic.

9

HFS ~ HTTP File Server / Re: Is utf-8 encoding supported? Chinese, Korean, Czech characters?

« on: March 04, 2021, 12:26:17 AM »

So creating a new folder from the webUI with Korean characters works just fine. And HFS even displays an existing folder w/ Korean chars.

However, when I try to create a folder using curl, it fails.
I tried:

curl.exe -v -u user:password "http://localhost:280/data/Battlespaces/?mode=section&id=ajax.mkdir" --data "name=안녕22"

curl.exe -v -u user:password "http://localhost:280/data/Battlespaces/?mode=section&id=ajax.mkdir" --data "name=안녕22" -H "Content-type: text/plain; charset=utf-8"

I printed out the value got from postvar|name in mkdir

[ajax.mkdir|no log]
{.check session.}
{.set|x|{.postvar|name.}.}
{.add to log|"Test:{.^x.}".}

For curl#1: The log prints "Test:??22"
For curl#2: The log prints "Test:"

I need similar support for curl for rename, deletion etc.

Does anyone have any suggestions?

Thank you in advance.

10

HFS ~ HTTP File Server / Re: Is utf-8 encoding supported? Chinese, Korean, Czech characters?

« on: March 03, 2021, 06:44:19 PM »

No....wait. I was wrong.
If I use my existing tpl HFS for some reason opened HFS with version:2.3m and it fails.
If I delete my tpl and restart HFS without it, the UI looks different and Korean characters are working.

So, this looks its working. I'll continue working w/ it, and come back w/ questions if I find anything odd.

Thanks all.

11

HFS ~ HTTP File Server / Re: Is utf-8 encoding supported? Chinese, Korean, Czech characters?

« on: March 03, 2021, 06:29:27 PM »

Thanks for your responses DJ and Danny.

- Tried hfs2.4beta (2.4.0 RC6). Didnt make a difference.
- Also tried adding that style to the tpl. It too didnt make a difference.

In both 2.3m and 2.4.0 RC6, if I use the webUI's NewFolder button and copy/paste enter "안녕" in there, the response is that it Failed.
The logs print out my print statement like my original comment, printing:
Test:??

12

HFS ~ HTTP File Server / Is utf-8 encoding supported? Chinese, Korean, Czech characters?

« on: March 02, 2021, 08:10:55 PM »

Hi,
I am trying to create/view files and folders with utf-8 encoding. For e.g:
if I send the following curl cmd mkdir fails:

curl.exe -v -u user:password "http://localhost:280/data/Battlespaces/?mode=section&id=ajax.mkdir" --data "name=안녕"

As per this https://www.rejetto.com/wiki/index.php/HFS:_Template
Unicode support requires the following line in the tpl. It does in mine.

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />

When I added a print-to-log line in the tpl in the mkdir script:

[ajax.mkdir|no log]
{.check session.}
{.set|x|{.postvar|name.}.}
{.add to log|"Test:{.^x.}".}
{.break|if={.pos|\|var=x.}{.pos|/|var=x.}|result=forbidden.}
{.break|if={.not|{.can mkdir.}.}|result=not authorized.}
{.set|x|{.force ansi|%folder%{.^x.}.}.}

It prints

Test:??

Also, doesnt the force ansi operation break utf-8 encoding?

Am I missing some other configuration setting?

Thanks

13

HFS ~ HTTP File Server / Re: HFS Scripting cmd: time and filetime

« on: December 03, 2019, 11:35:08 PM »

nm. I figured it out based on the comment here:
https://stackoverflow.com/questions/57184472/javascript-date-to-delphi-tdatetime

A Unix date/time is represented as the number of seconds since January 1 1970 00:00:00 UTC.
Delphi has a UnixDateDelta constant in the SysUtils unit which is defined as 25569, the number of days from December 31 1899 to January 1 1970.

So, using 25569, I was able to convert HFS time|YYYY:MM:DD:HH:NN:SS  to time_t, then do a difftime to get (double) seconds, divide by 86400 to get days, and then add 25569 to get the time in Delphi time.

But just saw your comment %now% works better. Thanks.

14

HFS ~ HTTP File Server / HFS Scripting cmd: time and filetime

« on: December 03, 2019, 11:10:31 PM »

I want to figure out the time difference between current server time and filetime of a file.
From https://www.rejetto.com/wiki/index.php/HFS:_scripting_commands
it looks like:
filetime returns a double value based on Delphi time ie days since 12/30/1899
&
time returns a value based on the optional argument (YY:MM:DD:HH:NN:SS) something like 2019:12:03:18:10:59

How can I get both times in one format, so that I can get the time since the file was created on the server.

15

HFS ~ HTTP File Server / Re: Deploy HFS configuration across various machines

« on: December 02, 2019, 09:39:12 PM »

I am using HFS 2.3m.

I am noticing however, that in some rare instances, HFS will start up as a service just fine, but the permissions will not be set. So AddFolder, Delete etc will not work.

And to reiterate, I am starting it as a service using a VFS that I am copying around which was created by:
- starting HFS on server1, then manually setting the permissions via the Properties page, then saved to a .vfs.
Now, this vfs is loaded across the various machines 1...n in HFS, where HFS is started as a service.