LARGE folders

[ferl]

Hi. A few of my shared folders have MANY files.

One has about 2300 files, takes about 45 secs to load but works.
The other has about 4800 files. This one simply refuses to load. Web browser just keeps waiting for the page to load.

I've tried REAL and VIRTUAL folder options, no difference.  I was using 2.3m, just tried 2.4b4, no luck.

Any suggestions?

Excellent software. If I can get this issue solved, I will definitely be donating.

[bmartino1]

I would have you reconfigure hfs back to default(redownload and start over). Use real folders (verfy your file permission to everyone read access)

There have been other who reported simlar issue.
Normally came down to a different template was used(error in code), file permission issues, file path name too long, not using real folder, moving hfs and it vfs to a new location and not have the filenames and path as original setup, many other error that can be found on the forum...

I would have you first check at file permission of the file that at least give everyone read access to the files.

[dj]

you can try to limit the listing (server pagination)

[Mars]

certainly we can impose a limit on the number of files that will be displayed but whatever the filtering of the names used, the complete analysis of the physical support is still necessary, and only the elements that will correspond to the filter mask will be retained.

[rejetto]

welcome ferl, and sorry for the late reply.
Try to disable
Menu > virtual file system > use system icons

[rejetto]

i just made a test with my windows/system32 folder: 170 folders, 3040 files.
First load in 16 seconds. Second load in 10 seconds.
Then i enable system icons, reload 20 seconds, reload again 10 seconds.
This with an SSD.

[Mars]

trial, with a directory in recursive mode, containing 83000 elements of all types, satisfactory result after 59'13 ''

slower than a snail

[LeoNeeson]

Idea: If somehow we find the way to directly read the hard drive's Master File Table (MFT), HFS could serve a list of hundred of thousands of files in just a few seconds (it may sound dangerous, but it's safe if we use a WinAPI). This method is already used with sucess by programs, like: UltraSearch, WizFile, Everything, FSearch, etc.

> The next logical step is finding a Windows API to read the NTFS-MFT. The most notable source code example in Delphi, that makes use of the needed Windows API, is DiskBuddy [Original+Mirror] (info found here). There also another open source projects (and libraries), like: NTFS-Search (using native Win32 Api), SwiftSearch, an NTFS Parser Lib, FSearch, Defragger, and some C# code (perhaps the source code of those projects could help Rejetto in some way). It took me a long time to find all those links, so, I hope they are useful...

[dj]

show waiting time in (default) template (works, but better show not )
URL parameter ~files.lst is faster

Code: [Select]

[+]
<script>
var ref=window['folder-stats']||document.querySelector('header').lastChild
ref.textContent += ', '+((Date.now()-sessionStorage.start)/1000).toFixed()+'s'
window.onbeforeunload = function(){sessionStorage.start=Date.now(); ref.textContent='\u231B'}
</script>

update: show entertainment (a running clock) while waiting
update: cssonly

[Mars]

You have to take into account that it's not just about displaying a file list like the old DIR command in MSDOS.
For my previous test, a DIR whose result was put in a text file took only about twenty seconds to finalize.

each file is subject to a number of mandatory tests for display in the web page, additional data associated with it even if some are empty in the end.

All data associated with a file on the disk is stored in a temporary memory record, and this space is freed after each refresh of the web page. converting this data to web representation takes a few milliseconds, depending on the machines, but this repeated action countless times may look like an eternity.
The VFS keeps this data indefinitely, whereas for the real files and files, it amounts to constructing a new building each time to create spaces there, to store carefully what will be used, then all methodically to clear to have a new location for the following requests.

we can see this as an album of photos for which we must at each consultation by the web systematically recreate the thumbnails

[Mars]

show waiting time in (default) template or better not

Code: [Select]

[+]
<script>
window['folder-stats'].textContent += ', '+((Date.now()-sessionStorage.start)/1000).toFixed()+'s'
window.onbeforeunload = function(){sessionStorage.start=Date.now(); window['folder-stats'].textContent='\u231B'}
</script>


you can update template with this at choice:

[folder panel]
<div id='folder-path'>
   {.breadcrumbs|{:<a class='pure-button' href="%bread-url%"/> {.if|{.length|%bread-name%.}|/ %bread-name%|<i class='fa fa-home'></i>.}</a>:} .}
</div>
<div id='folder-stats'>
{.if|%number%|%number-folders% {.!folders.}, %number-files% {.!files.}, {.add bytes|%total-size%.}, .}   
 {.!Build time.}: %build-time%''
</div>
{.123 if 2| <div id='foldercomment' class="comment"><i class="fa fa-quote-left"></i>|{.commentNL|%folder-item-comment%.}|</div> .}


[list panel]
{.if not| %number% |{:
   <div id='nothing'>{.!{.if|{.length|{.?search.}.}|No results|No files.}.}</div>
:}|{:
   <div id='files' class="hideTs {.for each|z|mkdir|comment|move|rename|delete|{: {.if|{.can {.^z.}.}|can-{.^z.} .}:}.}">
   %list%
   </div>
:}.}
<div id="serverinfo">
   <a href="http://www.rejetto.com/hfs/"><i class="fa fa-clock-o"></i> {.!Build time.}: %build-time% <i class="fa fa-coffee"></i> {.!Uptime.}: %uptime%</a>
</div>

[/list]