I try to limit the amount of subfolders in one main folder to about 50-100 for faster loading of the page as all images will have to be opened. Also, don't limit the connections or all images won't open.
it would make HSF run very slower if I were to create Virtual Folders.
With the amount of folders/subfolders you say you have HFS
start-up time will be much slower if you created virtual folders. That's true. But, once HFS is loaded and running, you shouldn't notice a difference in speed.
One thing to keep in mind is that if you do decide to use virtual folders, you have to manually update any changes or additions you make to those folders. However, judging from what you say the content of those subfolders are, the contents won't change for a movie or music album.
Real folders on the other hand update any changes automatically. You might even consider a mix of virtual and real folders.
At least now you have another idea of what you can do to solve your problem. Good luck.