i think the metadata (if not simply the file size) is not really necessary, because VLC is indeed doing the job correctly with HFS. I just made a test with an MKV to be sure, but i made use of it several times.
Extra information is probably necessary for "second perfect" seeking in case of VBR encoding. Without this, seeking is still possible nonetheless, and the error won't be huge.
Anyway, HFS doesn't support any extra information specific for videos, despite it is probably not impossible for an advanced user to create a solution, as HFS is scriptable.