It is a server side macro, not client side.
You would have to redirect to a 'virtual page' that executes the file you want, then redirects you back... or use AJAX for it to happen without page changing. It is a bit complex to explain... you seem to know the basics so here goes nothing, this will be untested but just a suggestion for how to manually do it.
Send the file name and the page you are on to a 'virtual page' in the HFS template:
onclick="window.location('/application.html?location=%encoded-folder%');"
Now for the 'virtual page', this is where it happens:
[application = application.html]
{.comment| Execute the file before sending the page and redirecting. .}
{.exec|C:\prog.exe|out=#temp.}
{.comment| Show some information and redirect. .}
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Run Application</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta name="robots" content="noindex,nofollow"/>
<meta http-equiv="REFRESH" content="2;url={.?location.}">
<script type="text/javascript">
// You can use this if meta wont work!
// window.location = "{.?location.}"
</script>
</head>
<body>
<h3>- Application <i>"prog.exe"</i> was executed, redirecting back to location <i>"{.?location.}"</i>...</h3>
</body>
</html>