Despite the fact that I have added this during my .htaccess file:

Header add Access-Control-Allow-Origin "*"

Which during my apache config: (In the section AND outdoors)

<Files ~ "\.(gif|jpe?g|png|js|json|xml)$">
        Header set Access-Control-Allow-Origin "*"

Still it doesn't work!

After I obtain the DIRECTORY it will produce the wanted header, but the moment I download a particular file the header is nowhere found!

How hard will it be?