Web Security: Should CSP be set for my HTML files only, or my HTML files and all my assets?

Yes. "Some web framework automatically generate html on error pages and we found xss issues in those in the past, so setting CSP on everything is best." --ulfr from Moznet