in stock nginx, you can use log_format to write json safely into the access log; and you can include whatever list of (sensibly-)named request header fields that you want; but you cannot trivially do “all request headers”.
If you use one of the embedded languages, you can write code to do it – I expect that each of them will be roughly the same level of “clean”, because they will be doing fundamentally the same thing.
A modified version of the njs example at Logging request headers with njs · GitHub may or may not count as clean enough for you. (Read all the comments, for the likely modifications.)
Alternatively, it might be achievable by arranging for an external module to exist that does the thing that you want (assuming that one does not exist already. A quick search did not show any to me).