Njs and OTel, can they interact?

Projects NGINX
,
1

I am doing a bit of a proof of concept with replacing our current Kong setup with “raw” nginx and some customisations using njs.

One of the goals is to be able to have much finer grained Otel information.

so ive installed and enabled the njs and Otel modules

RUN apt-get update && \
    apt-get install -y \
    nginx \
    nginx-module-njs \
    nginx-module-otel

and this has given some nice Otel traces out for each request, but id really like to be able to get spans for upstream requests (when using proxy_pass) and also be able to append attributes or new whole spans from within custom njs scripts (which may very well do their own http requests etc)

I’m very new to the intricacies of nginx but from first glance this isnt a well trodden path, and possibly isn’t doable without some serious nginx hacking capabilities?

any advice or guidance, reference links or anything would be much appreciated.

Chris

1 Like
7

Chris,

This is a valuable use case your exploring, I myself as well as potentially others would like to see this progress. While I am no expert in this area, the team and I may know a few. :face_savoring_food:

Rather than firing a bunch of information your way that may or may not be useful, perhaps we can begin with a quick RE: post around what information you already have and as things progress, we can help track down information that is needed to help support your journey.

8

Thanks for the quick response @Dylen .

So far my research / experimentation journey has been a little disappointing. As far as I can tell there is no way to add sub-spans during the request lifecycle with the standard Otel plugin. Spans can be re-names and attributes added, but I can find a way of causing a new span to be created and linked to the parent.

Further to this, some comments suggest there is no specific hook which would ensure 100% of outgoing http request were “visible” to any code trying to instrument them (possibly not a huge deal for my own needs, but a semi useful datapoint)

Copilot (not assumed to be an automotive answer) says that there is no generic way to call directives from NJS scripting, and I cant find anything in the docs that suggests its possible so I assume this to be the case.

The Otel plugin does expose variables for trace & span ID’s so I guess a possible work around would be to send traces direct from “NJS land” with the appropriate parent_id etc, but that would then become a re-plimentation of the Otel plugin, and not befit from request batching etc

Another tacky option might be to use variables to pass information from NJS scripts back into pre configured directives which could then add attributes etc, however this is still limited in usefulness by the above “no way to generate child spans”

this is about as far as I have dug into things so far. My guess is that it would be possible to create some C++ that would link the Otel plugin and NJS scripting API, but I think thats going to be way beyond my own capabilities :slight_smile:

1 Like
9

Chris,

Nothing in the way of new information to offer however, I took a stab at capturing your high-level ideas as list. I also wanted to toss out an idea that may help with ideation below in the form of hypothetical functions that may be useful in the njs module.

At the very least I can confirm that your findings are correct. Good news is that this remains, uncharted territory ripe for a prototype.

Correct (to the best of my knowledge):

  • NGINX directives cannot be called generically from njs scripts. But can interact indirectly via variables (e.g., r.variables) and objects like ngx.

  • Adding functions for this would require C++ and updating the njs module (should this be deemed useful by the maintainers)

Hypothetical Functions:

  • ngx.otel.traceUpstream(): Traces upstream requests for performance monitoring.
  • ngx.otel.appendAttribute(key, value): Adds custom attributes to spans for detailed context.
  • ngx.otel.createSpan(name): Starts new spans to track specific script operations.
  • ngx.otel.instrumentHttp(): Instruments outgoing HTTP for full visibility in traces.
  • ngx.otel.sendTrace(parentId): Sends traces with parent linkage for correlated logging.
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.