How to Correctly Package Nginx with Melange & Apko on Wolfi? (Permission Denied & Linter Errors)

General Discussion
,
1

Hello Nginx Community :waving_hand:t4:

I’m working on an advanced project to create a minimal, secure Nginx image, and I’ve hit a wall. I’m hoping to get some expert advice on how to solve this.

My goal is to move beyond standard multi-stage builds and adopt a more declarative, reproducible, and secure method using modern tooling:

  1. Build Environment: Use wolfi-base because of its minimal footprint
  2. Packaging Tool: Use Melange to build a custom Nginx package from source. This allows me to control every configure flag and file path.
  3. Final Image Tool: Use Apko to assemble a final image using only the packages I’ve defined, based on a minimal distroless/static-like base (chainguard/static).

The core idea is to separate the build logic (Melange YAML) from the final image assembly (Apko YAML), creating a highly optimized and secure final artifact.

The Problem:

I am facing two major roadblocks:

  1. Melange Linter/Path Conflicts: The Melange linter correctly complains about creating directories under /run (/run/nginx for the PID file), as this is a temporary filesystem that should be managed at runtime, not packaged. I’ve tried various workarounds, but I’m struggling to find the “correct” Melange way to handle PID files and other runtime directories.
  2. Runtime Permission Errors: Even when I manage to build the package, the final Apko image fails at runtime. Nginx starts, but immediately exits with (13: Permission denied) errors when trying to write to log files (/var/log/nginx/error.log) or create its PID file. This happens despite trying to set ownership with chown in the Melange pipeline.

I feel like I’m close, but I’m missing a key concept in how Melange and Apko are designed to handle runtime state, permissions, and temporary directories.

Below are my latest melange.yaml and apko.yaml files. I would be incredibly grateful for any guidance on how to fix the linter errors and ensure the final image has the correct permissions to run Nginx as a non-root user.

Here is my Melange configuration (nginx.melange.yaml):

package:
  name: nginx
  version: 1.27.0
  epoch: 0
  description: "nginx web server"
  copyright:
    - license: BSD-2-Clause
  dependencies:
    runtime:
      - ca-certificates

environment:
  contents:
    repositories:
      - https://packages.wolfi.dev/os
    keyring:
      - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    packages:
      - wolfi-base
      - build-base
      - openssl-dev
      - pcre2-dev
      - zlib-dev
      - linux-headers

pipeline:
  - uses: fetch
    with:
      uri: http://nginx.org/download/nginx-${{package.version}}.tar.gz
      expected-sha256: b7230e3cf87eaa2d4b0bc56aadc920a960c7873b9991a1b66ffcc08fc650129c
  - runs: |
      set -e
      tar xzf nginx-${{package.version}}.tar.gz
      cd nginx-${{package.version}}
      
      export CFLAGS="$CFLAGS -Wno-error=unterminated-string-initialization"
      
      # Configure paths to avoid writing to /var/lib/nginx
      ./configure \
        --prefix=/usr \
        --sbin-path=/usr/bin/nginx \
        --conf-path=/etc/nginx/nginx.conf \
        --error-log-path=/var/log/nginx/error.log \
        --http-log-path=/var/log/nginx/access.log \
        --pid-path=/run/nginx.pid \
        --lock-path=/run/nginx.lock \
        --user=nginx \
        --group=nginx \
        --with-http_ssl_module \
        --with-http_v2_module \
        --http-client-body-temp-path=/var/cache/nginx/client_temp \
        --http-proxy-temp-path=/var/cache/nginx/proxy_temp \
        --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
        --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
        --http-scgi-temp-path=/var/cache/nginx/scgi_temp
        
      make
      make install DESTDIR=${{targets.destdir}}

      # Attempt to fix the linter error by removing the /run subdir from the package
      rm -rf ${{targets.destdir}}/run

      # Create necessary runtime directories
      mkdir -p ${{targets.destdir}}/var/log/nginx
      mkdir -p ${{targets.destdir}}/var/cache/nginx

      # Set ownership for the nginx user (UID 101 on Wolfi)
      chown -R 101:101 ${{targets.destdir}}/var/cache/nginx
      chown -R 101:101 ${{targets.destdir}}/var/log/nginx

  - uses: strip

subpackages:
  - name: nginx-config
    pipeline:
      - runs: |
          mkdir -p ${{targets.subpkgdir}}/etc/nginx
          mkdir -p ${{targets.subpkgdir}}/var/log/nginx
          
          # Symlink logs to stdout/stderr for container logging
          ln -sf /dev/stdout ${{targets.subpkgdir}}/var/log/nginx/access.log
          ln -sf /dev/stderr ${{targets.subpkgdir}}/var/log/nginx/error.log
          
          cat > ${{targets.subpkgdir}}/etc/nginx/nginx.conf << 'EOF'
          user nginx;
          pid /run/nginx.pid;
          error_log /var/log/nginx/error.log warn;
          daemon off;

          events {
              worker_connections 1024;
          }

          http {
              include       mime.types;
              default_type  application/octet-stream;

              client_body_temp_path /var/cache/nginx/client_temp;
              proxy_temp_path       /var/cache/nginx/proxy_temp;

              server {
                  listen 8080;
                  
                  location / {
                      root /usr/share/nginx/html;
                      index index.html;
                  }
              }
          }
          EOF

And here is my Apko configuration (apko.yaml):

contents:
  repositories:
    - ./packages
    - https://packages.wolfi.dev/os
  keyring:
    - ./melange.rsa.pub
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
  packages:
    - nginx
    - nginx-config
    - ca-certificates-bundle
    - tzdata
    - openssl
    - pcre2
    - zlib

paths:
  # Explicitly create /run and grant ownership to the nginx user
  - path: /run
    type: directory
    uid: 101
    gid: 101
    permissions: 0o777
  
  - path: /var/log/nginx
    type: directory
    uid: 101
    gid: 101
    permissions: 0o755
    recursive: true

  - path: /var/cache/nginx
    type: directory
    uid: 101
    gid: 101
    permissions: 0o755
    recursive: true

entrypoint:
  command: /usr/bin/nginx

accounts:
  users:
    - username: nginx
      uid: 101
      gid: 101
  groups:
    - groupname: nginx
      gid: 101
  run-as: nginx

image
image1071×146 22.2 KB

Thank you for your time and expertise