DevOps
Systemd Services for Background Jobs: The Right Way to Run Node.js in Production
December 202510 min read
DevOps
Systemd manages over 70% of Linux systems in production and is the standard init system for Ubuntu, Debian, CentOS, and every major Linux distribution. Yet most tutorials for running Node.js apps in production still recommend pm2 — a userspace process manager that duplicates functionality already built into systemd. At Commsult Indonesia, I migrated all our NestJS background workers from pm2 to native systemd services, gaining automatic boot startup, structured logging via journald, resource limit enforcement, and better integration with monitoring tools.
pm2 is excellent for development and prototyping but adds an unnecessary layer in production. Systemd handles: automatic startup at boot, automatic restart on crash with configurable backoff, CPU and memory limits enforced by the kernel, structured logs via journald accessible with journalctl, and integration with systemctl for status and control. The single advantage pm2 retains is cluster mode for multi-process Node.js — but for NestJS, this is better handled by running multiple Docker containers behind Nginx.
A systemd service file lives in /etc/systemd/system/your-service.service and has three sections: [Unit] describes the service and its dependencies (After=network.target ensures the network is up before starting), [Service] defines how to run the service, and [Install] determines when the service starts (WantedBy=multi-user.target is the production default that starts the service in normal multi-user mode). Restart=on-failure restarts on crashes but not on clean exits — the right behavior for production.
Never hardcode secrets in service files. Use EnvironmentFile=/etc/your-service/.env to load environment variables from a file with 640 permissions owned by root and your service user. This file is readable by the service user and root only, not world-readable like a .env file in the application directory. Alternatively, use Environment= directives for non-sensitive config: Environment=NODE_ENV=production. Combine both approaches: EnvironmentFile for secrets, Environment for configuration.
# /etc/systemd/system/nestjs-app.service
[Unit]
Description=NestJS Production App
Documentation=https://nestjs.com
After=network.target postgresql.service
Wants=postgresql.service
[Service]
Type=simple
User=nestjs
WorkingDirectory=/opt/nestjs/app
ExecStart=/usr/bin/node dist/main.js
EnvironmentFile=/etc/nestjs/.env
Restart=on-failure
RestartSec=10s
StartLimitIntervalSec=60
StartLimitBurst=3
# Resource limits
MemoryMax=512M
LimitNOFILE=65535
# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/nestjs/app/uploads
# Logging
StandardOutput=journal+console
StandardError=journal+console
SyslogIdentifier=nestjs-app
[Install]
WantedBy=multi-user.targetFrom my experience running NestJS services at Commsult Indonesia, use StandardOutput=journal+console and StandardError=journal+console directives to send both stdout/stderr to journald AND to the console simultaneously. This means you get structured log storage with journalctl while still seeing output in systemctl status. Add a SyslogIdentifier=your-service-name directive and logs are tagged with your service name — critical for filtering when you have 10+ services all writing to journald.
Systemd restart behavior is controlled by Restart=, RestartSec=, and StartLimitIntervalSec= / StartLimitBurst=. For production services: Restart=on-failure restarts on non-zero exit codes, RestartSec=5s waits 5 seconds between restarts (prevents tight crash loops from hammering the CPU), StartLimitIntervalSec=60 and StartLimitBurst=5 means if the service restarts 5 times in 60 seconds, systemd gives up and puts it in a failed state — triggering an alert and requiring manual intervention. This is safer than infinite restart loops that can mask underlying issues.
Systemd can enforce resource limits directly in the service file: MemoryMax=512M (OOM-kills the service if it exceeds 512MB), CPUQuota=50% (limits to half of one CPU core), LimitNOFILE=65535 (file descriptor limit for the service), and security hardening directives: PrivateTmp=true (isolated /tmp), ProtectSystem=strict (read-only filesystem except /var /run /tmp), NoNewPrivileges=true (prevents privilege escalation), and User=nestjs (run as a dedicated non-root user). These hardening directives reduce the blast radius of a compromised service.
# systemctl management commands
systemctl daemon-reload # reload after editing service files
systemctl enable nestjs-app # start on boot
systemctl start nestjs-app # start now
systemctl status nestjs-app # status + last 10 log lines
systemctl restart nestjs-app # restart service
# journalctl log access
journalctl -u nestjs-app -f # tail live logs
journalctl -u nestjs-app --since today
journalctl -u nestjs-app -p err # errors only
# Systemd timer (cron replacement)
# /etc/systemd/system/backup.timer
# [Timer]
# OnCalendar=daily
# OnBootSec=15min
# Persistent=true
#
# [Install]
# WantedBy=timers.target
systemctl enable --now backup.timer
systemctl list-timers # show all timers + next runEssential systemctl commands: systemctl start/stop/restart your-service, systemctl enable your-service (starts on boot), systemctl status your-service (shows last 10 log lines plus PID and resource usage), systemctl daemon-reload (required after editing service files), and journalctl -u your-service -f (tail logs for the service). For debugging crashes: journalctl -u your-service --since today shows all log output since midnight, and journalctl -u your-service -p err shows only error-level logs.
I burned several hours debugging a service that was starting before its PostgreSQL dependency was ready. After=postgresql.service only guarantees ordering — it does not check if PostgreSQL is actually ready to accept connections. For a NestJS app that needs PostgreSQL, use After=postgresql.service plus application-level retry logic: configure TypeORM or Prisma with retryAttempts and retryDelay to handle temporary connection failures on startup. Do not rely on systemd dependency ordering alone for service readiness — it only controls start order, not readiness.
Systemd timers are a modern replacement for cron jobs, with better logging, dependency management, and missed-job handling. Create a .service file describing what to run and a .timer file describing when to run it. Timers appear in systemctl status output, missed jobs are tracked, and output goes to journald. Use OnCalendar=daily for once-daily jobs, OnBootSec=15min OnUnitActiveSec=1h for hourly jobs starting 15 minutes after boot. The main advantage over cron: timer logs tell you exactly when the job ran, how long it took, and whether it succeeded.
My production NestJS systemd service template includes: User=nestjs (dedicated service account), WorkingDirectory=/opt/nestjs/app, ExecStart=/usr/bin/node dist/main.js, EnvironmentFile=/etc/nestjs/.env, Restart=on-failure with RestartSec=10s and StartLimitBurst=3, MemoryMax=512M, LimitNOFILE=65535, NoNewPrivileges=true, PrivateTmp=true, StandardOutput=journal+console, and SyslogIdentifier=nestjs-app. This template handles 95% of NestJS production deployments and provides restart resilience, resource limits, and proper logging.