Integrating Third-Party APIs with a Custom ERP: Patterns, Webhooks, and Error Handling

Modern ERP systems don't operate in isolation. They need to talk to payment gateways, logistics platforms, e-invoicing systems, government portals, and marketplace APIs. Event-driven architecture with webhooks and message queues can reduce integration latency from minutes to seconds, according to API integration research. In Indonesia, this is particularly relevant: ERP systems need to connect to DJP's e-Faktur for VAT invoices, BPJS for social security reporting, and local payment gateways like Midtrans or Xendit. At Commsult, I've integrated over a dozen external APIs into our custom ERP. This post covers the patterns and pitfalls.

The Three Integration Patterns

Every ERP-to-external integration falls into one of three patterns: synchronous REST API calls (ERP calls external API and waits for response), asynchronous webhooks (external system notifies ERP when something happens), and file-based batch exchange (daily file drops, CSV imports, SFTP). Each pattern has appropriate use cases. Synchronous is right for real-time queries (check a payment status). Webhooks are right for event notifications (payment confirmed, delivery status changed). Batch is right for high-volume data that doesn't require real-time updates (daily transaction reconciliation).

Outbound Integration: Calling External APIs

For outbound API calls from NestJS, we use a dedicated IntegrationService per external system. Each service handles: authentication (API keys stored in environment variables, OAuth token refresh if needed), request building, response parsing, error classification (retry-able vs. permanent errors), and logging. We wrap all external calls in a Circuit Breaker pattern (using the opossum library) — if the external API fails repeatedly, the circuit opens and returns a cached or default response instead of hammering a failing service.

ERP Third-Party Integration Architecture

  ┌──────────────────────────────────────────────────────┐
  │                   ERP (NestJS)                        │
  │                                                      │
  │  Business Logic → IntegrationModule                  │
  │                        │                             │
  │         ┌──────────────┼──────────────┐              │
  │         ▼              ▼              ▼              │
  │  MidtransAdapter  EFakturAdapter  BpjsAdapter        │
  │  (payment gw)    (DJP SOAP)      (REST API)          │
  └──────────┬───────────────┬───────────┬───────────────┘
             │               │           │
  OUTBOUND   ▼               ▼           ▼
  ┌──────────────┐  ┌──────────────┐  ┌──────────┐
  │  Midtrans    │  │  DJP e-Faktur│  │  BPJS    │
  │  Payment GW  │  │  SOAP API    │  │  REST API│
  └──────────────┘  └──────────────┘  └──────────┘

  INBOUND WEBHOOKS:
  External System → POST /webhooks/{provider}
       │
       ▼ (< 200ms — quick return)
  ┌────────────────────────────────────┐
  │  Validate signature                │
  │  Create Bull job                   │
  │  Return HTTP 200 immediately       │
  └──────────────────┬─────────────────┘
                     │
                     ▼
  ┌────────────────────────────────────┐
  │  Bull Worker (async)               │
  │  Process event                     │
  │  Update ERP state                  │
  │  Retry on failure (exponential)    │
  │  → Dead Letter Queue after max retries │
  └────────────────────────────────────┘

Inbound Webhooks: Receiving External Events

Webhooks from external systems arrive at a NestJS endpoint as HTTP POST requests. The critical implementation requirements: validate the webhook signature (each provider has a different signing mechanism — Midtrans uses an MD5 hash, Xendit uses x-callback-token header, DHL uses HMAC-SHA256), make the handler idempotent (duplicate webhooks must not create duplicate records), and return 200 quickly (never do heavy processing synchronously in a webhook handler — push to a queue and return 200 within 2 seconds or the sender will retry).

Message Queue Architecture for Reliable Integration

We use Bull (Redis-backed job queue) for all asynchronous integration processing. When a webhook arrives, the handler validates the signature, creates a job in Bull, and returns 200. The Bull worker processes the job: parses the event, updates ERP state, triggers downstream actions (e.g., mark invoice paid, send notification). If the processing fails, Bull retries with exponential backoff. After a configurable number of retries, jobs go to a dead-letter queue for manual review. This architecture means no webhook event is ever lost, even if the ERP is temporarily down when it arrives.

// NestJS: Uniform Integration Adapter Interface
export interface IntegrationAdapter {
  connect(): Promise<void>;
  healthCheck(): Promise<boolean>;
  send(payload: unknown): Promise<{ referenceId: string }>;
  getStatus(referenceId: string): Promise<{ status: string }>;
}

// Example: Midtrans adapter
@Injectable()
export class MidtransAdapter implements IntegrationAdapter {
  private breaker: CircuitBreaker;

  constructor() {
    this.breaker = new CircuitBreaker(this.callMidtrans.bind(this), {
      timeout: 5000,         // 5s timeout
      errorThresholdPercentage: 50,
      resetTimeout: 30000,   // 30s before retry after open
    });
  }

  async send(payload: CreatePaymentPayload) {
    return this.breaker.fire(payload);
  }

  private async callMidtrans(payload: CreatePaymentPayload) {
    const res = await fetch('https://api.midtrans.com/v2/charge', {
      method: 'POST',
      headers: {
        'Authorization': 'Basic ' + Buffer.from(
          process.env.MIDTRANS_SERVER_KEY + ':'
        ).toString('base64'),
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(payload),
    });
    if (!res.ok) throw new Error(`Midtrans error: ${res.status}`);
    return res.json();
  }
}

// Bull queue: reliable webhook processing
@Processor('webhook-events')
export class WebhookProcessor {
  @Process('midtrans-payment')
  async handleMidtransPayment(@InjectQueue() job: Job<MidtransEvent>) {
    // Idempotent: skip if already processed
    const exists = await this.paymentRepo.findByReference(job.data.transaction_id);
    if (exists) return;

    await this.arService.recordPayment(job.data);
    this.logger.log(`Payment processed: ${job.data.transaction_id}`);
  }
}

How We Built This at Commsult

Our integration layer is a dedicated NestJS module with adapters per external system. The adapter interface is: connect(), healthCheck(), send(payload), and getStatus(referenceId). This uniform interface makes it easy to swap providers — we switched from one SMS gateway to another without changing any business logic, only the adapter implementation. The adapters are registered as NestJS providers with a symbol key, and the business services inject them by interface rather than concrete class.

Indonesian-Specific Integrations

The integrations specific to Indonesian business context: DJP e-Faktur (PPN invoice submission via SOAP API — yes, it's SOAP, not REST), BPJS API (monthly contribution reporting), Bank Indonesia payment gateway APIs (SNAP — Standard National Open API for banking interoperability), and LKPP (government procurement portal) for companies that sell to government entities. Each of these has quirks: e-Faktur uses a certificate-based authentication and a SOAP envelope format; SNAP requires an HMAC-SHA512 signature with a specific timestamp format. Document these integration details thoroughly — they're poorly documented by the Indonesian government.

Monitoring and Alerting for Integrations

Integration health monitoring is critical. We expose a /health/integrations endpoint that pings each external service and returns green/amber/red status. This is polled by our monitoring stack every 5 minutes. Failed integrations trigger a Slack alert to the engineering team immediately. We also maintain an integration_events table: every outbound call and inbound webhook is logged with the external reference ID, payload hash, response status, and processing time. This log is indispensable when debugging integration discrepancies — 'did we actually send that e-Faktur?' is answered by querying the table.