Deploying a Separate Worker-TopDF for Preprocessing

This guide explains how to deploy a second worker-topdf instance dedicated to preprocessing operations. This is an optional configuration that allows you to separate document export workflows from preprocessing pipelines.

Key Architecture Principle: The Output Organizer chart itself remains completely unchanged. The preprocessing worker is deployed either as an independent service (Option 1) or included in a custom wrapper chart (Option 2).

Architecture Overview

This configuration deploys two worker-topdf instances:

• Primary Worker: Handles document export for collections
• Preprocessing Worker: Handles document conversion in preprocessing pipelines

Both workers run the same jf-worker-topdf image but with different configurations and resource allocations.

---

Deployment Methods

You have two options for deploying the preprocessing worker.

Option 1: Separate Chart Deployment

Deploy the preprocessing worker as an independent Helm release in the same namespace as Output Organizer. This approach keeps the Output Organizer chart pure and allows you to update it independently. Use this if you deploy Output Organizer directly from the Helm chart repository without modifications.

Option 2: Wrapper Chart Method

Create a custom wrapper chart that includes both Output Organizer and the preprocessing worker as dependencies. This approach is ideal if you maintain your own deployment chart for managing your complete stack.

---

Option 1: Separate Chart Deployment

This approach deploys the preprocessing worker as an independent Helm chart in the same namespace as Output Organizer. The Output Organizer chart remains unchanged.

Step 1: Create Values Override File

Create a values-preprocessing.yaml file for the preprocessing worker:

# values-preprocessing.yaml - Preprocessing Worker Configuration

fullnameOverride: jf-worker-topdf-preprocessing

# General configuration
image:
  registry: registry.jadice.com
  pullPolicy: IfNotPresent

worker:
  application:
    enforceStreamAnalysis: false
    analyzeStreamBeforePublishing: false
    annotation:
      profiles: [ "file:/app/annotation/jwt-annotation-profile.xml" ]
      defaultProfile: "JWT-Default"
      redactionAnnotationTypes: "[\"Mask\", \"TextMask\" ]"
    uniqueOutputFilename: false
    topdf:
      conformanceValidationDisabled: true
      contentValidationDisabled: true
      imageResolution: 150  # Lower resolution for preprocessing

additionalConfigMaps:
  - mountPath: /app/annotation/jwt-annotation-profile.xml
    subPath: jwt-annotation-profile.xml
    name: anno-profile
    readOnly: true

# Deployment configuration
ingress:
  enabled: false

resources:
  requests:
    cpu: 300m
    memory: 1Gi
  limits:
    cpu: 1000m
    memory: 2Gi

livenessProbe:
  enabled: true
  initialDelaySeconds: 60
  periodSeconds: 30
  timeoutSeconds: 10
  failureThreshold: 10

readinessProbe:
  enabled: true
  initialDelaySeconds: 60
  periodSeconds: 30
  timeoutSeconds: 10
  failureThreshold: 10

serviceAccount:
  name: jadice-flow-worker

secrets:
  eureka:
    username: admin
    password: admin

enableJsonLogging: false

auth:
  existingSecrets:
    eureka: "eureka-secret"

eureka:
  endpoint: http://fusion-output-organizer

eventBus:
  enabled: false

Step 2: Deploy Preprocessing Worker

Deploy the preprocessing worker as a separate Helm release in the same namespace:

# Add the helm repository
helm repo add levigo https://artifacts.jadice.com/repository/helm-charts/ \
  --username <username> --password <password>

# Deploy preprocessing worker in the same namespace as Output Organizer
helm install jf-worker-topdf-preprocessing levigo/jf-worker-topdf \
  -f values-preprocessing.yaml \
  -n fusion-namespace \
  --version x.xx.x

Option 2: Wrapper Chart Method

This approach creates a separate wrapper chart that includes Output Organizer as a dependency along with the preprocessing worker. The Output Organizer chart itself remains completely unchanged. This is ideal for organizations that want to manage a complete deployment stack.

Step 1: Create a Wrapper Chart Structure

Create a new Helm chart directory structure for your deployment (e.g., custom-fusion-stack/):

custom-fusion-stack/
├── Chart.yaml
├── values.yaml
├── templates/
│   ├── cm-workers.yaml          (optional - if you need custom worker config)
│   └── cm-jobtemplates.yaml     (optional - if you need custom job config)
└── README.md

Step 2: Create Chart.yaml for Wrapper Chart

Create the wrapper chart that depends on both Output Organizer and the preprocessing worker:

# custom-fusion-stack/Chart.yaml
apiVersion: v2
name: custom-fusion-stack
description: Complete Fusion Output Organizer stack with preprocessing worker
type: application
version: 1.0.0

dependencies:
  # Include Output Organizer as a dependency (unchanged from upstream)
  - name: fusion-output-organizer
    version: 2.17.1
    repository: https://artifacts.jadice.com/repository/helm-charts/

  # Add preprocessing worker independently
  - name: jf-worker-topdf
    alias: worker-topdf-preprocessing
    version: 1.61.4
    repository: https://artifacts.jadice.com/repository/helm-charts/

Then update chart dependencies:

cd custom-fusion-stack/
helm dependency update

Step 3: Configure Wrapper Chart values.yaml

Create the complete values configuration for your deployment:

# custom-fusion-stack/values.yaml

# Configure the Output Organizer instance (included as dependency)
fusion-output-organizer:
  organizer:
    stage: prod
    replicaCount: 1
    image:
      registry: registry.jadice.com
      tag: "2.43.13"

  controller:
    enabled: true
    workersConfigMap: jadice-flow-workers-custom
    jobtemplatesConfigMap: jadice-flow-jobtemplates-custom
    resources:
      requests:
        cpu: 500m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 2Gi

  # Primary worker-topdf for collections export
  worker-topdf:
    enabled: true
    fullnameOverride: jf-worker-topdf
    resources:
      requests:
        cpu: 600m
        memory: 2Gi
      limits:
        cpu: 2000m
        memory: 4Gi
    worker:
      application:
        topdf:
          imageResolution: 300

# Configure the preprocessing worker (separate dependency)
worker-topdf-preprocessing:
  enabled: true
  fullnameOverride: jf-worker-topdf-preprocessing

  # General configuration
  image:
    registry: registry.jadice.com
    pullPolicy: IfNotPresent

  worker:
    application:
      enforceStreamAnalysis: false
      analyzeStreamBeforePublishing: false
      annotation:
        profiles: [ "file:/app/annotation/jwt-annotation-profile.xml" ]
        defaultProfile: "JWT-Default"
        redactionAnnotationTypes: "[\"Mask\", \"TextMask\" ]"
      uniqueOutputFilename: false
      topdf:
        conformanceValidationDisabled: true
        contentValidationDisabled: true
        imageResolution: 150  # Lower resolution for preprocessing

  additionalConfigMaps:
    - mountPath: /app/annotation/jwt-annotation-profile.xml
      subPath: jwt-annotation-profile.xml
      name: anno-profile
      readOnly: true

  # Deployment configuration
  resources:
    requests:
      cpu: 300m
      memory: 1Gi
    limits:
      cpu: 1000m
      memory: 2Gi

  livenessProbe:
    enabled: true
    initialDelaySeconds: 60
    periodSeconds: 30
    timeoutSeconds: 10
    failureThreshold: 10

  readinessProbe:
    enabled: true
    initialDelaySeconds: 60
    periodSeconds: 30
    timeoutSeconds: 10
    failureThreshold: 10

  serviceAccount:
    name: jadice-flow-worker

  secrets:
    eureka:
      username: admin
      password: admin

  enableJsonLogging: false

  auth:
    existingSecrets:
      eureka: "eureka-secret"

  eureka:
    endpoint: http://fusion-output-organizer

  eventBus:
    enabled: false

Step 4: Deploy the Wrapper Chart

Deploy your wrapper chart which includes both Output Organizer and the preprocessing worker:

# From the custom-fusion-stack directory
helm upgrade --install custom-fusion-stack . \
  -f values.yaml \
  -n fusion-namespace

# Or install with specific version
helm upgrade --install custom-fusion-stack . \
  -f values.yaml \
  -n fusion-namespace \
  --create-namespace