Installation standalone

Standalone distributions / products come with a predefined set of jadice flow components and workers to achieve specific use cases. In theory, almost every jadice flow product could also be shipped as standalone product. Since the main deployment platform is kubernetes, this is not always the case by default.

Products might provide more specific installation instructions. This documentation gives a general overview of a standalone product.

Package contents

This package contains:

Main controller and worker components:

• jf-controller: Contains the jadice flow controller. This is the main component controlling jobs and managing the workflows.
• jf-controller-ui: Contains the UI. This is a web based UI to manage the controller
• worker-*: Contains specific worker applications, e.g. worker-topdf

Additional components

• jre: Jadice flow comes with a bundled JRE (Java 17).
• mariadb (optional): MariaDB installation package is provided for convenience. This is an optional but recommended installation, since the H2 database will be used otherwise (not recommended in production). See installation chapter for more info.
• minio: (Scripts only) The S3 data storage - please see the specific storage chapter
• wrapper: Contains the java service wrapper which is used to start the java applications
• ZMC: (optional) Contains the Azul Zulu Mission Control. This is a diagnostic tool which can be used to attach to the services and read application metrics via JMX.

Prerequisites

Currently, the only prerequisite is a S3 compatible storage.

Configuration of products components

Extract the ZIP into a folder. The extracted folder contains a sub-folder for each component. In the root folder, scripts to install or run the services are available. The install-all-services-Script installs the main components. Additional components have to be installed seperately.

All components which require a configuration have a separate config subfolder which might contain further or specific configuration.

jf-controller

The jadice flow controller has the following configuration files:

• application.yaml: Main configuration file with server port etc.
• workers.yaml: Configuration file which defines which workers are available (this file is usually pre-configured and does not need to be changed)
• jobtemplates.yaml: Configuration file for job templates. Pre-configured with a default template. This template could be checked for parameter "placeholders". Either you can edit the file directly or use the controller-UI to check the configuration in the browser (recommended).
• wrapper.conf: Java Service Wrapper configuration file (pre-configured)

For more information see General configuration

jf-controller-ui

The jadice flow controller-ui has the following configuration files:

• application.yaml: Main configuration file with server port etc.
• wrapper.conf: Java Service Wrapper configuration file (pre-configured)

worker-*

The workers have in general the following configuration files:

• application.yaml: Main configuration file with server port etc.
• wrapper.conf: Java Service Wrapper configuration file (pre-configured)

Some workers may have additional files for configuration. Some workers also might provide a native binary instead of the wrapper.

Initial configuration to modify before start

Database

The jadice flow controller utilizes a jdbc database to store job runtime information. By default, the standalone distribution will use a H2 in memory runtime Database (DB) for testing / showcasing purposes.

For production, an external DB should be used to keep job information persistent. This product comes with a mariadb package which can be installed locally if needed.

Local MariaDB installation

By default, the in memory H2 DB is being used.

To use the maria db, the application.yaml file of the jf-controller-component needs to be modified:

  # JDBC Config in application.yaml
  datasource:
    initialization-mode: always
    url: jdbc:mariadb://localhost:3306/jadice_flow?createDatabaseIfNotExist=true
    username: jadice-flow
    password: jadice-flow
    hikari:
      schema: JADICE_FLOW
      maximum-pool-size: 50

s3 storage

The application.yaml files are pre-configured to use a local s3 with the following configuration:

publisher:
  s3:
    bucket: jadice-flow-bucket
    endpoint: localhost:9000
    access-key: jadice-flow
    secret-key: jadice-flow
    protocol: http

If an external S3 shall be used, the application.yaml files have to be adjusted accordingly.

Local S3 using MinIO

One possible local S3 solution is MinIO. This open source S3 implementation is licensed under the AGPL-License. For this reason, MinIO does not come bundled with this package.

Please check the minio folder for a setup guide: minio-installation.html.

Alternatives for local S3 storage:

• MinIO
• Ceph -

This (external) ceph installation tutorial shows the basic steps required to run ceph locally.

Usage

The components can be installed as service, or run directly on the command line. Several scripts are provided for these cases:

• install-all-services: Installs the main components as service. Additional components like mariadb have to be installed seperately via separate single install scripts, e.g. install-mariadb.bat.
• uninstall-all-services: Uninstalls all services which were installed via install-all
• The start-all-services and stop-all-services scripts can be used to start and stop the services

If no service installation is needed, the components can be started directly:

• run-jf-controller-console
• run-jf-controller-ui-console
• run-worker-* : Here with a wildcard - there is a run script for each provided worker