Obtain OCR results with the flow Java Client

This tutorial shows how to request OCR for a document and get OCR results by using jadice flow worker. We will implement a small java application to send an image to OCR recognition and retrieve the result.

Prerequisites

• A running jadice flow bundle with OCR cloud worker. In Tutorial 01 - Docker-compose: Local OCR we explain how to create and run this kind of environment locally.
• A maven capable java IDE

As already noted in Tutorial 01: To use the jadice flow components, a security token is needed (JADICE-FLOW-ACCESS-TOKEN). You should get a flow access token together with your license. For testing purposes, a test license can also be obtained by sending a request to jadice-support@levigo.de.

Check your access to https://artifacts.jadice.com/ and https://registry.jadice.com/. Images will be pulled from there. The "start-compose"-scripts we use later perform a docker login to the jadice registry.

How to complete this tutorial

You can start from scratch and complete each step, or you can get the full example code and apply only the necessary changes.

To start from scratch, move on to Creating the Java Project.

To skip the basics, do the following:

• Download and unzip the source repository for this tutorial, or clone it using Git:

  git clone https://github.com/levigo/jadice-flow-getting-started.git
• cd into jadice-flow-getting-started/jadice-flow-tutorial-02/
• in the file OCRTest.java replace THE-JADICE-FLOW-ACCESS-TOKEN with your access token
• Jump ahead to Run the test.

Creating the Java Project

Create a new Java Maven project in your local IDE.

Create pom.xml

Create the initial project object model pom.xml.

Add the controller-client dependency to the <dependencies>-Section:

pom.xml: Dependencies

<dependency>
  <groupId>com.jadice.flow</groupId>
  <artifactId>controller-client</artifactId>
  <version>0.40.0</version>
  <exclusions>
    <exclusion>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-core</artifactId>
    </exclusion>
  </exclusions>
</dependency>

The final pom.xml could look like:

pom.xml: Full example

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.jadice</groupId>
    <artifactId>jadice-flow-getting-started</artifactId>
    <version>0.0.1-SNAPSHOT</version>
  </parent>
  <artifactId>jadice-flow-tutorial-02</artifactId>

  <properties>
    <java.version>11</java.version>
    <maven.compiler.source>${java.version}</maven.compiler.source>
    <maven.compiler.target>${java.version}</maven.compiler.target>
    <controller-client.version>0.40.0</controller-client.version>
    <junit-jupiter.version>5.9.3</junit-jupiter.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.jadice.flow</groupId>
      <artifactId>controller-client</artifactId>
      <version>${controller-client.version}</version>
      <exclusions>
        <exclusion>
          <groupId>org.hibernate</groupId>
          <artifactId>hibernate-core</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit-jupiter.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>${junit-jupiter.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <version>${junit-jupiter.version}</version>
      <scope>test</scope>

    </dependency>
  </dependencies>
</project>

Java Sample Application

We create a JUnit test which will perform the following steps:

1. Create the FlowJobClient and EurekaStorageService
2. Create an image
3. Upload the image to the eureka storage
4. Invoke the jadice flow OCR worker and retrieve the OCR result
5. Check if the result is matching the expectation

In the sample test class below, the THE-JADICE-FLOW-ACCESS-TOKEN has to be replaced with your actual token.

Create the FlowJobClient and EurekaStorageService

To create the FlowJobClient and EurekaStorageService, these four parameters are required:

• JF_CONTROLLER_URL: URL of the jadice flow controller
• EUREKA_ENDPOINT: URL of eureka for uploads
• EUREKA_USERNAME: username for eureka
• EUREKA_PASSWORD: password for eureka

Required properties

private static final String JF_CONTROLLER_URL = "http://localhost:8080";
private static final String EUREKA_USERNAME = "user";
private static final String EUREKA_PASSWORD = "password";
private static final String EUREKA_ENDPOINT = "http://localhost:8085";

Then, the clients are created with the given parameters:

Client initialization

private static final EurekaStorageService storageService = new EurekaStorageService(EUREKA_ENDPOINT, EUREKA_USERNAME, EUREKA_PASSWORD);
private static final FlowJobClient flowJobClient = new FlowJobClient(JF_CONTROLLER_URL,
        Duration.ofSeconds(60), TokenProvider.NOOP_PROVIDER);

Create an image

The test generates a new image with the contents "OCR test". Of course, the test can be modified to load an image directly via ImageIO.read(), for example. The expected outcome of the test has to be changed in that case (or remove the assert for documents with unknown result).

Upload to eureka

Uploading to eureka can be performed with the EurekaStorageService via String uploadFile = storageService.upload(fileName, mimeType, new ByteArrayInputStream(imgData));. This method returns a URI to the uploaded file.

Create a job request

To call the jadice flow, we need to create a JobRequest which provides the necessary parameters to the controller.

Create the JobRequest

Item item = new Item();
Part p = new Part();
p.setMimeType(mimeType);
p.setUrl(uploadUrl);
item.addParts(Collections.singletonList(p));
JobRequest request = new JobRequest();
request.setJobTemplateName(JOB_TEMPLATE_NAME);
request.getItems().add(item);

The configuration for the jadice flow worker is done in the job-template - see tutorial-01. The jf-worker-tessocr allows setting the output-format to one or more of these:

• text - Plain text result
• hocr - HOCR XML output
• pdf - PDF output (recognized text in a PDF document) In our case the jobtemplate 'ocr' sets "output-formats" to TEXT_AND_HOCR, which leads to the same order of formats in the result - plain text stream first, hOCR second.

Create a jadice flow job and wait for the result

Create Job and wait for result

// Invoke Job with FlowJobClient
JobRequest jobRequest = prepareJobRequest(uploadFile, contentType);
JobInformation job = flowJobClient.createJob(jobRequest);
long jobExecutionID = job.getJobExecutionID();

// Wait for completion
JobInformation jobInformation = flowJobClient.awaitFinalJobState(jobExecutionID, 60000, TimeUnit.MILLISECONDS);
String status = jobInformation.getStatus();

Part[] resultParts = flowJobClient.getResultParts(jobExecutionID);
Part resultPart = resultParts[0];
if (resultPart != null) {
  String resultUri = resultPart.getUrl();
  
  // download result with StorageService
  System.out.println("downloading file from " + resultUri + " ...");
  InputStream inputStream = storageService.download(resultUri);
  
  return getResultText(inputStream);
} else {
  throw new Exception(
  String.format("No result for job with id=%d - finished with status %s", job.getJobExecutionID(), status));
}

Full example

Here you can find the complete example. Please note that the authToken has to be replaced by the actual one.

Full example: OCRTest

package org.jadice.tutorial;


import static org.junit.jupiter.api.Assertions.assertEquals;

import java.awt.Color;
import java.awt.Font;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.time.Duration;
import java.util.Arrays;
import java.util.Collections;
import java.util.Optional;
import java.util.concurrent.TimeUnit;

import javax.imageio.ImageIO;

import org.junit.jupiter.api.Test;

import com.jadice.flow.controller.client.connector.FlowJobClient;
import com.jadice.flow.controller.client.connector.TokenProvider;
import com.jadice.flow.controller.client.impl.EurekaStorageService;
import com.jadice.flow.controller.model.Item;
import com.jadice.flow.controller.model.Part;
import com.jadice.flow.controller.rest.JobInformation;
import com.jadice.flow.controller.rest.JobRequest;

/**
 * A simple OCR test which performs the following actions:
 * <ul>
 * <li>Create an image</li>
 * <li>Use jadice flow OCR worker to retrieve the OCR result</li>
 * <li>Compare the result with the expected text</li>
 * </ul>
 */
class OCRTest {
  // Parameters to access this jadice flow bundle
  private static final String JF_CONTROLLER_URL = "http://localhost:8080";
  private static final String EUREKA_USERNAME = "user";
  private static final String EUREKA_PASSWORD = "password";
  private static final String EUREKA_ENDPOINT = "http://localhost:8085";
  private static final String JOB_TEMPLATE_NAME = "ocr";

  // create one client for the storage and one for the controller
  private static final EurekaStorageService storageService = new EurekaStorageService(EUREKA_ENDPOINT, EUREKA_USERNAME, EUREKA_PASSWORD);
  private static final FlowJobClient flowJobClient = new FlowJobClient(JF_CONTROLLER_URL,
          Duration.ofSeconds(60), TokenProvider.NOOP_PROVIDER);

  /**
   * A simple OCR test which performs the following actions:
   * <ul>
   * <li>Create an image containing text</li>
   * <li>Use jadice flow OCR worker to retrieve the OCR result</li>
   * <li>Compare the result with the expected text</li>
   * </ul>
   *
   * @throws Exception if something goes wrong
   */
  @Test
  void test_correctOCRResult() throws Exception {
    // The text to recognize
    String text = "OCR test";

    // Create an image (or load via ImageIO.read(...) and adjust expected text)
    BufferedImage image = createImage();

    // Get byte contents (png)
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ImageIO.write(image, "png", baos);
    byte[] imgData = baos.toByteArray();

    // Upload file with StorageService
    String fileName = "temp_ocr.png";
    String mimeType = "image/png";
    String uploadFile = storageService.upload(fileName, mimeType, new ByteArrayInputStream(imgData));
    System.out.println("uploaded file to " + uploadFile);

    // Call jadice flow
    String ocrResultPlainText = invokeJadiceFlowOCRWorker(uploadFile, mimeType);
    System.out.println("OCR result text: " + ocrResultPlainText);

    // Assert the result
    assertEquals(text, ocrResultPlainText);
  }

  /**
   * Creates the configuration parameters and send the OCR request to the jadice flow controller.
   * <p>
   *
   * The timeouts in this Demo implementation are hard coded to 60sec.
   *
   * @param uploadFile the URI for the uploaded image data
   * @param contentType mime type of image
   * @return the plain text OCR result
   * @throws Exception if something goes wrong
   */
  private String invokeJadiceFlowOCRWorker(String uploadFile, String contentType) throws Exception {
    // Invoke Job with FlowJobClient
    JobRequest jobRequest = prepareJobRequest(uploadFile, contentType);
    JobInformation job = flowJobClient.createJob(jobRequest);
    long jobExecutionID = job.getJobExecutionID();

    // Wait for completion
    JobInformation jobInformation = flowJobClient.awaitFinalJobState(jobExecutionID, 60000, TimeUnit.MILLISECONDS);
    String status = jobInformation.getStatus();

    Part[] resultParts = flowJobClient.getResultParts(jobExecutionID);
    Part resultPart = resultParts[0];
    if (resultPart != null) {
      String resultUri = resultPart.getUrl();

      // download result with StorageService
      System.out.println("downloading file from " + resultUri + " ...");
      InputStream inputStream = storageService.download(resultUri);

      return getResultText(inputStream);
    } else {
      throw new Exception(
              String.format("No result for job with id=%d - finished with status %s", job.getJobExecutionID(), status));
    }
  }

  private static JobRequest prepareJobRequest(String uploadUrl, String mimeType) {
    Item item = new Item();
    Part p = new Part();
    p.setMimeType(mimeType);
    p.setUrl(uploadUrl);
    item.addParts(Collections.singletonList(p));
    JobRequest request = new JobRequest();
    request.setJobTemplateName(JOB_TEMPLATE_NAME);
    request.getItems().add(item);
    return request;
  }

  /**
   * Utility method to get the result as string.
   *
   * @return the result as string
   * @throws IOException if something goes wrong
   */
  private String getResultText(InputStream inputStream) throws IOException {
    StringBuilder sb = new StringBuilder();
    try (BufferedReader br = new BufferedReader(new InputStreamReader(inputStream))) {
      String line;
      while ((line = br.readLine()) != null) {
        if (sb.length() > 0) {
          sb.append(System.lineSeparator());
        }
        sb.append(line.trim());
      }
    }
    return sb.toString().trim();
  }

  /**
   * Creates a BufferedImage containing the text "OCR test".
   *
   * @return the BufferedImage
   */
  private static BufferedImage createImage() {
    String text = "OCR test";
    int width = 400;
    int height = 150;
    BufferedImage img = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
    Graphics2D g = (Graphics2D) img.getGraphics();

    g.setColor(Color.white);
    g.fillRect(0, 0, width, height);

    g.setColor(Color.black);
    g.setFont(new Font("dialog", Font.PLAIN, 24));
    g.drawString(text, 10, height / 2);

    return img;
  }
}

Run the test

Run the test in your IDE or with mvn test. It should print "OCR result text: OCR test" to the console. You can now modify the test to use your own test files instead of the created image. Or maybe tryout the different return formats.