Building a simple JSON processor using Java 17 and GraalVM


This week I finally decided to play with GraalVM to build a simple command-line JSON processor based on JsonPath. I find jq syntax too complex for my taste so I decided to build JSON processor based on JsonPath. Since, I wanted to release this as a native executable GraalVM seemed like a good soluton.

GraalVM is a relatively new JVM and JDK implementation built using Java itself. It supports additional programming languages and execution modes, like ahead-of-time compilation of Java applications for fast startup and low memory footprint.

Install GraalVM

I use SDKMAN to install and manage multiple JDK versions on my machine. You can install SDKMAN using following command.

curl -s "https://get.sdkman.io" | bash

Once installed you can install GraalVM Java 17 release version using following command.

sdk install java 22.3.r17-grl

You can enable a particular Java install using the use command shown below.

sdk use java 22.3.r17-grl

You can check Java version

java -version
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08, mixed mode, sharing)

So, now that we have JDK 17 GraalVM JDK and Java VM installed on our machine we will start developing our simple JSON processor.

Create a Gradle project

We will use IntelliJ to create a new project. IntelliJ is my IDE of choice and it has excellent support to create Gradle projects. You can also use gradle init command to create a Gradle project as well. I prefer IntelliJ so I am using it.

IntelliJ automatically detects that I have multiple JDK installations. I selected GraalVM version 17.0.5. Rest all options are pretty standard that all Java developers already use. Press Create after entering all the details.

Add GraalVM support in Gradle

We first have to update the settings.gradle to declare plugin repository.

The plugin isn’t available on the Gradle Plugin Portal yet, so you will need to declare a plugin repository in addition

pluginManagement {
    repositories {
        mavenCentral()
        gradlePluginPortal()
    }
}

rootProject.name = 'jp'
include('jp')

Next, we will update build.gradle

plugins {
    id 'application'
    id 'org.graalvm.buildtools.native' version '0.9.17'
}

group 'com.shekhargulati'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}

dependencies {
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.9.0'
}

application {
    mainClass = 'com.shekhargulati.jp.JpApp'
}

graalvmNative {
    binaries.all {
        resources.autodetect()
    }
    toolchainDetection = false
}

test {
    useJUnitPlatform()
    testLogging {
        events("passed", "failed", "skipped")
    }
}

In the build.gradle show above

  1. We added GraalVM specific plugins
  2. We defined our mainClass. This will be invoken when we will run jp.
  3. Finally, we configured graalvmNative as covered in GraalVM docs.

Next, we will create our Main class JpApp as shown below.

package com.shekhargulati.jp;

public class JpApp {

    public static void main(String[] args) {
        System.out.println("Hello, JpApp!!");
    }
}

Now, we are ready to build our native image.

./gradlew nativeBuild

This will build the native image. You will find the image in the ./build/native/nativeCompile.

ll ./build/native/nativeCompile/jp
Size User          Date Modified Name
11M shekhargulati  6 Nov 20:05  jp

The jp binary for Mac is 11M. We can run the image as shown below.

./build/native/nativeCompile/jp

We will get successful response as shown below.

Hello, JpApp!!

Using JsonPath to build a JSON processor

Add jsonpath dependency in build.gradle dependencies section.

implementation 'com.jayway.jsonpath:json-path:2.7.0'
implementation 'org.slf4j:slf4j-simple:2.0.3'

We will update JpApp to handle invalid input.

public class JpApp {

    public static void main(String[] args) {
        if (args == null || args.length < 1) {
            System.out.println("Please provide jsonpath you want to evaluate");
            System.out.println("Usage: jp '<jsonpath>'");
            System.out.println("For example: jp '$.total_count'");
            System.exit(0);
        }
    }
}

We will next use JsonPath library to read from the System.in and read the value at JsonPath.

package com.shekhargulati.jp;

import com.jayway.jsonpath.DocumentContext;
import com.jayway.jsonpath.JsonPath;

public class JpApp {

    public static void main(String[] args) {
        if (args == null || args.length < 1) {
            System.out.println("Please provide jsonpath you want to evaluate");
            System.out.println("Usage: jp '<jsonpath>'");
            System.out.println("For example: jp '$.store.book[*].author'");
            System.exit(0);
        }
        String jsonPath = args[0];
        DocumentContext documentContext = JsonPath.parse(System.in);
        Object extracted = documentContext.read(jsonPath);
        System.out.println(extracted);
    }
}

Let’s build the image and run a couple of tests.

./gradlew nativeBuild

Once image is built we can run against the example Json data in data/data.json

{
  "store": {
    "book": [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      },
      {
        "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      },
      {
        "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  },
  "expensive": 10
}

Now, we will run our tests against this JSON structure.

cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[*].author'
["Nigel Rees","Evelyn Waugh","Herman Melville","J. R. R. Tolkien"]

Let’s run one more JsonPath.

cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[?(@.price < 10)]'
[{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95},{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99}]

The above output does not look prett. To make it look pretty we can pass the output to jq 🙂

cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[?(@.price < 10)]' | jq .
[
  {
    "category": "reference",
    "author": "Nigel Rees",
    "title": "Sayings of the Century",
    "price": 8.95
  },
  {
    "category": "fiction",
    "author": "Herman Melville",
    "title": "Moby Dick",
    "isbn": "0-553-21311-3",
    "price": 8.99
  }
]

Let’s try one more example.

cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'
Exception in thread "main" com.jayway.jsonpath.InvalidPathException: Function of name: length cannot be created
    at com.jayway.jsonpath.internal.function.PathFunctionFactory.newFunction(PathFunctionFactory.java:76)
    at com.jayway.jsonpath.internal.path.FunctionPathToken.evaluate(FunctionPathToken.java:38)
    at com.jayway.jsonpath.internal.path.RootPathToken.evaluate(RootPathToken.java:66)
    at com.jayway.jsonpath.internal.path.CompiledPath.evaluate(CompiledPath.java:99)
    at com.jayway.jsonpath.internal.path.CompiledPath.evaluate(CompiledPath.java:107)
    at com.jayway.jsonpath.JsonPath.read(JsonPath.java:179)
    at com.jayway.jsonpath.internal.JsonContext.read(JsonContext.java:88)
    at com.jayway.jsonpath.internal.JsonContext.read(JsonContext.java:77)
    at com.shekhargulati.jp.JpApp.main(JpApp.java:17)
Caused by: java.lang.InstantiationException: com.jayway.jsonpath.internal.function.text.Length
    at java.base@17.0.5/java.lang.Class.newInstance(DynamicHub.java:639)
    at com.jayway.jsonpath.internal.function.PathFunctionFactory.newFunction(PathFunctionFactory.java:74)
    ... 8 more
Caused by: java.lang.NoSuchMethodException: com.jayway.jsonpath.internal.function.text.Length.<init>()
    at java.base@17.0.5/java.lang.Class.getConstructor0(DynamicHub.java:3585)
    at java.base@17.0.5/java.lang.Class.newInstance(DynamicHub.java:626)
    ... 9 more

Graal Reflection Metadata

You have to run Graal in agent mode to generate metadata that GraalVM when it encounters code that uses reflection.

./gradlew -Pagent run

Then, we have to copy generated files to resources/META-INF directory.

./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image

Now, build the image again.

./gradlew nativeBuild

Run the command again.

cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'

You will again get the same error.

The answer is to provide reflection metadata manually. Update reflect-config.json to the following.

[
  {
    "name": "com.jayway.jsonpath.internal.function.text.Length",
    "methods": [
      { "name": "<init>", "parameterTypes": [] }
    ]
  }
]

Now, again the build the image

./gradlew nativeBuild

Run the command again.

cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'

This time your command will run successfully and you will see right answer in your console.

4

Let’s check the size of the image.

ll ./build/native/nativeCompile/jp
Size User          Date Modified Name
14M shekhargulati  6 Nov 21:18  ./build/native/nativeCompile/jp

Nice, so we have 14M native image.

GitHub Actions

Finally, we will setup CI for our native image and upload artifacts.

Create a new GitHub repository and configure your project to use GitHub repository as remote.

Next, create a new file ci.yml under .github/workflows directory and populate with following content.

name: jp CI

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: jp on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [macos-latest, windows-latest, ubuntu-latest]
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v2
      - uses: graalvm/setup-graalvm@v1
        with:
          version: 'latest'
          java-version: '17'
          components: 'native-image'
          github-token: ${{ secrets.GITHUB_TOKEN }}

      - name: Build with Gradle
        uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
        with:
          arguments: nativeBuild

      - name: Smoke test jp
        run: |
          ./build/native/nativeCompile/jp

      - name: Upload binary
        uses: actions/upload-artifact@v2
        with:
          name: jp-${{ matrix.os }}
          path: build/native/nativeCompile/jp*

This GitHub workflow will build on all Windows, Linux, and Mac containers and upload the artifacts. You can download from here.

Download the one for your operating system. For Mac, you will have to make the jp executable.

sudo chmod +x jp

Next, you can copy to path that is already on your PATH.

cp jp /usr/local/bin

Now, you can run jp command.

curl https://jsonplaceholder.typicode.com/posts | jp '$.length()'
100

GitHub Repository

All code is available on GitHub https://github.com/shekhargulati/jp

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: