This week I finally decided to play with GraalVM to build a simple command-line JSON processor based on JsonPath. I find jq syntax too complex for my taste so I decided to build JSON processor based on JsonPath. Since, I wanted to release this as a native executable GraalVM seemed like a good soluton.
GraalVM is a relatively new JVM and JDK implementation built using Java itself. It supports additional programming languages and execution modes, like ahead-of-time compilation of Java applications for fast startup and low memory footprint.
Install GraalVM
I use SDKMAN to install and manage multiple JDK versions on my machine. You can install SDKMAN using following command.
curl -s "https://get.sdkman.io" | bash
Once installed you can install GraalVM Java 17 release version using following command.
sdk install java 22.3.r17-grl
You can enable a particular Java install using the use command shown below.
sdk use java 22.3.r17-grl
You can check Java version
java -version
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08, mixed mode, sharing)
So, now that we have JDK 17 GraalVM JDK and Java VM installed on our machine we will start developing our simple JSON processor.
Create a Gradle project
We will use IntelliJ to create a new project. IntelliJ is my IDE of choice and it has excellent support to create Gradle projects. You can also use gradle init command to create a Gradle project as well. I prefer IntelliJ so I am using it.
IntelliJ automatically detects that I have multiple JDK installations. I selected GraalVM version 17.0.5. Rest all options are pretty standard that all Java developers already use. Press Create after entering all the details.
Add GraalVM support in Gradle
We first have to update the settings.gradle to declare plugin repository.
The plugin isn’t available on the Gradle Plugin Portal yet, so you will need to declare a plugin repository in addition
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
rootProject.name = 'jp'
include('jp')
Next, we will update build.gradle
plugins {
id 'application'
id 'org.graalvm.buildtools.native' version '0.9.17'
}
group 'com.shekhargulati'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.9.0'
}
application {
mainClass = 'com.shekhargulati.jp.JpApp'
}
graalvmNative {
binaries.all {
resources.autodetect()
}
toolchainDetection = false
}
test {
useJUnitPlatform()
testLogging {
events("passed", "failed", "skipped")
}
}
In the build.gradle show above
- We added GraalVM specific plugins
- We defined our
mainClass. This will be invoken when we will runjp. - Finally, we configured graalvmNative as covered in GraalVM docs.
Next, we will create our Main class JpApp as shown below.
package com.shekhargulati.jp;
public class JpApp {
public static void main(String[] args) {
System.out.println("Hello, JpApp!!");
}
}
Now, we are ready to build our native image.
./gradlew nativeBuild
This will build the native image. You will find the image in the ./build/native/nativeCompile.
ll ./build/native/nativeCompile/jp
Size User Date Modified Name
11M shekhargulati 6 Nov 20:05 jp
The jp binary for Mac is 11M. We can run the image as shown below.
./build/native/nativeCompile/jp
We will get successful response as shown below.
Hello, JpApp!!
Using JsonPath to build a JSON processor
Add jsonpath dependency in build.gradle dependencies section.
implementation 'com.jayway.jsonpath:json-path:2.7.0'
implementation 'org.slf4j:slf4j-simple:2.0.3'
We will update JpApp to handle invalid input.
public class JpApp {
public static void main(String[] args) {
if (args == null || args.length < 1) {
System.out.println("Please provide jsonpath you want to evaluate");
System.out.println("Usage: jp '<jsonpath>'");
System.out.println("For example: jp '$.total_count'");
System.exit(0);
}
}
}
We will next use JsonPath library to read from the System.in and read the value at JsonPath.
package com.shekhargulati.jp;
import com.jayway.jsonpath.DocumentContext;
import com.jayway.jsonpath.JsonPath;
public class JpApp {
public static void main(String[] args) {
if (args == null || args.length < 1) {
System.out.println("Please provide jsonpath you want to evaluate");
System.out.println("Usage: jp '<jsonpath>'");
System.out.println("For example: jp '$.store.book[*].author'");
System.exit(0);
}
String jsonPath = args[0];
DocumentContext documentContext = JsonPath.parse(System.in);
Object extracted = documentContext.read(jsonPath);
System.out.println(extracted);
}
}
Let’s build the image and run a couple of tests.
./gradlew nativeBuild
Once image is built we can run against the example Json data in data/data.json
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{
"category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}
Now, we will run our tests against this JSON structure.
cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[*].author'
["Nigel Rees","Evelyn Waugh","Herman Melville","J. R. R. Tolkien"]
Let’s run one more JsonPath.
cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[?(@.price < 10)]'
[{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95},{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99}]
The above output does not look prett. To make it look pretty we can pass the output to jq 🙂
cat data/data.json| ./build/native/nativeCompile/jp '$.store.book[?(@.price < 10)]' | jq .
[
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
}
]
Let’s try one more example.
cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'
Exception in thread "main" com.jayway.jsonpath.InvalidPathException: Function of name: length cannot be created
at com.jayway.jsonpath.internal.function.PathFunctionFactory.newFunction(PathFunctionFactory.java:76)
at com.jayway.jsonpath.internal.path.FunctionPathToken.evaluate(FunctionPathToken.java:38)
at com.jayway.jsonpath.internal.path.RootPathToken.evaluate(RootPathToken.java:66)
at com.jayway.jsonpath.internal.path.CompiledPath.evaluate(CompiledPath.java:99)
at com.jayway.jsonpath.internal.path.CompiledPath.evaluate(CompiledPath.java:107)
at com.jayway.jsonpath.JsonPath.read(JsonPath.java:179)
at com.jayway.jsonpath.internal.JsonContext.read(JsonContext.java:88)
at com.jayway.jsonpath.internal.JsonContext.read(JsonContext.java:77)
at com.shekhargulati.jp.JpApp.main(JpApp.java:17)
Caused by: java.lang.InstantiationException: com.jayway.jsonpath.internal.function.text.Length
at java.base@17.0.5/java.lang.Class.newInstance(DynamicHub.java:639)
at com.jayway.jsonpath.internal.function.PathFunctionFactory.newFunction(PathFunctionFactory.java:74)
... 8 more
Caused by: java.lang.NoSuchMethodException: com.jayway.jsonpath.internal.function.text.Length.<init>()
at java.base@17.0.5/java.lang.Class.getConstructor0(DynamicHub.java:3585)
at java.base@17.0.5/java.lang.Class.newInstance(DynamicHub.java:626)
... 9 more
Graal Reflection Metadata
You have to run Graal in agent mode to generate metadata that GraalVM when it encounters code that uses reflection.
./gradlew -Pagent run
Then, we have to copy generated files to resources/META-INF directory.
./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image
Now, build the image again.
./gradlew nativeBuild
Run the command again.
cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'
You will again get the same error.
The answer is to provide reflection metadata manually. Update reflect-config.json to the following.
[
{
"name": "com.jayway.jsonpath.internal.function.text.Length",
"methods": [
{ "name": "<init>", "parameterTypes": [] }
]
}
]
Now, again the build the image
./gradlew nativeBuild
Run the command again.
cat data/data.json| ./build/native/nativeCompile/jp '$..book.length()'
This time your command will run successfully and you will see right answer in your console.
4
Let’s check the size of the image.
ll ./build/native/nativeCompile/jp
Size User Date Modified Name
14M shekhargulati 6 Nov 21:18 ./build/native/nativeCompile/jp
Nice, so we have 14M native image.
GitHub Actions
Finally, we will setup CI for our native image and upload artifacts.
Create a new GitHub repository and configure your project to use GitHub repository as remote.
Next, create a new file ci.yml under .github/workflows directory and populate with following content.
name: jp CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
jobs:
build:
name: jp on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
timeout-minutes: 10
steps:
- uses: actions/checkout@v2
- uses: graalvm/setup-graalvm@v1
with:
version: 'latest'
java-version: '17'
components: 'native-image'
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Build with Gradle
uses: gradle/gradle-build-action@67421db6bd0bf253fb4bd25b31ebb98943c375e1
with:
arguments: nativeBuild
- name: Smoke test jp
run: |
./build/native/nativeCompile/jp
- name: Upload binary
uses: actions/upload-artifact@v2
with:
name: jp-${{ matrix.os }}
path: build/native/nativeCompile/jp*
This GitHub workflow will build on all Windows, Linux, and Mac containers and upload the artifacts. You can download from here.
Download the one for your operating system. For Mac, you will have to make the jp executable.
sudo chmod +x jp
Next, you can copy to path that is already on your PATH.
cp jp /usr/local/bin
Now, you can run jp command.
curl https://jsonplaceholder.typicode.com/posts | jp '$.length()'
100
GitHub Repository
All code is available on GitHub https://github.com/shekhargulati/jp
Shekhar Gulati 