KNOX-3278: Update JLine to 3.30.6

[bonampak]

KNOX-3278 - Update jline to 3.30.6

What changes were proposed in this pull request?

Update groovy to 5.0.4, jline to 3.30.6, jna to 5.18.1, aspectj to 1.9.25.1 and rest-assured to 6.0.0.
Rewrote gateway-shell using JLine 3 and Groovy 5. As

How was this patch tested?

Ran example tests and all repl commands (ExampleWebHdfsLs.groovy and ExampleManagerResourceDeployment.groovy both in REPL with :load and as knoxshell.sh arguments. Both work as expected.)

Build:

mvn -Prelease,package clean install -am -pl gateway-shell,gateway-shell-release

Setup:

• Unzip the target/3.0.0-SNAPSHOT/knoxshell-3.0.0-SNAPSHOT.zip to a directory.
• For datasource testing, copied the Postgres JDBC driver postgresql-42.7.10.jar to knoxshell-3.0.0-SNAPSHOT/lib/
• Change localhost in the samples groovy scripts to your cluster:

export KNOX_GATEWAY_HOST="example-knox-gateway-host"
export KNOX_GATEWAY_PORT="8443"

sed -i -e "s/localhost/${KNOX_GATEWAY_HOST}/g" samples/Example*.groovy

• Build the gateway truststore in your home directory:

./bin/knoxshell.sh buildTrustStore "https://${KNOX_GATEWAY_HOST}:${KNOX_GATEWAY_PORT}"

Tests

Logging

./bin/knoxshell.sh -e "org.apache.logging.log4j.LogManager.getLogger('test').error('hello from knoxshell')"
cat ./logs/knoxshell.log
2026-08-18T13:08:47,990 ERROR test (IndyInterface.java:fromCache(344)) - hello from knoxshell

Datasources command

Prerequisites:

• copied postgresql-42.7.10.jar to knoxshell-3.0.0-SNAPSHOT/lib
• created a local postgres db with docker.

./bin/knoxshell.sh
Apache Knox Shell
Type ':help' (':h' or '?') for help, ':exit' or ':quit' (':x' or ':q') to quit.
knox> :ds add ds-postgres jdbc:postgresql://localhost:15432/knoxshell_test_db org.postgresql.Driver basic
Knox DataSources
+----------------+--------------------------------------------------------+--------------------------+--------------+
|      Name      |                     Connect String                     |          Driver          |  Authn Type  |
+----------------+--------------------------------------------------------+--------------------------+--------------+
|  ds-postgres   |  jdbc:postgresql://localhost:15432/knoxshell_test_db   |  org.postgresql.Driver   |    basic     |
+----------------+--------------------------------------------------------+--------------------------+--------------+

knox> :ds select ds-postgres
Knox DataSource Selected
+----------------+--------------------------------------------------------+--------------------------+--------------+
|      Name      |                     Connect String                     |          Driver          |  Authn Type  |
+----------------+--------------------------------------------------------+--------------------------+--------------+
|  ds-postgres   |  jdbc:postgresql://localhost:15432/knoxshell_test_db   |  org.postgresql.Driver   |    basic     |
+----------------+--------------------------------------------------------+--------------------------+--------------+

knox> :sql

(insert query to popup)

SELECT 1 as one;
+--------+
|  one   |
+--------+
|   1    |
+--------+

fs command

knox> :fs mount https://${KNOX_GATEWAY_HOST}:${KNOX_GATEWAY_PORT}/gateway/sandbox myfs
(enter username and password in pop-up dialog window)
https://${KNOX_GATEWAY_HOST}:${KNOX_GATEWAY_PORT}/gateway/sandbox mounted as myfs


knox> :fs mounts
+----------------+--------------------------------------------------------------------------------+
|  Mount Point   |                                  Topology URL                                  |
+----------------+--------------------------------------------------------------------------------+
|      myfs      |  https://${KNOX_GATEWAY_HOST}:${KNOX_GATEWAY_PORT}/gateway/sandbox             |
+----------------+--------------------------------------------------------------------------------+


:fs ls /myfs/tmp/

knox> :fs ls /myfs/tmp/
/tmp/
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+
|  permission  |  owner   |    group     |  length  |             modtime              |                    name                    |
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+
|      0       |   hdfs   |  supergroup  |    0     |  Tue Aug 18 15:35:25 CEST 2026   |  .file                                    |
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+


knox> :fs put /tmp/testing/example-file.txt /myfs/tmp/example-file.txt
Successfully put: /tmp/testing/example-file.txt to: /myfs/tmp/example-file.txt

knox> :fs ls /myfs/tmp/
/tmp/
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+
|  permission  |  owner   |    group     |  length  |             modtime              |                    name                    |
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+
|      0       |   hdfs   |  supergroup  |    0     |  Tue Aug 18 15:43:25 CEST 2026   |              .file                         |
|     755      |   tom    |  supergroup  |    14    |  Tue Aug 18 15:41:16 CEST 2026   |              example-file.txt              |
+--------------+----------+--------------+----------+----------------------------------+--------------------------------------------+

knox> :fs cat /myfs/tmp/example-file.txt
demo

Invoking groovy scripts with knoxshell

./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
Enter username: tom
Enter password:
[tmp, user]

./bin/knoxshell.sh samples/ExampleManagerListResources.groovy
Enter username: tom
Enter password:
Provider Configurations
  • manager
  • admin
  • knoxsso
Descriptors
...

./bin/knoxshell.sh samples/ExampleManagerResourceDeployment.groovy
Enter username: tom
Enter password:

Existing Provider Configurations
  • kerberos-providers
  • manager.groovy
...

• ExampleWebHdfsPutGet.groovy
  Prerequisite: hdfs home directory for your user exists on the cluster.

kinit -kt <keytab> <hdfs-kerberos-principal>
hdfs dfs -mkdir -p /user/tom
hdfs dfs -chown tom:tom /user/tom

./bin/knoxshell.sh samples/ExampleWebHdfsPutGet.groovy

Enter username: tom
Enter password:
*********************************************************************
KnoxShell README
*********************************************************************
Client Details
...

On the cluster, verified the put was successful:

hdfs dfs -ls /user/tom/example
Found 1 items
-rwxrwxrwx   2 tom tom       1452 2026-08-18 13:10 /user/tom/example/README

Legacy Groovysh commands

Commands skipped (not applicable to CLI/JLine3 architecture and current feature set):
:display, :inspect, :edit, :record, :alias, :set, :register, :doc, :grab

Also not implemented:
:save / :s — dump session history/buffer to a file
:history (jline3 history file will be in ~/.knoxshell_history)

Implemented:

:import, :show, :load, :purge (also for old :clear)

Tested filenames with spaces:

./bin/knoxshell.sh "samples/Example WebHdfs Ls.groovy"
./bin/knoxshell.sh samples/Example\ WebHdfs\ Ls.groovy

inside the shell, only the quoted one works:

knox> . "samples/Example WebHdfs Ls.groovy"

knox> . samples/Example\ WebHdfs\ Ls.groovy
Failed to load 'samples/Example': File not found: testing/knoxshell/knoxshell-3.0.0-SNAPSHOT/samples/Example
Failed to load 'WebHdfs': File not found: testing/knoxshell/knoxshell-3.0.0-SNAPSHOT/WebHdfs
Failed to load 'Ls.groovy': File not found: testing/knoxshell/knoxshell-3.0.0-SNAPSHOT/Ls.groovy
knox>

Integration Tests

There is another ticket for adding integration tests:
KNOX-3280 Add integration tests for KnoxShell and KnoxCLI

UI changes

None

[github-actions]

Test Results

46 tests   46 ✅  6s ⏱️
 3 suites   0 💤
 3 files     0 ❌

Results for commit 9230a8a.

♻️ This comment has been updated with latest results.

[bonampak]

if we use maven-shade-plugin:3.4.1 and org.apache.logging.log4j:log4j-transform-maven-shade-plugin-extensions, this can be fixed by adding
<transformer implementation="org.apache.logging.log4j.maven.plugins.shade.transformer.Log4j2PluginCacheFileTransformer"/>
Otherwise the Log4j2Plugins.dat will be overwritten by our plugins and will not contain the ones shipped by log4j2.

[pzampino]

Why the change from shell.Shell to launcher.Launcher?

[bonampak]

I kept Launcher and removed the duplicate classpath option.

It's because our logging configuration is referring to sys:launcher.dir and sys:launcher.name.
The shaded jar is also using Launcher as main class.

Either we specify these properties in the launcher script or we use Launcher.

Otherwise the log file will not be created:

./bin/knoxshell.sh -e "org.apache.logging.log4j.LogManager.getLogger('test').error('hello from knoxshell')"

main ERROR Unable to create file ${sys:launcher.dir}/../logs/${sys:launcher.name}.log java.io.IOException: No such file or directory at java.base/java.io.UnixFileSystem.createFileExclusively(Native Method) at java.base/java.io.File.createNewFile(File.java:1043) at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.lambda$getFileManager$0(RollingFileManager.java:315)

There are currently classloader issues if we would use -jar and we place extra jars, for example database driver jars in the lib folder. So the -cp flag is needed in that case and -jar cannot be used (the shaded jar does not specify a classpath in its manifest). I'll create a separate issue to use launcher and be consistent with other modules.

[pzampino]

This appears to be a re-implementation of the credential collector logic in the name of the JLine upgrade? Is it necessary for the dependency upgrade?

[bonampak]

You're right, it's not necessary. Reverted back to KnoxLoginDialog.

[moresandeep]

Thanks @bonampak! Looks good