Bump jruby 10.0 logstash 8.19 by andsel · Pull Request #19291 · elastic/logstash

andsel · 2026-07-10T11:44:19Z

Release notes

Update the bundled JRuby to version 10.0.6 including all the necessary updates that were applied on the 9.4 to make it work.

What does this PR do?

This PR backports ecadb4a to 8.19 with some adaptations, that make it non clean:

in 8.19 we don't have batch metrics
BufferedTokenizerExt is not a wrapped around the pure Java BufferedTokenizer but uses JRuby arrays
CompiledPipeline doesn't have the chunker

Description of commits:

commit is the cherry-pick with some adaptations related to deprecated JRuby methods.
commit fix remaining deprecations introduced by JRuby 10.0.
commit renames lockfile from Ruby 3.1 to 3.4
commit pin the Bundler version shipped with JRuby 10.0 (2.7.2 instead of 2.6.3).
commit regenerates NOTICE.TXT
commit copy the regenerated lock file to the release
commit backport pin multi_json to 1.19.1 since 1.20.0 has java variant with newer concurrent-ruby pin #18977 pinning multi_json and updating the lockfile
commit remove pins of gems now shipped with JRuby, but keep uri one because else Bundler would select 0.12.5
commit fixes a failing test due to spacing in verification of credentials logging. Same code was changed in behaviour on main in Ensure plugin config marked :deprecated logs to deprecation logger #16833.
commit fixes a missed update of version 3.1.0 to 3.4.0.
commmit backport the fix for Logstash termination contained in fix forced-shutdown (double SIGINT) behaviour for JRuby 10 #19017.
commit backport default to jdk21 on all ci/build tasks #16464 to default to JDK 21 for CI.
commit backport upgrade JRuby to 10.0.4.0 #18856 to update to JRuby 10.0.4.0
commit backport Update jruby to 10.0.5.0 #18965 to update to JRuby 10.0.5.0
commit backport Update jruby to 10.0.6.0 #19264 to update to JRuby 10.0.6.0

Why is it important/What is the impact to the user?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files (and/or docker env variables)
I have added tests that prove my fix is effective or that my feature works

Author's Checklist

verify BK pipeline logstash-exhaustive-tests-pipeline has at least the same errors as in main
this PR test https://buildkite.com/elastic/logstash-supported-plugins-test-pipeline/builds/64
main branch https://buildkite.com/elastic/logstash-supported-plugins-test-pipeline/builds/65

How to test this PR locally

Related issues

Closes Migrate JRuby to 10.x. #19303

github-actions · 2026-07-10T11:44:27Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
run exhaustive tests : Run the exhaustive tests Buildkite pipeline.

cla-checker-service · 2026-07-10T11:44:30Z

💚 CLA has been signed

mergify · 2026-07-10T11:50:36Z

This pull request does not have a backport label. Could you fix it @andsel? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
If no backport is necessary, please add the backport-skip label

andsel · 2026-07-14T15:11:11Z

run exhaustive tests

github-actions · 2026-07-14T15:41:29Z

TL;DR

All 10 jobs in exhaustive build 4100 fail before acceptance tests at :logstash-core:compileJava. The retained artifacts show 3037 total compiler errors, but omit the actual error diagnostics, so the root cause cannot be identified reliably from the available logs.

Remediation

Re-run one representative job or publish the complete compileJava output, including the first error: / cannot find symbol diagnostics before the retained warning tail.
Preserve the Java/JRuby toolchain versions and dependency classpath from the failing job; the current tail is insufficient to distinguish a source/API regression from a JRuby artifact or JDK mismatch.
After the first compiler error is available, validate with ./gradlew :logstash-core:compileJava before rerunning the full acceptance matrix.

Investigation details

Root Cause

Inconclusive from the captured logs. Every OS variant reaches :logstash-core:compileJava and exits with status 1, but each retained log begins in the middle of warning output and ends with 100 errors / 3037 total; no actionable compiler error, source location, or missing symbol is included.

Evidence

Build: https://buildkite.com/elastic/logstash-exhaustive-tests-pipeline/builds/4100
Jobs: all 10 listed in /tmp/gh-aw/buildkite-failures.txt (Ubuntu, Debian, RHEL/Oracle/Rocky/Alma, and openSUSE).
Representative log: logstash-exhaustive-tests-pipeline-ubuntu-2204.txt:109-129 reports 100 errors, 3037 total, and :logstash-core:compileJava FAILED.
The same compile failure summary appears in all ten retained logs.

Verification

No repository tests were run because compilation fails before tests start and the retained artifacts do not contain enough diagnostic detail to reproduce or attribute the failure. No matching open issue labeled flaky-test was found.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

andsel · 2026-07-15T12:06:56Z

run exhaustive tests

andsel · 2026-07-22T10:14:22Z

The vulnerability check failure https://github.com/elastic/logstash/actions/runs/29907347910/job/88881842211?pr=19291 due to GHSA-574f-3g2m-x479 which is critical . It's related to Bouncy Castle and GOST 28147 which, to my knowledge Logstash doesn't use. However, this PR includes 2 bcprov-jdk18on :

./vendor/bundle/jruby/3.4.0/gems/jruby-openssl-0.16.1-java/lib/org/bouncycastle/bcprov-jdk18on/1.84/bcprov-jdk18on-1.84.jar
./vendor/jruby/lib/ruby/stdlib/org/bouncycastle/bcprov-jdk18on/1.81/bcprov-jdk18on-1.81.jar

So Logstash override the one bundled with JRuby forcing jruby-openssl-0.16.1-java.

With subsequent update to JRuby 10.0.6 (which bundled jruby-openssl-0.16.0 and bcprov-jdk18on-1.84.jar) the issue goes away.

Move Logstash from JRuby 9.4 / Ruby 3.1 to JRuby 10.0.3.0 / Ruby 3.4. Adjust the runtime integration for the new JRuby APIs: - update Java/Ruby bridge code for events, timestamps, codecs, metrics, queues, and pipeline execution - align logging and monitoring behavior with the new runtime - update packaged runtime paths and Ruby 3.4 stdlib gem dependencies Fix bootstrap and plugin-management flows affected by the upgrade: - patch Bundler self-manager behavior in the embedded environment - ensure QA and plugin install paths apply the same Bundler patching - keep offline pack and packaged install workflows working Refresh tests for the changed integration points so startup, plugin management, and operational APIs continue to work after the runtime upgrade.

… else Bundler would select 0.12.5

…9017)

* Update jruby to 10.0.6.0 * Update array instantiation to new pattern

Copilot

Pull request overview

This PR upgrades Logstash 8.19’s bundled JRuby to 10.0.6.0 and updates the surrounding Java/Ruby integration code, gem/bundler packaging, licensing artifacts, and CI configuration to stay compatible with JRuby 10 / Ruby 3.4 semantics.

Changes:

Bump bundled JRuby and shift vendored Ruby ABI paths from jruby/3.1.0 to jruby/3.4.0, including updated lockfiles and gem dependencies.
Update JRuby-extension Java code and tests to use JRuby 10 APIs (ThreadContext-aware calls, Create/Convert/Define/Access helpers, numeric conversions, etc.).
Refresh NOTICE/licensing resources and adjust CI/build scripts (notably defaulting CI Java tooling to JDK 21 in several places).

Reviewed changes

Copilot reviewed 100 out of 104 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
x-pack/lib/filters/geoip/database_manager.rb	Replace deprecated `exists?` call
versions.yml	Bump JRuby version + sha256
tools/release/bump_plugin_versions.rb	Use `Gemfile.jruby-3.4.lock.release`
tools/dependencies-report/src/main/resources/notices/tempfile-NOTICE.txt	Add license notice
tools/dependencies-report/src/main/resources/notices/syslog-NOTICE.txt	Add license notice
tools/dependencies-report/src/main/resources/notices/syntax_suggest-NOTICE.txt	Add license notice
tools/dependencies-report/src/main/resources/notices/resolv-NOTICE.txt	Add license notice
tools/dependencies-report/src/main/resources/licenseMapping.csv	Map new gem licenses
spec/unit/bootstrap/bundler_spec.rb	Add bundler self-manager test; update fetcher error type
rubyUtils.gradle	QA bundling uses LogStash bundler patch; update gem dir
qa/integration/specs/plugin_name_log_spec.rb	Relax log regex for spacing changes
qa/integration/specs/pipeline_log_spec.rb	Make log regex whitespace/format tolerant
qa/integration/specs/monitoring_api_spec.rb	Adjust logger expectations; skip license reader
qa/integration/specs/mixed_codec_spec.rb	Adjust timing/log assertions for fast exit
qa/integration/specs/fatal_error_spec.rb	Escape dots and allow whitespace in log regex
qa/integration/specs/direct_heap_allocator_flag_spec.rb	Relax parsing regex for memory dump logs
qa/integration/specs/deprecation_log_spec.rb	Escape dots and allow whitespace in deprecation logs
qa/integration/rspec.rb	Update GEM_HOME path to jruby/3.4.0
qa/integration/integration_tests.gemspec	Add bigdecimal dev dependency
qa/integration/fixtures/plugins/generate-gems.sh	Prefer vendored JRuby; clear GEM_* env
qa/Gemfile	Add base64 gem
qa/docker/shared_examples/xpack.rb	Update expected log output format
NOTICE.TXT	Regenerated NOTICE to match new deps
logstash-core/src/test/java/org/logstash/ValuefierTest.java	Update Java proxy class API usage
logstash-core/src/test/java/org/logstash/RubyfierTest.java	ThreadContext-aware conversions; BigDecimal changes
logstash-core/src/test/java/org/logstash/plugins/MetricTestCase.java	Use `Create` + context-aware intern/asLong
logstash-core/src/test/java/org/logstash/plugins/factory/PluginFactoryExtTest.java	Use `Create.newEmptyArray`
logstash-core/src/test/java/org/logstash/log/RubyBasicObjectSerializerTest.java	Context-aware RubyArray operations
logstash-core/src/test/java/org/logstash/instrument/metrics/MetricExtFactory.java	Context-aware `intern`
logstash-core/src/test/java/org/logstash/instrument/metrics/gauge/ListGaugeTest.java	Update expected JRuby 10 hash formatting
logstash-core/src/test/java/org/logstash/ext/JrubyTimestampExtLibraryTest.java	Context-aware numeric conversions; time ctor
logstash-core/src/test/java/org/logstash/execution/ObservedExecutionTest.java	Use `Create.newArray` for namespace array
logstash-core/src/test/java/org/logstash/EventTest.java	Context-aware `substr` call
logstash-core/src/test/java/org/logstash/config/ir/RubyEnvTestCase.java	Update jruby bundle path to 3.4.0
logstash-core/src/test/java/org/logstash/config/ir/PluginConfigNameMethodDouble.java	Use `Define` and context-aware defineMethods
logstash-core/src/test/java/org/logstash/config/ir/PipelineConfigTest.java	Use `Access/Create` and context-aware intern
logstash-core/src/test/java/org/logstash/config/ir/IRHelpers.java	Use `Create.newArray` for SWM arrays
logstash-core/src/test/java/org/logstash/config/ir/EventConditionTest.java	Use `Create.newArray` for batches
logstash-core/src/test/java/org/logstash/config/ir/compiler/PluginDelegatorTestCase.java	Context-aware `intern/asLong`; `Create.allocArray`
logstash-core/src/test/java/org/logstash/config/ir/compiler/OutputDelegatorTest.java	`Define/Create` updates for JRuby 10
logstash-core/src/test/java/org/logstash/config/ir/compiler/DatasetCompilerTest.java	Use `Create.newArray` for batches
logstash-core/src/test/java/org/logstash/config/ir/CompiledPipelineTest.java	Use `Create` and shared ThreadContext
logstash-core/src/main/java/org/logstash/util/UtilExt.java	RaiseException-based type errors (JRuby 10)
logstash-core/src/main/java/org/logstash/RubyUtil.java	Switch to JRuby 10 `Define`/context-aware defineMethods
logstash-core/src/main/java/org/logstash/RubyJavaIntegration.java	Context-aware `defineMethods` and proxy lookups
logstash-core/src/main/java/org/logstash/Rubyfier.java	Use `Create.allocArray` when building Ruby arrays
logstash-core/src/main/java/org/logstash/plugins/RootMetricImpl.java	Context-aware intern for metric keys
logstash-core/src/main/java/org/logstash/plugins/NamespacedMetricImpl.java	Context-aware intern/to_s calls
logstash-core/src/main/java/org/logstash/plugins/factory/RubyCodecDelegator.java	Context-aware class lookup + block invocation
logstash-core/src/main/java/org/logstash/plugins/factory/PluginMetricsFactoryExt.java	Context-aware intern usage
logstash-core/src/main/java/org/logstash/plugins/factory/PluginFactoryExt.java	Context-aware intern for plugin ids
logstash-core/src/main/java/org/logstash/plugins/CounterMetricImpl.java	Context-aware intern for metric key
logstash-core/src/main/java/org/logstash/ObjectMappers.java	JRuby 10-safe numeric conversions/serializers
logstash-core/src/main/java/org/logstash/Logstash.java	Handle JRuby SystemExit in uncaught handler; update production error detection
logstash-core/src/main/java/org/logstash/log/SlowLoggerExt.java	RaiseException-based type errors; numeric conversions
logstash-core/src/main/java/org/logstash/log/RubyBasicObjectSerializer.java	Context-aware `to_s` and inspect-method lookup
logstash-core/src/main/java/org/logstash/log/LoggableExt.java	Use `defineMethods`; update logger name derivation
logstash-core/src/main/java/org/logstash/Javafier.java	JRuby 10-safe numeric conversions
logstash-core/src/main/java/org/logstash/instrument/metrics/NullNamespacedMetricExt.java	Context-aware `op_plus`
logstash-core/src/main/java/org/logstash/instrument/metrics/NamespacedMetricExt.java	Context-aware `op_plus`
logstash-core/src/main/java/org/logstash/instrument/metrics/MetricExt.java	JRuby 10-safe numeric conversion
logstash-core/src/main/java/org/logstash/ext/JRubyWrappedWriteClientExt.java	Use `Create` for arrays; context-aware intern
logstash-core/src/main/java/org/logstash/ext/JrubyWrappedSynchronousQueueExt.java	Convert size via `Convert.toInt`
logstash-core/src/main/java/org/logstash/ext/JrubyTimestampExtLibrary.java	Remove Arity.scanArgs dependency; Convert boolean/int; RaiseException
logstash-core/src/main/java/org/logstash/ext/JrubyEventExtLibrary.java	RaiseException type errors; Convert booleans; Create arrays
logstash-core/src/main/java/org/logstash/execution/ShutdownWatcherExt.java	Convert ints via `Convert`; num2long
logstash-core/src/main/java/org/logstash/execution/QueueReadClientBase.java	Convert batch dimensions/metrics via `Convert`
logstash-core/src/main/java/org/logstash/execution/PipelineReporterExt.java	Create arrays via `Create`; Convert ints; context-aware intern/append
logstash-core/src/main/java/org/logstash/execution/MemoryReadBatch.java	Use `Create.allocArray` and context-aware append
logstash-core/src/main/java/org/logstash/execution/AbstractPipelineExt.java	Context-aware pipeline id, arrays, and numeric conversions
logstash-core/src/main/java/org/logstash/ackedqueue/QueueFactoryExt.java	Compute memory queue size with `Convert.toInt`
logstash-core/src/main/java/org/logstash/ackedqueue/ext/JRubyWrappedAckedQueueExt.java	Arity scan with context; Convert booleans/ints
logstash-core/src/main/java/org/logstash/ackedqueue/ext/JRubyAckedQueueExt.java	Convert ints; Convert booleans
logstash-core/src/main/java/org/logstash/ackedqueue/AckedReadBatch.java	Use `Create.allocArray` and context-aware append
logstash-core/spec/logstash/util/plugin_version_spec.rb	Stub gemspec lookup for JRuby 10 changes
logstash-core/spec/logstash/plugin_spec.rb	Avoid `expect_any_instance_of`; adjust milestone test
logstash-core/spec/logstash/java_pipeline_spec.rb	Update expected error text for JRuby 10
logstash-core/spec/logstash/event_spec.rb	Update NoMethodError quoting format
logstash-core/spec/logstash/config/mixin_spec.rb	Adjust formatting expectations around `=>`
logstash-core/spec/logstash/api/modules/logging_spec.rb	Add coverage for live loggers in API
logstash-core/logstash-core.gemspec	Add runtime deps (observer/csv); pin multi_json
logstash-core/lib/logstash/api/modules/logging.rb	Expand logger listing logic
lib/pluginmanager/gem_installer.rb	Update GEM_HOME path to jruby/3.4.0
lib/bootstrap/bundler.rb	Update lockfile name; patch bundler self-manager; platform filtering change
Gemfile.template	Update uri pin; comment out some default gem pins
Gemfile.jruby-3.4.lock.release	New lockfile for JRuby/Ruby 3.4; bundler 2.7.2
ci/observabilitySREsmoke_tests.sh	Whitespace/format-only change
build.gradle	Update GEM_HOME/GEM_PATH to jruby/3.4.0
bin/logstash.lib.sh	Update GEM_HOME default to jruby/3.4.0
.ruby-version	Update local JRuby version selector
.github/workflows/critical_vulnerability_scan.yml	Update checkout action; set up JDK 21
.ci/updatecli/bump-logstash-version.yml	Track new lockfile name
.buildkite/windows_jdk_matrix_pipeline.yml	Narrow JDK matrix to 21 variants
.buildkite/scripts/setup_java.sh	Install OpenJDK 21 on CI images
.buildkite/linux_jdk_matrix_pipeline.yml	Narrow JDK matrix to 21 variants

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+            // Fallback to getBaseName for compatibility
+            return (metaClass.getBaseName() != null && metaClass.getBaseName().equals("Bundler::ProductionError"));
        }


+      # Generator process may exit before log file is fully flushed; check if file exists before reading
+      if File.exist?(plainlog_file)
+        logs = IO.read(plainlog_file)
+        expect(logs).to_not include("ERROR")
+      else


infra-vault-gh-plugin-prod · 2026-08-14T13:14:23Z

💚 Build Succeeded

Buildkite Build
Commit: 5e6bbc9

History

💚 Build #5096 succeeded 5ac1cff
💚 Build #5092 succeeded 620edad
💚 Build #5076 succeeded a305de1
💛 Build #5071 was flaky 5e791f3
💛 Build #5070 was flaky ef40b7f
💔 Build #5069 failed 60dd822

cc @andsel

andsel · 2026-08-14T13:30:48Z

run exhaustive tests

andsel self-assigned this Jul 10, 2026

andsel changed the base branch from main to 8.19 July 10, 2026 11:45

This comment has been minimized.

Sign in to view

andsel force-pushed the bump_jruby_10.0_logstash_8.19 branch from 1078e7f to a305de1 Compare July 15, 2026 10:13

andsel marked this pull request as ready for review July 16, 2026 09:42

andsel mentioned this pull request Jul 20, 2026

Avoid to run on JDK < 21 #19311

Merged

2 tasks

andsel force-pushed the bump_jruby_10.0_logstash_8.19 branch from a305de1 to 620edad Compare July 22, 2026 09:17

andsel mentioned this pull request Jul 22, 2026

[Meta] Update JRuby to 10 on branch 8.19 #19164

Open

12 tasks

jsvd and others added 7 commits August 14, 2026 14:45

Fixed deprecation warnings

529e018

Moved lock file from Ruby 3.1 to 3.4

f124ac9

Fixed version of Bundler. JRuby 10.0.3 ships with 2.7.2

a75ae3e

Regenerate NOTICE.TXT

684f9ad

Updated gemfile lock

31e6f26

Backport elastic#18977 pinning multi_json and updating the lockfile

238cf4a

andsel and others added 8 commits August 14, 2026 14:45

Removed pins of gems now shipped with JRuby, but kept uri one becuase…

ed5329e

… else Bundler would select 0.12.5

Fixed failing test due to spacing

0c43997

Fixed leftover of 3.1.0 version

e6f419b

fix forced-shutdown (double SIGINT) behaviour for JRuby 10 (elastic#1…

a70cbba

…9017)

default to jdk 21 on all ci/build tasks (elastic#16464)

724120e

upgrade jruby to 10.0.4.0 (elastic#18856)

3428efd

Update jruby to 10.0.5.0 (elastic#18965)

1ffdf8b

Update jruby to 10.0.6.0 (elastic#19264)

5e6bbc9

* Update jruby to 10.0.6.0 * Update array instantiation to new pattern

andsel force-pushed the bump_jruby_10.0_logstash_8.19 branch from 5ac1cff to 5e6bbc9 Compare August 14, 2026 12:47

Copilot AI lite review requested due to automatic review settings August 14, 2026 12:47

Copilot started reviewing on behalf of andsel August 14, 2026 12:48 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

Uh oh!

Conversation

andsel commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release notes

What does this PR do?

Why is it important/What is the impact to the user?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Uh oh!

github-actions Bot commented Jul 10, 2026

🤖 GitHub comments

Uh oh!

cla-checker-service Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify Bot commented Jul 10, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

andsel commented Jul 14, 2026

Uh oh!

github-actions Bot commented Jul 14, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Uh oh!

andsel commented Jul 15, 2026

Uh oh!

andsel commented Jul 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

infra-vault-gh-plugin-prod Bot commented Aug 14, 2026

💚 Build Succeeded

History

Uh oh!

andsel commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andsel commented Jul 10, 2026 •

edited

Loading

cla-checker-service Bot commented Jul 10, 2026 •

edited

Loading