Skip to content

fix(datasets): Set up Java 17 for Windows spark jobs - #1554

Open
lrcouto wants to merge 1 commit into
mainfrom
fix-datasets-build-failure
Open

lrcouto wants to merge 1 commit into
mainfrom
fix-datasets-build-failure

Conversation

@lrcouto

@lrcouto lrcouto commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Description

The nightly kedro-datasets unit tests fail on Ubuntu whenever a job lands on the new ubuntu-26.04 runner image, which ubuntu-latest is gradually moving to.

The workflow doesn't pin a JDK, so Spark uses the image's newer default Java (24+). On those versions, Hadoop's UserGroupInformation.getCurrentUser throws UnsupportedOperationException: getSubject is not supported, the Spark JVM never starts, and 4 doctests fail (ManagedTableDataset, DeltaTableDataset, SparkDataset, SparkDatasetV2). The same failure happened in 4 attempts in, so it isn't flakiness.

Development notes

  • Added an actions/setup-java@v4 step (Temurin 17) to .github/workflows/unit-tests.yml. It only runs for kedro-datasets on Linux, since Windows runs test-no-spark.
  • To verify: check that the Ubuntu datasets jobs pass on an ubuntu-26.04 runner.

Signed-off-by: Laura Couto <laurarccouto@gmail.com>
@lrcouto lrcouto changed the title Set up Java 17 for Windows spark jobs fix(datasets): Set up Java 17 for Windows spark jobs Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants