Skip to content

Add serverless Scala JAR job example#925

Open
imran-a-hasan wants to merge 5 commits into
databrickslabs:mainfrom
imran-a-hasan:serverless-jar-job-example
Open

Add serverless Scala JAR job example#925
imran-a-hasan wants to merge 5 commits into
databrickslabs:mainfrom
imran-a-hasan:serverless-jar-job-example

Conversation

@imran-a-hasan

@imran-a-hasan imran-a-hasan commented Jul 22, 2026

Copy link
Copy Markdown

A minimal, self-contained Scala JAR job that runs on Databricks serverless compute, packaged as a Databricks Asset Bundle. It's meant as a copy-and-go starting point — the bundle builds the JAR with sbt assembly, uploads it, and defines a serverless JAR job.

The example demonstrates three basic building blocks:

  • Spark basics — get the session, build a DataFrame, read a Unity Catalog table (samples.nyctaxi.trips).
  • UDF basics — scalar, map-returning, SQL-registered, and a UDF closing over a locally-defined class (exercises closure serialization).
  • dbutils basics — the job/notebook context and a dbutils.fs read/write round-trip against a UC volume.

Serverless specifics it follows: Spark Connect session via SparkSession.builder().getOrCreate() (no SparkContext/RDD), builds against databricks-connect + databricks-dbutils-scala (both % provided), targets Scala 2.13.16 / JDK 17, serverless environment client: "5".

Deploy: databricks bundle deploy -t dev then databricks bundle run -t dev serverless-jar-job-example. Verified end-to-end on serverless — the run completes with ServerlessJarJobExample: all examples completed successfully.

See serverless-jar-job-example/README.md for full docs.

@imran-a-hasan
imran-a-hasan requested a review from a team as a code owner July 22, 2026 21:26
@CLAassistant

CLAassistant commented Jul 22, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants