Restructure examples into interactive/app/functions; add PTF examples - #47
Open
Cole Bailey (colebaileygit) wants to merge 2 commits into
Open
Restructure examples into interactive/app/functions; add PTF examples#47Cole Bailey (colebaileygit) wants to merge 2 commits into
Cole Bailey (colebaileygit) wants to merge 2 commits into
Conversation
Ports the internal FTAB-57 example rework to the public repo: - Split the flat examples.table package into three themed packages: interactive/ (inline, print-to-console learning examples), app/ (deployable reference apps submitted as long-running statements), and functions/ (shared PTF code used by both). - ProcessTableFunction example is now a pair sharing one ClickInactivityMonitor implementation: Example_10 (inline) and ReferenceApp_02 (deployable). The former deployment example becomes ReferenceApp_01_IntegrationAndDeployment. - Add Example_11_StatefulProcessTableFunction (stop/resume state durability), plus local (ProcessTableFunctionTestHarness) and Confluent Cloud tests. - Tooling: bin/run.sh (prefix-matched class resolution) and bin/jshell.sh; a .env / .env.template workflow for local runs; cloud.properties replaced by cloud.properties.template (copy it to cloud.properties, which stays git-ignored). - Update the shaded-jar mainClass and the deploy/manage workflow templates to the new FQNs. The artifact-management example (Example_12_ManagingIndependentArtifacts) is omitted for now because it requires a newer plugin than the public 2.3-3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR reorganizes the Table API Java examples into clearer “interactive” (inline learning programs) vs “app” (deployable, long-running reference apps) groupings, and adds new ProcessTableFunction (PTF) examples plus supporting local/unit and Confluent Cloud integration tests. It also updates local/dev tooling and configuration templates to align with the new structure.
Changes:
- Restructures example packages and renames/migrates existing examples into
interactive/andapp/. - Adds shared PTF implementation (
ClickInactivityMonitor) with new interactive and deployable examples plus tests. - Introduces local run tooling (
bin/run.sh,bin/jshell.sh) and refreshes docs/config templates for the new workflow.
Reviewed changes
Copilot reviewed 30 out of 33 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/java/io/confluent/flink/examples/table/Example_08_IntegrationAndDeploymentIT.java | Removes old integration test in the legacy examples.table package. |
| src/test/java/io/confluent/flink/examples/interactive/Example_10_ProcessTableFunctionTest.java | Adds local unit tests for the shared PTF via ProcessTableFunctionTestHarness. |
| src/test/java/io/confluent/flink/examples/interactive/Example_08_FunctionsTest.java | Moves/renames UDF unit tests into the interactive package and updates class references. |
| src/test/java/io/confluent/flink/examples/app/ReferenceApp_02_ProcessTableFunctionIT.java | Adds Confluent Cloud integration test for the deployable PTF reference app. |
| src/test/java/io/confluent/flink/examples/app/ReferenceApp_01_IntegrationAndDeploymentTest.java | Renames/moves the local pipeline unit test to match the new reference app class. |
| src/test/java/io/confluent/flink/examples/app/ReferenceApp_01_IntegrationAndDeploymentIT.java | Adds Confluent Cloud integration tests for the reworked deployable reference app. |
| src/main/resources/cloud.properties.template | Updates template to emphasize sql.current-catalog / sql.current-database and global API key usage. |
| src/main/java/io/confluent/flink/examples/TableProgramTemplate.java | Moves template to top-level package and switches to builder-style settings setup. |
| src/main/java/io/confluent/flink/examples/table/Example_11_ProcessTableFunction.java | Removes legacy PTF example from the old package. |
| src/main/java/io/confluent/flink/examples/table/Example_08_IntegrationAndDeployment.java | Removes legacy deployable example from the old package (replaced by reference app). |
| src/main/java/io/confluent/flink/examples/interactive/Example_11_StatefulProcessTableFunction.java | Renames/moves the stateful PTF example and updates configuration approach. |
| src/main/java/io/confluent/flink/examples/interactive/Example_10_ProcessTableFunction.java | Adds interactive inline PTF example wired to the shared ClickInactivityMonitor. |
| src/main/java/io/confluent/flink/examples/interactive/Example_09_StructuredObjects.java | Moves/renames structured objects example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/interactive/Example_08_Functions.java | Moves/renames functions example into interactive and updates settings/config guidance. |
| src/main/java/io/confluent/flink/examples/interactive/Example_07_Changelogs.java | Moves changelog example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/interactive/Example_06_ValuesAndDataTypes.java | Moves values/data types example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/interactive/Example_05_TablePipelines.java | Moves pipelines example into interactive and updates settings/config guidance. |
| src/main/java/io/confluent/flink/examples/interactive/Example_04_CreatingTables.java | Moves table creation example into interactive and updates settings/config guidance. |
| src/main/java/io/confluent/flink/examples/interactive/Example_03_TransformingTables.java | Moves transform example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/interactive/Example_02_UnboundedTables.java | Moves unbounded example into interactive and clarifies non-terminating behavior. |
| src/main/java/io/confluent/flink/examples/interactive/Example_01_CatalogsAndDatabases.java | Moves catalog/database example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/interactive/Example_00_HelloWorld.java | Moves hello-world example into interactive and updates settings builder usage. |
| src/main/java/io/confluent/flink/examples/functions/ClickInactivityMonitor.java | Introduces shared PTF implementation reused by interactive + deployable examples. |
| src/main/java/io/confluent/flink/examples/app/ReferenceApp_02_ProcessTableFunction.java | Adds deployable PTF reference app using ClickInactivityMonitor. |
| src/main/java/io/confluent/flink/examples/app/ReferenceApp_01_IntegrationAndDeployment.java | Adds deployable “integration and deployment” reference app in the new app package. |
| README.md | Updates docs for new structure, config workflow, scripts, and lifecycle semantics. |
| pom.xml | Updates shaded JAR mainClass to the new interactive FQN. |
| bin/run.sh | Adds a convenience runner that resolves example class names from the shaded JAR and loads .env. |
| bin/jshell.sh | Adds a JShell launcher that loads .env and ensures the shaded JAR exists. |
| .gitignore | Adds .envrc ignore and keeps local config files uncommitted. |
| .github/workflows-examples/manage.yml | Updates workflow to target the new reference app FQN. |
| .github/workflows-examples/deploy.yml | Updates workflow to deploy/list the new reference app FQN. |
| .env.template | Adds a template for local environment-variable based configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+69
to
72
| # --action.await makes stop/resume/delete block until the target phase so the exit code | ||
| # reflects the outcome; list and describe ignore it. | ||
| run: | | ||
| args=("$ACTION" --application-name "$APPLICATION_NAME" --action.await) |
Contributor
Author
There was a problem hiding this comment.
From what version is this changed?
- Update the build prerequisite from Java 11 to Java 17 (matches pom target).
- Remove the "Open Preview stage" notes and batch-mode limitation entries.
- Fix typos ("opinonated", "feature are").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| - `stop` waits until the statement reaches `STOPPED`. | ||
| - `delete` waits until the statement does not exist. | ||
|
|
||
| Tune the maximum wait by passing a duration directly to `--action.await` (e.g. `--action.await 10min`, default: |
Member
There was a problem hiding this comment.
There seems to be a mismatch here. The current docs here are correct already, the option is called --action.await.
| - many configuration options | ||
| - limited SQL syntax | ||
|
|
||
| ### Statement Management |
Member
There was a problem hiding this comment.
Can we keep this part in the docs? I think having a place where we directly state all the limitations and constraints makes sense.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirrors the internal FTAB-57 example rework into the public repo.
examples.tablepackage intointeractive/(inline, print-to-console learning examples),app/(deployable reference apps submitted as long-running statements), andfunctions/(shared PTF code used by both).ClickInactivityMonitorimplementation:Example_10(inline) andReferenceApp_02(deployable). The former deployment example becomesReferenceApp_01_IntegrationAndDeployment.Example_11_StatefulProcessTableFunction(stop/resume state durability), plusProcessTableFunctionTestHarnessunit tests and Confluent Cloud integration tests.bin/run.sh(prefix-matched class resolution) andbin/jshell.sh; a.env/.env.templateworkflow for local runs;cloud.propertiesreplaced bycloud.properties.template.mainClassand the deploy/manage workflow templates to the new FQNs. Stays on the public plugin 2.3-3.Deferred: the artifact-management example (
Example_12_ManagingIndependentArtifacts) is omitted for now — it requires a newer plugin than the public 2.3-3.Reviewer note: local configuration now uses
cloud.properties.template(copy it tocloud.properties, which stays git-ignored) instead of a committedcloud.properties.🤖 Generated with Claude Code