Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions reference/components/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ Extensions require an `extensionModule` option pointing to the extension source.
| [`loadEnv`](../environment-variables/overview.md) | Load environment variables from `.env` files |
| [`rest`](../rest/overview.md) | Enable automatic REST endpoint generation |
| [`roles`](../users-and-roles/overview.md) | Define role-based access control from YAML files |
| [`scheduler`](./scheduler.md) | Run recurring jobs on a cron or interval schedule |
| [`static`](../static-files/overview.md) | Serve static files via HTTP |

## Known Custom Components
Expand Down
105 changes: 105 additions & 0 deletions reference/components/scheduler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
title: Scheduler
---

# Scheduler

<VersionBadge version="v5.2.0" />

The scheduler is a built-in plugin that runs recurring jobs declared in a component's configuration. Harper invokes a designated export from your component on a cron or interval schedule, and in a cluster each job fires exactly once per occurrence - on a single, automatically elected node - rather than once per node or worker.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The companion implementation has no distributed lock/CAS and explicitly allows duplicate delivery during failover, DST fall-back, and split-brain recovery. Calling this "exactly once per occurrence" contradicts the idempotency warning below and may lead users to omit deduplication. Describe this as leader-coordinated single execution under normal operation with possible duplicates, and update the same absolute claim in the 5.2 release notes.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in af1a837: the intro now describes leader-coordinated single execution under normal operation, with explicit duplicate-delivery cases (failover, DST fall-back, split-brain recovery) tied to the idempotency requirement. The 5.2 release-notes entry carried the same overstatement and is aligned too. (AI-generated response)


Use it for the maintenance passes applications otherwise hand-roll with `setInterval`: nightly cleanups, periodic re-aggregation, cache refreshes, digest generation.

## Configuration

In your component's `config.yaml`, use the `scheduler` key to declare jobs:

```yaml
scheduler:
jobs:
- name: nightly-cleanup
cron: '0 2 * * *'
timezone: America/New_York
handler: ./jobs.js#cleanupOldRecords
- name: refresh-summaries
interval: 90s
handler: ./jobs.js#refreshSummaries
```

Each job entry supports:

### `name`

Type: `string` (required)

A name for the job, unique within the component. Used in logs and to key the job's run state.

### `cron`

Type: `string`

A five-field cron expression: minute, hour, day-of-month, month, day-of-week. Supports `*`, lists (`1,15`), ranges (`9-17`), steps (`*/15`, `0-59/5`), month and day names (`JAN`, `MON-FRI`), and the common macros (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@yearly`). Day-of-week `0` and `7` both mean Sunday. Following the POSIX cron rule, when both day-of-month and day-of-week are restricted, the job runs when either matches.

There is no seconds field. Expressions that can never fire (such as `0 0 30 2 *` - February 30th) are rejected when the component loads.

Exactly one of `cron` or `interval` is required.

### `interval`

Type: `string` (duration)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The schema accepts interval as either a duration string or a number of seconds, so Type: string is incomplete. Runtime validation also rejects intervals longer than 365 days, but only the one-second minimum is documented. List both accepted types and both bounds so configurations do not fail unexpectedly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in af1a837: the field now reads 'Type: string (duration) or number (seconds)' and documents both bounds (1 second to 365 days). (AI-generated response)


A simple cadence instead of a cron expression: a number of seconds, or a value like `90s`, `5m`, `1h`, `1d`. Minimum one second. Use this for "every N" maintenance passes that do not align to wall-clock times.

### `timezone`

Type: `string`

An IANA timezone (for example `America/Chicago`) the cron expression is evaluated in. Defaults to the server's timezone. Only valid with `cron`.

During daylight-saving transitions: a job scheduled inside the spring-forward gap (a wall-clock time that does not exist that day) runs at the shifted instant rather than being skipped, and a job scheduled inside the fall-back overlap (a wall-clock time that occurs twice) runs once, at the first occurrence.

### `handler`

Type: `string` (required)

The module and export to invoke, as `<module path>#<named export>`, relative to the component directory - for example `./jobs.js#cleanupOldRecords`. Omit the `#...` suffix to use the module's default export. The module is loaded in your component's environment, so it has access to the same globals (`tables`, `databases`, `server`, and so on) as the rest of your component code.

A bad handler reference (missing module, missing export, non-function export) fails the component load at deploy time rather than failing silently at the first scheduled fire.

## Writing a Handler

The handler is called with a single context argument and may return a promise:

```javascript
export async function cleanupOldRecords(context) {
// context.jobName - the configured job name
// context.scheduledAt - the occurrence this run is for (Date)
// context.catchUp - true when this run is making up a missed occurrence
const cutoff = Date.now() - 30 * 24 * 60 * 60 * 1000;
Comment thread
jcohen-hdb marked this conversation as resolved.
Outdated
for await (const record of tables.Events.search([{ attribute: 'timestamp', comparator: 'less', value: cutoff }])) {
await tables.Events.delete(record.id);
}
}
```

**Handlers should be idempotent.** Harper's clustering has no distributed lock, so leadership failover and daylight-saving fall-back can occasionally deliver the same logical occurrence twice. Design handlers so that running twice for one occurrence is harmless.

Runs of the same job never overlap: if a run is still going when its next occurrence arrives, that occurrence is skipped (with a log entry) rather than stacked.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation does not consistently skip an occurrence when a handler outlasts its cadence. It arms the next timer only after the handler settles; an overdue interval then runs immediately, while the cron catch-up sweep can run the latest missed occurrence afterward. This matters because a slow interval handler can execute back-to-back. Either align the implementation with this statement or document the actual late/catch-up behavior.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in af1a837, documenting the implementation as-is: runs never overlap; missed time is not queued; an overdue interval job runs back-to-back after a slow handler; cron makes up at most its single most recent missed occurrence (context.catchUp = true). (AI-generated response)


## Cluster Behavior

One node in the cluster - the scheduler leader - runs all scheduled jobs; the others watch. Leader election is automatic and requires no configuration:

- The leader maintains a heartbeat in the replicated `system` database. If it stops heartbeating (crash, shutdown, partition) for more than five minutes, the next node in line promotes itself; with default timings, failover completes within about six and a half minutes.
- A restarting node defers to an actively heartbeating leader, so leadership is stable across deploys and restarts.
- On a single-node instance, that node simply runs the jobs.

### Missed Occurrences

If the most recent occurrence of a cron job was missed - the leader was down, a failover was in progress, or daylight-saving time skipped the slot - the scheduler fires one catch-up run for it (with `context.catchUp` set to `true`). Only the single most recent missed occurrence is made up, not every occurrence missed during an outage. Interval jobs resume their cadence from their last recorded run.

A newly deployed job waits for its first scheduled time; it does not fire immediately on deploy.

### Run State

Each job's last run time, status, duration, and last error (if any) are recorded in the `hdb_scheduler_state` system table, alongside the leader lease. This state replicates across the cluster so a newly promoted leader knows what has already run.
6 changes: 6 additions & 0 deletions release-notes/v5-lincoln/5.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ All patch release notes for 5.2.x are available on the [releases page](https://g

Vector searches combined with filters now evaluate the filter during HNSW graph traversal, so the query keeps exploring until it has enough matching nearest neighbors instead of post-filtering a fixed candidate set (which under-filled results under selective filters). Filters can come from query conditions, a JS-API `vectorFilter` function, or a record-scoped `allowRead` override — overriding `allowRead` on a table now makes it a row-level access check, evaluated per record with `this` bound to the record (closing the gap where a collection scan could return rows a single-record GET would deny). With it, a restricted user's vector search returns the k nearest records they are allowed to see. Very selective conditions automatically use an exact scan instead of graph traversal, and a `filterExpansion` visit budget bounds traversal cost. See [Vector Indexing](/reference/v5/database/schema#vector-indexing).

## Components

### Scheduler: Recurring Jobs from Component Config

Components can now declare recurring jobs in their configuration with a new built-in `scheduler` plugin. Jobs run on a five-field cron expression or a simple interval (`90s`, `5m`, `1h`), invoking a designated export from the component. In a cluster, each job fires exactly once per occurrence on an automatically elected leader node, with heartbeat-based failover, catch-up for missed occurrences, and per-job run state recorded in a replicated system table. See [Scheduler](/reference/v5/components/scheduler).

## Configuration

### Replicated `set_configuration`
Expand Down
5 changes: 5 additions & 0 deletions sidebarsReference.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,11 @@ const sidebars: SidebarsConfig = {
id: 'components/javascript-environment',
label: 'JavaScript Environment',
},
{
type: 'doc',
id: 'components/scheduler',
label: 'Scheduler',
},
{
type: 'doc',
id: 'components/nextjs',
Expand Down