Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
keywords: [scheduled compaction, compaction cronjob, GreptimeDB Enterprise, Metasrv, plugin]
description: Configure scheduled compaction in GreptimeDB Enterprise and manage compaction jobs through the Metasrv HTTP API.
---

# Scheduled Compaction

Scheduled Compaction is a GreptimeDB Enterprise feature that periodically submits regular compaction requests for all physical Regions that have a leader in a cluster. It runs on the leader Metasrv, uses a cron expression and an IANA time zone to determine the schedule, and stores recent job reports in the Metasrv KV backend.

This feature complements GreptimeDB's automatic and [manual compaction](/user-guide/deployments-administration/manage-data/compaction.md#strict-window-compaction-strategy-swcs-and-manual-compaction). Use it when you want to compact all Regions regularly during a predictable maintenance window.

## Configure the plugin

Add the `compaction_cronjob` plugin to the Metasrv configuration:

```toml
[[plugins]]
[plugins.compaction_cronjob]
enable = true
timezone = "Asia/Shanghai"
cron = "0 0 0 * * *"
max_concurrent_regions = 4
compact_parallelism = 1
manual_trigger_cooldown_secs = 3600
max_history_jobs = 100
```

The cron expression includes seconds. For example, `0 0 0 * * *` runs at midnight every day in the configured time zone.

| Option | Default | Description |
| --- | --- | --- |
| `enable` | `false` | Enables scheduled compaction. |
| `timezone` | Local IANA time zone, or `UTC` if it cannot be detected | The time zone used to evaluate `cron`, for example `Asia/Shanghai`. |
| `cron` | `0 0 0 * * *` | The compaction schedule. |
| `max_concurrent_regions` | `4` | The maximum number of Region compaction requests submitted concurrently by one job. Must be greater than `0`. |
| `compact_parallelism` | `1` | The compaction parallelism passed to each Region request. Must be greater than `0`. |
| `manual_trigger_cooldown_secs` | `3600` | The minimum interval between a manual trigger and the most recent job, whether that job was scheduled or manually triggered. Set it to `0` to disable the cooldown. Scheduled jobs are not delayed by this option. |
| `max_history_jobs` | `100` | The maximum number of completed job reports retained in the Metasrv KV backend. Must be greater than `0`. |

Only the leader Metasrv runs the scheduler. After leadership changes, the new leader continues scheduling future jobs from the configured cron expression.

## HTTP endpoints

The plugin adds the following endpoints to the Metasrv HTTP server. Replace `<metasrv-http-address>` with the configured Metasrv HTTP address.

### Trigger a job

```bash
curl -X POST \
"http://<metasrv-http-address>/admin/compaction_cronjob/trigger"
```

Send this request to the leader Metasrv. The request waits for the job to finish submitting Region compaction requests and returns its report.

- Returns `409 Conflict` when sent to a follower Metasrv.
- Returns `429 Too Many Requests` when the manual trigger is still within `manual_trigger_cooldown_secs` of the most recent scheduled or manual job.
- Returns `500 Internal Server Error` for other failures.

### Get a job

```bash
curl --get \
--data-urlencode "job_id=2026-06-16T00:00:00+00:00" \
"http://<metasrv-http-address>/admin/compaction_cronjob/job"
```

The `job_id` is returned by the trigger and job-list endpoints. It is an RFC 3339 timestamp and should be URL encoded. The endpoint returns `404 Not Found` when the job does not exist.

### List recent jobs

```bash
curl --get \
--data-urlencode "limit=20" \
"http://<metasrv-http-address>/admin/compaction_cronjob/jobs"
```

Jobs are ordered by scheduled time in descending order. The optional `limit` parameter defaults to `20`.

## Job report

The trigger and query endpoints return JSON job reports. A completed successful job resembles:

```json
{
"job_id": "2026-06-16T00:00:00+00:00",
"scheduled_at": "2026-06-16T00:00:00Z",
"started_at": "2026-06-16T00:00:00.010Z",
"finished_at": "2026-06-16T00:00:01.010Z",
"status": "succeeded",
"execution_result": {
"total_regions": 8,
"submitted_regions": 8,
"failed_regions": []
}
}
```

Possible job states are `running`, `succeeded`, and `partial_failed`. A failed job carries its reason as `{"failed": "<reason>"}`. The execution result records how many Region requests were attempted and submitted, together with Region-level submission failures.
Comment thread
MichaelScofield marked this conversation as resolved.
Outdated
Comment thread
MichaelScofield marked this conversation as resolved.
Outdated
1 change: 1 addition & 0 deletions docs/enterprise/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ which are described in detail in the documentation in this section:
user activity with detailed audit logs.
- [Automatic region load balance](./autopilot/region-balancer.md): Auto balance
datanodes workload by moving regions between them.
- [Scheduled Compaction](./deployments-administration/scheduled-compaction.md): Periodically submit regular compaction requests for all physical Regions, with HTTP endpoints for manual triggers and job queries.
- [Elasticsearch query compatibility](./elasticsearch-compatible/overview.md): Use GreptimeDB backed Kibana for your logs.
- [Greptime Enterprise Management Console](./console-ui.md): An enhanced version of our dashboard UI,
carries more cluster management and monitoring features.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
keywords: [定时 Compaction, Compaction Cronjob, GreptimeDB 企业版, Metasrv, plugin]
description: 介绍如何在 GreptimeDB 企业版中配置定时 Compaction,并通过 Metasrv HTTP API 管理 Compaction 任务。
---

# 定时 Compaction

定时 Compaction 是 GreptimeDB 企业版功能,用于周期性地为集群中所有存在 leader 的物理 Region 提交 regular Compaction 请求。该功能运行在 leader Metasrv 上,根据 cron 表达式和 IANA 时区执行任务,并将最近的任务报告保存在 Metasrv KV backend 中。

该功能是 GreptimeDB 自动 Compaction 和[手动 Compaction](/user-guide/deployments-administration/manage-data/compaction.md)的补充。当你希望在固定的维护时间窗口内定期 Compaction 所有 Region 时,可以使用该功能。

## 配置 plugin

在 Metasrv 配置文件中添加 `compaction_cronjob` plugin:

```toml
[[plugins]]
[plugins.compaction_cronjob]
enable = true
timezone = "Asia/Shanghai"
cron = "0 0 0 * * *"
max_concurrent_regions = 4
compact_parallelism = 1
manual_trigger_cooldown_secs = 3600
max_history_jobs = 100
```

cron 表达式包含秒字段。例如,`0 0 0 * * *` 表示每天在所配置时区的午夜执行。

| 配置项 | 默认值 | 说明 |
| --- | --- | --- |
| `enable` | `false` | 是否启用定时 Compaction。 |
| `timezone` | 本地 IANA 时区;无法检测时使用 `UTC` | 计算 `cron` 时使用的时区,例如 `Asia/Shanghai`。 |
| `cron` | `0 0 0 * * *` | Compaction 执行计划。 |
| `max_concurrent_regions` | `4` | 单个任务并发提交 Region Compaction 请求的最大数量,必须大于 `0`。 |
| `compact_parallelism` | `1` | 传递给每个 Region Compaction 请求的并行度,必须大于 `0`。 |
| `manual_trigger_cooldown_secs` | `3600` | 人工触发与最近一次任务之间的最小间隔。最近的任务可以是定时触发或人工触发。设置为 `0` 可禁用冷却时间;该配置不会延迟定时任务。 |
| `max_history_jobs` | `100` | Metasrv KV backend 中最多保留的已完成任务报告数量,必须大于 `0`。 |

只有 leader Metasrv 会运行 scheduler。leader 发生切换后,新的 leader 会继续根据所配置的 cron 表达式调度后续任务。

## HTTP endpoints

该 plugin 会在 Metasrv HTTP server 上增加以下 endpoints。请将 `<metasrv-http-address>` 替换为 Metasrv 配置的 HTTP 地址。

### 触发任务

```bash
curl -X POST \
"http://<metasrv-http-address>/admin/compaction_cronjob/trigger"
```

该请求必须发送到 leader Metasrv。请求会等待任务完成 Region Compaction 请求的提交,并返回任务报告。

- 请求发送到 follower Metasrv 时返回 `409 Conflict`。
- 人工触发距离最近一次定时或人工任务不足 `manual_trigger_cooldown_secs` 时返回 `429 Too Many Requests`。
- 其他错误返回 `500 Internal Server Error`。

### 查询指定任务

```bash
curl --get \
--data-urlencode "job_id=2026-06-16T00:00:00+00:00" \
"http://<metasrv-http-address>/admin/compaction_cronjob/job"
```

`job_id` 由触发任务和任务列表 endpoints 返回。它是 RFC 3339 时间戳,在请求中应进行 URL 编码。任务不存在时,该 endpoint 返回 `404 Not Found`。

### 查询最近任务

```bash
curl --get \
--data-urlencode "limit=20" \
"http://<metasrv-http-address>/admin/compaction_cronjob/jobs"
```

任务按计划执行时间倒序排列。可选参数 `limit` 的默认值为 `20`。

## 任务报告

触发和查询 endpoints 返回 JSON 格式的任务报告。以下是一个执行成功的任务报告:

```json
{
"job_id": "2026-06-16T00:00:00+00:00",
"scheduled_at": "2026-06-16T00:00:00Z",
"started_at": "2026-06-16T00:00:00.010Z",
"finished_at": "2026-06-16T00:00:01.010Z",
"status": "succeeded",
"execution_result": {
"total_regions": 8,
"submitted_regions": 8,
"failed_regions": []
}
}
```

任务状态包括 `running`、`succeeded` 和 `partial_failed`。失败任务的状态包含失败原因,格式为 `{"failed": "<reason>"}`。`execution_result` 记录尝试提交和成功提交的 Region 请求数量,以及各 Region 的提交失败信息。
Comment thread
MichaelScofield marked this conversation as resolved.
Outdated
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ GreptimeDB Enterprise 包括以下高级功能,
- [部署 GreptimeDB](./deployments-administration/overview.md):设置认证信息及其他关键配置后,将 GreptimeDB 部署在 Kubernetes 上并监控关键指标。
- [审计日志](./deployments-administration/monitoring/audit-logging.md):记录数据库用户行为的日志。
- [自动分区平衡](./autopilot/region-balancer.md):通过分区监控和迁移在 datanode 之间自动平衡负载。
- [定时 Compaction](./deployments-administration/scheduled-compaction.md):定期为所有物理 Region 提交 regular Compaction 请求,并提供用于人工触发和任务查询的 HTTP endpoints。
- [Elasticsearch 查询兼容性](./elasticsearch-compatible/overview.md):在 Kibana 中以 GreptimeDB 作为后端。
- [Greptime 企业版管理控制台](./console-ui.md):加强版本的管理界面,提供更多的集群管理和监控功能。
- [读副本](./read-replicas/overview.md):专门运行复杂的查询操作的 datanode,避免影响实时写入。
Expand Down
1 change: 1 addition & 0 deletions sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,7 @@ const sidebars: SidebarsConfig = {
]
},
'enterprise/deployments-administration/backup',
'enterprise/deployments-administration/scheduled-compaction',
],
},
{
Expand Down
Loading