Forward analysisd JSON alerts over syslog so agent_name is a first-cl… - #2301
Closed
atomicturtle wants to merge 4 commits into
Closed
atomicturtle wants to merge 4 commits into
atomicturtle wants to merge 4 commits into
Conversation
…ass field. JSON syslog_output now tails alerts.json (enabled by default) instead of the compact alerts.log schema, with an e2e suite that prints the forwarded object.
Keep the 4.4.0 JSON syslog changelog entry together with the Dovecot and cdb mmap fixes that landed on main.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates OSSEC’s syslog forwarding so the JSON syslog_output format is sourced from alerts.json (instead of the compact alerts.log schema), making fields like agent_name first-class in forwarded events. It also enables jsonout_output by default and adds compile-time and Linux E2E coverage to validate the new JSON-forwarding path.
Changes:
- Add a JSON “file queue” reader (
jqueue_*) foralerts.jsonand route JSON syslog destinations through it. - Enable
jsonout_outputby default (config default + shipped configs + installer). - Add build-time verification and a Linux E2E harness/suites for install/upgrade/smoke + JSON syslog validation.
Reviewed changes
Copilot reviewed 40 out of 42 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| testsuite/run-all.sh | Wrapper to run compile matrix then optionally Linux E2E. |
| testsuite/README.md | Document new post-build checks and E2E JSON syslog suite. |
| testsuite/e2e-linux/suites/05-alerts-json-syslog.sh | E2E suite to capture UDP JSON syslog datagram and validate agent_name. |
| testsuite/e2e-linux/suites/04-upgrade-server.sh | E2E suite for server upgrade validation. |
| testsuite/e2e-linux/suites/03-feature-smoke.sh | E2E feature smoke (logtest + syscheck touch). |
| testsuite/e2e-linux/suites/02-install-agent.sh | E2E agent install + enrollment suite. |
| testsuite/e2e-linux/suites/01-install-server.sh | E2E server install suite. |
| testsuite/e2e-linux/run.sh | E2E runner/selector for suites. |
| testsuite/e2e-linux/README.md | Documentation for Linux E2E layout and suites. |
| testsuite/e2e-linux/lib/transport.sh | Backend-agnostic remote exec/copy helpers (ssh/podman). |
| testsuite/e2e-linux/lib/syscheck.sh | Shared syscheck E2E helper library. |
| testsuite/e2e-linux/lib/ssh.sh | SSH helper functions for E2E transport. |
| testsuite/e2e-linux/lib/podman_target.sh | Podman target lifecycle (image/network/container management). |
| testsuite/e2e-linux/lib/pkg.sh | Install/uninstall helpers (RPM/Deb/source) for E2E. |
| testsuite/e2e-linux/lib/inventory.sh | YAML inventory parsing/loading helpers. |
| testsuite/e2e-linux/lib/enroll.sh | Agent enrollment helper (manage_agents + key install). |
| testsuite/e2e-linux/lib/common.sh | Common E2E constants/logging/wait helpers. |
| testsuite/e2e-linux/lib/assert.sh | Remote assertions (process/log checks, logtest). |
| testsuite/e2e-linux/lib/artifacts.sh | Build/locate artifacts (tarball, RPM via mock, deb build). |
| testsuite/e2e-linux/inventory.ssh-server.yaml | Checked-in inventory for SSH server target. |
| testsuite/e2e-linux/inventory.example.yaml | Example inventory template (intended to be copied locally). |
| testsuite/e2e-linux/inventory.agent-test.yaml | Helper inventory for server+agent combo testing. |
| testsuite/e2e-linux/images/ubuntu2404/Containerfile | Systemd-enabled Ubuntu 24.04 podman image for E2E installs. |
| testsuite/e2e-linux/images/rocky10/Containerfile | Systemd-enabled Rocky Linux 10 podman image for E2E installs. |
| testsuite/e2e-linux/fixtures/syscheck/sample.txt | Syscheck fixture content for report_changes testing. |
| testsuite/e2e-linux/fixtures/sshd-failed-login.log | Fixture log for logtest/localfile smoke. |
| testsuite/build-test/verify-alerts-json.sh | Post-server-build verification for alerts.json + JSON syslog objects. |
| testsuite/build-test/build.sh | Pass TEST into server build and route through build-linux.sh. |
| testsuite/build-test/build-linux.sh | Add per-target post-build verification (agent audit vs server JSON). |
| testsuite/build-test/alerts-json/test_jqueue.c | Unit-style test program for alerts.json queue parsing/rotation behavior. |
| src/os_csyslogd/json-queue.c | Implement JSON file queue reader for alerts.json (tail/rotate/truncate handling). |
| src/os_csyslogd/csyslogd.h | Expose JSON syslog forwarding API and include cJSON. |
| src/os_csyslogd/csyslogd.c | Drain alerts.json for JSON destinations and alerts.log for others (fairness). |
| src/os_csyslogd/alert.c | Move JSON syslog formatting to OS_Alert_SendSyslog_JSON; keep legacy formats via alerts.log. |
| src/headers/json-queue.h | Public header for alerts.json queue reader. |
| src/analysisd/config.c | Enable jsonout_output by default in analysisd global config. |
| install.sh | Ensure server install writes <jsonout_output>yes</jsonout_output> by default. |
| etc/ossec.conf | Enable <jsonout_output>yes</jsonout_output> in default config. |
| etc/ossec-server.conf | Enable <jsonout_output>yes</jsonout_output> in server config. |
| etc/ossec-local.conf | Enable <jsonout_output>yes</jsonout_output> in local config. |
| contrib/specs/ossec-hids.logrotate | Add alerts.json to logrotate set. |
| .gitignore | Ignore local E2E inventory + build scratch outputs. |
Suppressed comments (2)
testsuite/e2e-linux/inventory.example.yaml:30
- This example inventory hard-codes a real server IP in the per-agent
server:override. Prefer a placeholder value so the repo does not publish a real manager address.
family: deb
server: "10.66.6.82"
testsuite/e2e-linux/README.md:37
- This line repeats a hard-coded real server address (
root@10.66.6.82). To avoid publishing real infrastructure details, use a placeholder here (and keep the real address only in local, gitignored inventory files).
Primary validated path: **SSH Rocky 10 server** (`root@10.66.6.82`) with **Rocky 10 Podman agent** using el10 RPMs from `mock -r rocky-10-x86_64`.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+18
to
+22
| if [[ "$target" == root@* ]]; then | ||
| ssh_exec "$target" "$@" | ||
| else | ||
| ssh_exec "$target" sudo -n bash -s <<<"$(printf '%q ' "$@")" | ||
| fi |
Comment on lines
+368
to
+378
| cJSON *rr = cJSON_CreateObject(); | ||
| item = cJSON_GetObjectItem(rule, "sidid"); | ||
| if (cJSON_IsNumber(item)) { | ||
| cJSON_AddNumberToObject(rr, "sidid", item->valueint); | ||
| } | ||
| item = cJSON_GetObjectItem(rule, "level"); | ||
| if (cJSON_IsNumber(item)) { | ||
| cJSON_AddNumberToObject(rr, "level", item->valueint); | ||
| } | ||
| cJSON_AddItemToObject(stub, "rule", rr); | ||
| } |
Comment on lines
+1
to
+8
| # SSH-only inventory for community Rocky 10 server validation | ||
| hosts: | ||
| - name: rocky10-community-server | ||
| backend: ssh | ||
| ssh: root@10.66.6.82 | ||
| role: server | ||
| family: rpm | ||
| distro: rockylinux-10 |
Comment on lines
+1
to
+14
| hosts: | ||
| - name: rocky10-community-server | ||
| backend: ssh | ||
| ssh: root@10.66.6.82 | ||
| role: server | ||
| family: rpm | ||
| distro: rockylinux-10 | ||
|
|
||
| - name: rocky10-agent-podman | ||
| backend: podman | ||
| image: localhost/ossec-e2e-rocky10 | ||
| role: agent | ||
| family: rpm | ||
| server: "10.66.6.82" |
Comment on lines
+6
to
+7
| - **Dedicated hosts** over SSH (community primary: Rocky Linux 10 at `root@10.66.6.82`) | ||
| - **Podman** systemd containers (local / CI-friendly) |
Comment on lines
+3
to
+8
| - name: rocky10-community-server | ||
| backend: ssh | ||
| ssh: root@10.66.6.82 | ||
| role: server | ||
| family: rpm | ||
| distro: rockylinux-10 |
Comment on lines
+22
to
+23
| # When testing against the community manager: | ||
| server: "10.66.6.82" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JSON syslog_output now tails alerts.json (enabled by default) instead of the compact alerts.log schema, with an e2e suite that prints the forwarded object.