Skip to content

fix(guestos): allow systemd-resolved to connect to networkd's resolve-hook socket - #11413

Merged
basvandijk merged 1 commit into
masterfrom
ai/guestos-resolved-selinux-2026-09-01
Sep 2, 2026
Merged

fix(guestos): allow systemd-resolved to connect to networkd's resolve-hook socket#11413
basvandijk merged 1 commit into
masterfrom
ai/guestos-resolved-selinux-2026-09-01

Conversation

@basvandijk

Copy link
Copy Markdown
Collaborator

Why

While root-causing a flake of //rs/tests/consensus/upgrade:upgrade_downgrade_unassigned_nodes_test_head_nns (2026-08-28 17:40 UTC), the unassigned GuestOS node turned out to have no working DNS for its entire ~11-minute life:

  • 443 × enforced SELinux denials in its journal:
    avc: denied { connectto } for comm="systemd-resolve" path="/run/systemd/resolve.hook/io.systemd.Network"
    scontext=...:systemd_resolved_t:... tcontext=...:systemd_networkd_t:... tclass=unix_stream_socket permissive=0
    
  • 790 × Failed to start query: Permission denied from systemd-resolved — every DNS lookup failed;
  • consequently, all 58 of the orchestrator's attempts to download the elected GuestOS upgrade image failed within ~2 ms (error sending request), zero bytes ever fetched, and the test timed out.

systemd-resolved connects to systemd-networkd's varlink "resolve hook" socket (/run/systemd/resolve.hook/io.systemd.Network) to obtain per-link DNS configuration. Our policy does not allow that connectto, so the denial fires on every boot (a single Failed to connect to resolve.hook: Permission denied appears in healthy boots too). Usually resolved still ends up with working DNS through other paths; nondeterministically — as in this run — it does not recover and the node is left without name resolution, with no self-healing for the rest of the boot. SSH/IP-based connectivity keeps working, which is why the node looks healthy to the test driver while every DNS-dependent operation (such as upgrade-image downloads) is dead.

Fix

Allow the connection in the systemd-fixes policy module:

allow systemd_resolved_t systemd_networkd_t : unix_stream_socket { connectto };

(The sock-file write permission is evidently already granted — the denial fires on the final connectto check.)

Validation

  • bazel build //ic-os/guestos/envs/dev:rootfs-tree.tar — pass (this is the action that compiles the SELinux policy modules via the refpolicy devel Makefile, so it validates the new rule); //ic-os/components:check_unused_components_test — pass. (dev_component_file_references_test fails identically on unmodified master on this machine — a local UnicodeDecodeError reading the open_rootfs_dev binary — so it is left to CI.)

Found while root-causing the //rs/tests/consensus/upgrade:... flakiness following .claude/skills/fix-flaky-tests/SKILL.md. The tightened driver/bazel timeouts for the affected test are in #11412.

🤖 Generated with Claude Code

…-hook socket

systemd-resolved connects to systemd-networkd's varlink resolve-hook socket
(/run/systemd/resolve.hook/io.systemd.Network) to obtain per-link DNS
configuration. Our policy denies that connectto, so an enforced AVC denial
fires on every GuestOS boot; usually resolved still recovers, but
nondeterministically it is left without any DNS servers for the lifetime of
the boot, with every lookup failing with 'Failed to start query: Permission
denied'. This broke DNS for the whole life of an unassigned node in a run of
upgrade_downgrade_unassigned_nodes_test_head_nns on 2026-08-28, making all
58 attempts of the orchestrator to download the elected GuestOS upgrade
image fail within ~2ms each, and timing out the test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The rule directly addresses the observed SELinux denial with an appropriately scoped permission.

Pull request overview

Allows systemd-resolved to retrieve per-link DNS configuration from systemd-networkd.

Changes:

  • Grants the required SELinux connectto permission.
  • Documents the DNS failure caused by the denial.
File summaries
File Description
ic-os/components/guestos/selinux/systemd-fixes/systemd-fixes.te Adds the targeted socket permission.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@basvandijk
basvandijk marked this pull request as ready for review September 2, 2026 14:14
@basvandijk
basvandijk requested a review from a team as a code owner September 2, 2026 14:14
@github-actions github-actions Bot added the @node label Sep 2, 2026
@basvandijk
basvandijk added this pull request to the merge queue Sep 2, 2026
Merged via the queue into master with commit 7360f8f Sep 2, 2026
48 checks passed
@basvandijk
basvandijk deleted the ai/guestos-resolved-selinux-2026-09-01 branch September 2, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants