Production-grade infrastructure auditor linking Cloudflare DNS records to Hetzner Cloud virtual servers.
CloudMesh is a production-grade infrastructure auditing tool that matches active Cloudflare DNS A records to Hetzner Cloud virtual servers by resolving and matching public IP addresses. It identifies orphaned DNS records (subdomains pointing to missing servers), aggregates monthly cloud costs per project, checks domain expiration dates, pushes telemetry to a Prometheus/Pushgateway stack, and provides a modern, interactive React/Vite dashboard.
- Extended Resource Mapping: Integrates and maps Cloudflare DNS zones (supporting both
AandAAAArecords) to multiple Hetzner Cloud resource types, including Virtual Servers, Load Balancers, and Floating IPs. - Network Security Port Audit: Concurrently scans ports
22(SSH),80(HTTP),443(HTTPS), and3389(RDP) on all resolved IPs using a high-performance multi-threaded socket verification pool. - Stale VM & Cost Optimization Engine: Detects financial waste and potential subdomain takeovers by recommending removal of dangling DNS entries and idle/unmapped Hetzner resources.
- Interactive SVG Topology Graph: Renders a dynamic node connection map (Domain -> Cloudflare Record -> Public IP -> Hetzner Resource -> Project) showing orphaned nodes in crimson, clickable resource nodes, and cost-weighted projects.
- Drift & Historical Diff Engine: Computes modifications (added, removed, or changed records/resources) compared to prior audit runs, with a client-side selector for arbitrary historical snapshot comparison.
- DNS & HTTP Latency Telemetry: Measures DNS resolution speed and TCP connection latency asynchronously for all subdomains.
- Scheduled Monitor (Daemon Mode): Runs continuously inside containers or servers at a configurable interval (
MONITORING_INTERVAL_SECS) to automate ongoing scans. - Advanced Filtering: Filters records on the fly by Cloudflare proxy states, DNS type (A/AAAA), wildcard formats, cost ranges, projects, and domain expiration health.
- Disk-Persistent WHOIS Cache: Automatically caches domain registration lookups in
reports/whois_cache.jsonto prevent rate-limit blocks and speed up execution. - API Robustness & Retries: Utilizes a customized connection pool configured with a 5-retry policy and exponential backoff to handle transient errors (
429,500,502,503,504). - Label Sanitization (Data Safety): Masks sensitive metadata keys (e.g.,
key,token,secret,auth,password) in VM labels as********before exporting reports. - Multi-Channel Alerting: Dispatches summary metrics and domain expiration warnings to Slack channels (supporting PDF uploads) and Google Chat webhook cards.
- Prometheus Telemetry: Pushes metrics (running counts, processing durations, domain mappings, server uptimes) to Prometheus Pushgateway.
βββ docs/ # Detailed guides (Deployment, Configuration, Troubleshooting)
βββ frontend/ # React / Vite TypeScript dashboard application
β βββ src/ # React dashboard component codebase
β βββ public/ # Static assets & generated data.json source
βββ reports/ # Generated HTML, PDF, and JSON audit artifacts (gitignored)
βββ src/ # Modular Python package engine
β βββ api/ # Cloudflare and Hetzner API client components
β βββ core/ # Matcher calculations and persistent cache adapters
β βββ notifications/ # Slack and Google Chat webhook integrations
β βββ reports/ # Print HTML and JSON file writers
β βββ config.py # Configuration validation and environment loader
β βββ metrics.py # Prometheus metric registry declarations
β βββ main.py # Orchestrator flow
βββ docker-compose.yml # Telemetry containers (Prometheus & Pushgateway)
βββ prometheus.yml # Prometheus scrape targets configuration
βββ script.py # Thin backward-compatible script wrapper
βββ PROJECT.md # System specification and architecture design doc
For a rapid setup and run on a Linux server (e.g., target directory /home/cloudmesh), execute the following:
cd /home/cloudmesh
# 1. Stop old services
docker compose down --remove-orphans || true
# 2. Set up virtualenv (only needed once)
python3 -m venv venv
./venv/bin/pip install --upgrade pip
./venv/bin/pip install -r requirements.txt
# 3. Configure credentials (only needed once)
cp .env.example .env
nano .env # Fill in CLOUDFLARE_TOKEN, HETZNER_TOKEN_1, etc.
# 4. Create output directories
mkdir -p reports/snapshots
mkdir -p frontend/dist/snapshots
# 5. Start infrastructure
docker compose up -d
# 6. Run the audit
./venv/bin/python script.py- Python 3.8+
- NodeJS 18+ & pnpm
- wkhtmltopdf (optional, required to export PDF reports)
- Ubuntu/Debian:
sudo apt-get update && sudo apt-get install wkhtmltopdf - macOS:
brew install wkhtmltopdf - Windows: Download from wkhtmltopdf.org and add to system PATH.
- Ubuntu/Debian:
Create a virtual environment and install engine dependencies:
# Create and activate environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Or manually:
pip install requests pdfkit prometheus_client python-dotenv python-whoisCopy .env.example to .env and fill in API credentials and daemon configurations:
CLOUDFLARE_TOKEN=your_cloudflare_api_token
HETZNER_TOKEN_1=your_hetzner_project_api_token
HETZNER_PROJECT_NAME_1=Hetzner_Project_Name_1
PUSHGATEWAY_URL=http://localhost:9913
# (Optional) Scheduled Daemon Mode Configuration
MONITORING_INTERVAL_SECS=300 # Run audits every 5 minutes (set to 0 or omit to run once)
MAX_HISTORICAL_SNAPSHOTS=10 # Maximum history snapshots to persist for diffs and sparklinesExecute the Python script to fetch, map, and export the infrastructure mapping reports:
python script.pyThis script validates credentials, computes mappings, and outputs:
reports/mapping.html: Static printable report.reports/data.json&frontend/public/data.json: Raw structured audit data.reports/whois_cache.json: Persistent WHOIS queries.reports/mapping_YYYYMMDD_HHMMSS.pdf: Archived PDF report.
Run the frontend dev server to visualize findings interactively:
# Navigate to frontend and install dependencies
cd frontend
pnpm install
# Start local server
pnpm devOpen http://localhost:5173 in your browser. The dashboard automatically syncs with frontend/public/data.json on page refresh or load.
To collect execution metrics, spin up the telemetry containers:
# Start Prometheus and Pushgateway
docker compose up -d
# Verify services
# Prometheus target page: http://localhost:9912
# Pushgateway metrics: http://localhost:9913- Fork the repository.
- Create a feature branch (
git checkout -b feature/name). - Commit changes using standard Conventional Commit guidelines.
- Submit a Pull Request.
This project is licensed under the MIT License - see the LICENSE file for details.