Skip to content

Latest commit

 

History

History
487 lines (403 loc) · 18.6 KB

deployment_notes.md

File metadata and controls

487 lines (403 loc) · 18.6 KB

Documentation

Deploment Notes

Follow these steps to configure and deploy the application 0. Verify pre-requisites and set up environment

  1. Clone code from Github repository into environment
  2. Add required configurations
  3. Add optional configurations
  4. Configure and secure webserver
  5. Build app
  6. Open app and login

Platform / Pre-requisites

  • Provision environment that can run Laravel code [unix?]
    Digital Ocean Droplet
    	Ubuntu LEMP on 16.014
    	2vCPUs
    	4GB / 80 GB Disk
    
  • PHP 8.4+ [verify version]
     gmaddock@Wrecked:/var/www/events-tracker$ php -v
    PHP 8.4.17 (cli) (built: Jan 18 2026 14:18:33) (NTS)
    
  • Verify required extensions are installed: pdo_mysql, zip
  • MySQL 8 [verify version - may not require 8]
    gmaddock@Wrecked:/var/www/events-tracker$ mysql -V
    mysql  Ver 8.0.45-0ubuntu0.22.04.1 for Linux on x86_64 ((Ubuntu))
    
  • NodeJS 24.14.1+
    gmaddock@Wrecked:/var/www/events-tracker$ node -v
    v24.14.1
    
  • Provision a database and add a user with access
        - CREATE DATABASE
        ```
        CREATE DATABASE `stage_events_tracker` /*!40100 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci */ /*!80016 DEFAULT ENCRYPTION='N' */
        ```
        - CREATE USER
        ```
        CREATE USER 'stage_user'@'%' IDENTIFIED BY 'ActualPassword';
        GRANT ALL ON stage_events_tracker.* TO 'stage_user'@'%';
        FLUSH PRIVILEGES; 
        ```
        - Log in with this user to test

Installing / Getting started

  • Connect to the server
  • Go to the default path for deployments
cd /var/www
  • Clone the repo.
$ git clone git@github.com:geoff-maddock/events-tracker.git project-name
$ cd project-name
  • Install the PHP dependencies.
$ composer install
  • Install node dependencies
$ npm install

Configuration

  • Copy .env.example to .env and add values ** Add a database configuration that matches your provisioned database
    	APP_ENV=local
      APP_DEBUG=false
      APP_KEY=THIS_VALUE_ADDED_BY_KEY_GENERATION
      APP_URL=https://project-domain.com
      APP_FEEDBACK_EMAIL=your-feedback-email@domain.com
      APP_NOREPLY_EMAIL=noreply@domain.com
      APP_ADMIN_EMAIL=your-admin-email@domain.com
      APP_SUPERUSER=1
      APP_FB_APP_ID=1111111111
      FACEBOOK_APP_ID=999
      FACEBOOK_APP_SECRET=999
      FACEBOOK_GRAPH_VERSION=v5.0
    
      DB_CONNECTION=mysql
      DB_HOST=127.0.0.1
      DB_PORT=3306
      DB_DATABASE=stage_events_tracker
      DB_USERNAME=stage_user
      DB_PASSWORD=database_password
    
  • Run key generation php artisan key:generate
  • Run composer install
  • Run npm install
  • Run node build for your environment
    • npm run build
  • Run migrations to create the initial database (new installs only — never use migrate:fresh on an existing deployment as it drops all data).
    • php artisan migrate
  • Seed database tables from one of the provided default seeders. Only run this when starting the production app the first time.
    • php artisan db:seed --class=ProdBasicDatabaseSeeder
      • The most basic data to run the app, some additional config will be required.
    • php artisan db:seed --class=ProdExtraDatabaseSeeder
      • This includes base data for all modules and more fleshed out permissions. No specific content.
    • php artisan db:seed --class=ProdPittsburghDatabaseSeeder
      • This includes everything in the ProdExtra seeder, plus some base specific data for Pittsburgh.
  • Add a DNS A record with your authoritative DNS Provider for your domain
  • Point web server to /html/index.php
    • Configure NGINX [Recommended]
    • [EXAMPLE]
          server {
      
          root /var/www/events-tracker/public;
          index index.php index.html index.htm;
      
          # Make site accessible from http://localhost/
          server_name events-tracker.com;
      
          location / {
                  # First attempt to serve request as file, then
                  # as directory, then fall back to displaying a 404.
                  try_files $uri $uri/ /index.php?$args;
                  # Uncomment to enable naxsi on this location
                  # include /etc/nginx/naxsi.rules
          }
      
          error_page 404 /404.html;
          error_page 500 502 503 504 /50x.html;
          location = /50x.html {
                  root /usr/share/nginx/html;
          }
      
          location ~ \.php$ {
          include snippets/fastcgi-php.conf;
          fastcgi_pass unix:/run/php/php8.4-fpm.sock;
          }
      
  • Set up SSL
  • Modify permissions
	sudo chmod 777 /var/www/project-name/storage/logs/laravel.log
	sudo chgrp -R www-data storage bootstrap/cache
	sudo chmod -R ug+rwx storage bootstrap/cache

The scheduled sitemap:generate command writes sitemap*.xml straight into public/, so the sitemap files must be writable by the user that runs schedule:run (www-data — check with sudo grep -rn 'schedule:run' /var/spool/cron/crontabs/ /etc/cron.d/ /etc/crontab). A manual run as the deploying user leaves them owned by that user, and every subsequent cron run then fails:

	sudo chgrp www-data public/sitemap*.xml
	sudo chmod 664 public/sitemap*.xml

Deliberately not making public/ itself group-writable: replacing an existing file only needs write permission on the file, so the daily run works without handing PHP the ability to create files in the docroot. The trade-off is that new sitemap files cannot be created by the scheduler — if a section outgrows one file (45k URLs) or a new section is added to the command, the first run after that must be done by a user who can write public/:

	php artisan sitemap:generate && sudo chgrp www-data public/sitemap*.xml && sudo chmod 664 public/sitemap*.xml

The command checks all of this up front and exits non-zero naming the offending files, so the condition shows up as a failed command rather than a silent staleness — Spatie's writeToFile() is a bare file_put_contents() that only raises a warning on failure. Verify a fix with a run as the scheduler user:

	sudo -u www-data php /var/www/project-name/artisan sitemap:generate
  • Build assets
npm run build

Optional Configurations

  • S3 Storage [RECOMMENDED]
  • Mailgun API key [RECOMMENDED]
  • Facebook API key [OPTIONAL]
  • Twitter API key [OPTIONAL]
  • AI Provider API key [OPTIONAL]

Log in to running app

Developers

CI and Testing

Pull requests and pushes to main run .github/workflows/php.yml:

  • composer validate
  • config:cache, route:cache and view:cache, as production runs them, then cleared
  • ESLint (npm run lint) and the Vite production build (npm run build)
  • PHPStan (Larastan, level 5, with phpstan-baseline.neon)
  • the PHPUnit suite against a MySQL 8 service container (migrate + seed first)

The deploy (build-prod.yml) only runs for a commit whose php.yml run passed.

The dependency audits (composer audit and the npm audit in .github/scripts/npm-audit.mjs) are a separate workflow, .github/workflows/audit.yml. It runs every day, on pull requests that change a dependency file, and on demand from the Actions tab. It does not block a deploy: a failed daily run is the notice that an advisory was published, and GitHub emails it to whoever last changed that workflow file.

Run parts of CI manually:

composer phpstan
composer tests            # clears cached config, migrate:fresh --seed on the testing DB, then phpunit
npm run lint && npm run build

Deploying to production

.github/workflows/build-prod.yml (Actions → Build Prod → Run workflow) deploys main. It first checks that the PHP Composer CI run for that exact commit passed, and refuses to deploy otherwise. On the server it runs these steps, which are also the checklist for a manual deploy:

cd /var/www/events-tracker
rm -f bootstrap/cache/*.php                 # first: stale caches can fatal artisan, and a leftover manifest can block the pull
git checkout main && git pull
composer install --no-dev --optimize-autoloader --no-interaction   # always: lockfile changes need it
php artisan migrate --force                 # additive migrations ship with the code that needs them
npm ci && npm run build
php artisan config:cache && php artisan route:cache && php artisan view:cache
php artisan queue:restart                   # the supervised worker keeps old code until restarted

Only cache config and routes on production. On the dev checkout, a cached config makes PHPUnit ignore .env.testing. The test suite refuses to run in that case, and php artisan config:clear && php artisan route:clear fixes it.

Environments: Dev, Testing, Production

  • Dev environment notes
  • Testing environment notes
  • Prod environment notes

Upgrading to v2026.05.01

This is a major release with a full UI refresh (Tailwind CSS / Vite), new features, and database schema changes. Follow these steps to upgrade an existing installation.

Pre-upgrade checklist

  • Back up the database before making any changes
  • Put the app into maintenance mode: php artisan down
  • Confirm you are on PHP 8.4+ and Node 24+

1. Pull the latest code

git pull origin master

2. Update PHP dependencies

composer install --no-dev --optimize-autoloader

3. Update Node dependencies and rebuild assets

The build pipeline has migrated from Laravel Mix/webpack to Vite. If you have old webpack.mix.js-based build artifacts, they will be replaced.

npm install
npm run build

4. Run database migrations

Five new migrations are included. Run them against the existing database — do not use migrate:fresh as that will drop all data.

php artisan migrate

Migrations included in this release:

  • add_description_to_tags_table
  • create_event_shares_table
  • create_click_tracks_table
  • add_user_id_to_click_tracks_table
  • create_blog_photo_table

5. Clear caches

php artisan config:cache
php artisan route:cache
php artisan view:cache

6. Bring the app back online

php artisan up

7. Verify

  • Log in and confirm the UI loads correctly (dark theme by default)
  • Check that events, entities, and series pages render as expected
  • Confirm click-tracking redirects (/go/evt-{id}, /go/ser-{id}) are working if you use ticket links

Background queue worker

Long-running actions (Instagram posting, account data exports) run as queued jobs instead of blocking the web request. This requires the database queue driver and a running worker process.

One-time setup

# In .env
QUEUE_DRIVER=database
# Optional: also email users when a queued job finishes
QUEUE_NOTIFY_EMAIL=false

# Create the queue tables (jobs, failed_jobs, job_batches, notifications, job_statuses)
php artisan migrate

Run the worker

A worker must be running for queued jobs to process. In production, supervise it:

; /etc/supervisor/conf.d/events-worker.conf
[program:events-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/events/artisan queue:work --sleep=3 --tries=3 --timeout=700
autostart=true
autorestart=true
stopwaitsecs=1260
user=www-data
numprocs=1
redirect_stderr=true
stdout_logfile=/var/www/events/storage/logs/queue-worker.log
supervisorctl reread && supervisorctl update && supervisorctl start events-queue:*
# Restart the worker after every deploy so it picks up new code:
php artisan queue:restart

For local development, run php artisan queue:work in a separate terminal, or set QUEUE_DRIVER=sync to execute jobs inline (the test suite always uses sync).

Users see queued job progress and completion under the Notifications item in the sidebar (/job-status).

Scheduled tasks

Everything registered in app/Console/Kernel::schedule() — weekly notifications, entity notifications, user cleanup, export cleanup, series event creation, feedback invitations, the admin activity summary, and the Discord reminder and digest commands — runs from a single cron entry:

* * * * * php /var/www/events/artisan schedule:run >> /dev/null 2>&1

Laravel's scheduler works by having cron invoke schedule:run every minute; the framework then decides which commands are actually due. Without that one line, nothing scheduled runs at all, and there is no error — the commands simply never fire. That failure mode is silent, so check it explicitly rather than assuming.

Confirm what the app believes is scheduled:

php artisan schedule:list

Confirm cron is actually invoking it — the entry may live in the deploying user's crontab, www-data's, root's, or /etc/cron.d/:

sudo grep -rn 'schedule:run' /var/spool/cron/crontabs/ /etc/cron.d/ /etc/crontab

Scheduled commands inherit the timezone set per-task in the Kernel (most use America/New_York), not the server timezone.

Some scheduled commands queue jobs rather than doing the work inline, so they also need the queue worker above to be running.

Monitoring (#2399)

What tells you something has stopped, and what you have to set up once.

Failure How it is noticed
Site or database down GET /up returns 503 (or nothing). Needs an external uptime check, see below.
Cron or the scheduler stopped monitor:check runs every 15 minutes and checks in to the Sentry cron monitor scheduler-heartbeat. Missed check-ins raise a Sentry issue.
A scheduled command failed An exception is reported to Sentry as usual. A non-zero exit without one is reported as MonitoringAlert: Scheduled command failed ....
Queue worker stopped or stuck monitor:check reports Queue is not being worked when a due job has waited MONITOR_QUEUE_MAX_WAIT_MINUTES (default 45).
A queued job failed The worker reports the exception to Sentry; the row is in failed_jobs (php artisan queue:failed).
Facebook/Instagram token expiring monitor:check asks the Graph API once a day and reports when the page token is invalid or within MONITOR_TOKEN_WARN_DAYS (default 14) of expiry.

All of the Sentry reports need SENTRY_DSN set, and the same problem is reported again only after MONITOR_REPEAT_HOURS (default 6). Run php artisan monitor:check by hand to see what it finds.

One-time setup:

  1. Uptime check. Point an external service (UptimeRobot, Better Stack, Sentry Uptime) at https://<site>/up, alerting on anything but 200. The route uses no session and makes one trivial database query.
  2. Sentry cron monitor. Nothing to create: the first check-in from production creates scheduler-heartbeat with its schedule. Open it under Crons in Sentry and confirm who is notified. It is the only cron monitor the app uses.

Not covered: mail delivery. A send that throws is reported like any error, but mail that is accepted and never arrives is not noticed; the weekly adminTest email is still the only end-to-end check.

Database backups are made on the server by automysqlbackup, outside the app; nothing here checks that they ran.

Email unsubscribe and SES bounce handling (#2103)

Bulk mail (weekly/daily digests, follower and forum alerts, entity contact mail) carries List-Unsubscribe and List-Unsubscribe-Post headers and a footer link. All of them are Laravel signed URLs with no expiry, so they need no login and old emails keep working. They are signed with APP_KEY: rotating APP_KEY breaks every unsubscribe link already sent.

  • GET /email/unsubscribe/{user}/{list} turns one list off and shows a confirmation; POST to the same URL is RFC 8058 one-click (CSRF-exempt, returns 204).
  • GET|POST /email/preferences/{user} is the logged-out preference page.
  • GET|POST /email/unsubscribe/contact?email=… opts an entity contact address out of entity mail (email_opt_outs), without suppressing account mail to that address.

SES bounces and complaints

POST /webhooks/ses takes SES notifications through SNS, verifies the SNS signature, and writes permanent bounces and complaints to email_suppressions (source ses). It only acts on topics listed in SES_SNS_TOPIC_ARNS; with that unset it rejects everything.

  1. In SNS (us-east-2), create a topic, e.g. ses-notifications.
  2. Add the ARN to .env, then php artisan config:clear:
    SES_SNS_TOPIC_ARNS=arn:aws:sns:us-east-2:<account>:ses-notifications
    
    (comma-separated for more than one)
  3. Subscribe https://arcane.city/webhooks/ses to the topic (protocol HTTPS). The endpoint confirms the subscription itself; the log shows SesWebhook: confirmed the SNS subscription.
  4. In SES, point the arcane.city identity's Bounce and Complaint notifications at the topic (or add an SNS event destination for Bounce and Complaint on SES_CONFIGURATION_SET; both formats are handled).
  5. Check it with the SES mailbox simulator: send to bounce@simulator.amazonses.com and complaint@simulator.amazonses.com, then look for the two rows in email_suppressions.

Digest engagement gate (#2083)

The first gated run of each digest sends a one-time wave of "paused" notices to every dormant subscriber who would have received that digest. Check the numbers first:

php artisan notifyWeekly --dry-run
php artisan notify --dry-run

Each prints e.g. DRY RUN: would send 120 weekly digest(s), 85 paused notice(s); skipped 0 paused and 448 with nothing this week. After the first run, dormant users are skipped quietly (the "paused" count). Run the first gated digests while still on Mailgun, which already suppresses its known bounces; the notice wave then never reaches SES (#2087).

Hero image variants (#1932)

New uploads get a 1200px lg- WebP that entity and event heroes serve through srcset. Existing photos fall back to the full image until the backfill gives them one. It covers primary (hero) photos only, newest first, in batches, and is safe to re-run:

php -d memory_limit=512M artisan photos:generate-variants --dry-run --thumbnails --limit=50
php -d memory_limit=512M artisan photos:generate-variants --thumbnails --limit=200   # repeat until 0 remain

The first line of output says how many primary photos are left. --thumbnails also rebuilds each WebP thumbnail with today's encoder and keeps it only if it is at least 10% smaller. Thumbnails from before about mid-July 2026 came out 30–50% smaller in testing; newer ones are left alone. Legacy main images that were never size-capped can be large, which is why the memory limit is raised. A photo whose file is missing is logged, skipped and retried next run.