Internal documentation — not for external distribution.

Database Schema

The database currently has 47 migrations in pkg/storage/migration/sql/. The server binary auto-runs pending migrations on startup.

Timestamps: All timestamp columns use TIMESTAMPTZ (timestamp with time zone). Data is stored in UTC. Migration 035 standardized the 64 columns that previously used TIMESTAMP (without timezone). The database-level timezone is set to UTC via ALTER DATABASE ... SET timezone = 'UTC'.

Scope Annotations: All sqlc queries carry -- scope: annotations validated by CI (TestQueryScopeCoverage). These document the tenant isolation intent of each query (org, global, system, resource, join).

Tables

Core Tables

TableDescription
organizationsMulti-tenant boundary
usersUser accounts (including system users); is_approved gates beta access
organization_membersMulti-org membership with role-based access
organization_invitesOrganization invite tokens and tracking
api_keysAPI authentication (Argon2id hashed)
servicesService registry
service_instancesHealth check tracking
settingsOrganization-aware settings with global defaults
casbin_ruleCasbin authorization policy storage (policy rules p, org-scoped role assignments g, global role assignments g2)
role_metadataRole display info (name, description, system flag) linked to casbin_rule via role_key; client_visible boolean controls visibility in Client API (superadmin is hidden)

Job System Tables

TableDescription
gather_jobsUpstream release fetch jobs (GitHub, Helm, OCI, endoflife.date); has labels JSONB column for categorization, sync_state JSONB column for incremental sync tracking, and suggest_global boolean for global pool contribution nominations
scrape_jobsRepository version scraping jobs
helm_sync_jobsHelm repository auto-sync jobs that create child gather jobs; has labels JSONB column for categorization and suggest_global boolean for global pool contribution nominations
task_executionsIndividual job run tracking with execution tokens
agent_submission_logsDebugging logs for agent task submissions

Release & Version Tables

TableDescription
upstream_releasesReleases discovered by gather jobs
version_snapshotsVersions extracted by scrape jobs

Rules & Alerts Tables

TableDescription
monitoring_rulesRule definitions
alert_configsLinks scrape jobs + gather jobs + rules
alertsGenerated alerts (supports soft delete via resolved_at)
audit_trailPolymorphic audit trail for entity state changes (alert actions, future: job actions)

Notification System Tables

TableDescription
notification_channelsWebhook endpoints and channel config
notification_rulesSeverity/event routing rules
notification_deliveriesDelivery tracking with retry logic
notification_deliveries_archiveArchive for old deliveries
notification_ack_tokensInbound acknowledgment tokens for webhooks

EOL Sync Tables

TableDescription
eol_sync_statusendoflife.date sync progress tracking
eol_productsCached product list from endoflife.date; status enum (approved, denied, pending) controls sync eligibility. State transitions: pending → approved, pending → denied, denied → approved. approved is terminal (cannot be denied — denying would orphan gather jobs and releases)

Feedback Table

TableDescription
user_feedbackUser feedback collection (bug reports, feature requests)

Separate Schema

Schema.TableDescription
earlyaccess.waitlist_requestsEarly access waitlist entries (separate PostgreSQL schema)

Key Enums

job_status:                   'pending', 'queued', 'in_progress', 'completed', 'failed'
source_type:                  'github_releases', 'helm_repository', 'endoflife_date', 'oci_registry'
parse_type:                   'yq', 'jq', 'regex', 'manual'
rule_type:                    'days_behind', 'majors_behind', 'minors_behind'
alert_severity:               'moderate', 'high', 'critical'
notification_channel_type:    'webhook', 'pagerduty', 'telegram', 'smtp'
notification_delivery_status: 'pending', 'in_progress', 'succeeded', 'failed', 'dead_letter'
org_role:                     'owner', 'admin', 'member'
feedback_category:            'bug_report', 'feature_request', 'general', 'ui_ux'
audit_entity_type:            'alert'
audit_action_type:            'created', 'escalated', 'acknowledge', 'unacknowledge', 'resolve'
audit_source:                 'api', 'ui', 'webhook', 'system'
eol_product_status:           'approved', 'denied', 'pending'

Warning: Only webhook has an actual sender implementation in pkg/notifications/webhook/. The pagerduty, telegram, and smtp channel types exist in the database enum but have no sender implementation yet. They are schema placeholders for future development.

Global Seed Data

On first startup, the system creates:

-- Default organization for global release data
INSERT INTO organizations (name, description, is_active)
VALUES ('Global', 'Default Organization that houses all default data', TRUE);

-- System users
INSERT INTO users (organization_id, email, first_name, is_system, metadata)
VALUES
    (1, 'system@planekeeper.com', 'Global System', TRUE, '{}'),
    (1, 'agent@planekeeper.com', 'Global Agent', TRUE, '{}');

-- Service registry
INSERT INTO services (id, service_name) OVERRIDING SYSTEM VALUE VALUES
    (1, 'server'),
    (2, 'serveragent'),
    (3, 'taskengine'),
    (4, 'eolsync'),
    (5, 'clientui'),
    (6, 'internalui'),
    (7, 'notifier'),
    (8, 'clientagent');

Pre-Seeded Global Jobs (Migration 034)

Migration 034 seeds ~173 global gather/helm sync jobs covering common infrastructure software. These provide immediate upstream release data for new organizations.

CategoryCountDescription
Helm Sync Jobs~29Full repository syncs (ingress-nginx, cert-manager, Argo, etc.)
Individual Helm Charts~11Cherry-picked charts from large repos (Prometheus, Grafana stacks)
GitHub Releases~85Direct release tracking (Kubernetes, Terraform, Go, Node.js, etc.)
OCI Registry~34Container image tags (Alpine, PostgreSQL, Nginx, k8s components)

All seeded jobs use organization_id = 0 (global scope), have staggered cron schedules across 1:00-7:38 AM UTC, and carry labels for categorization (e.g., {"category": "kubernetes", "ecosystem": "cncf"}).

The migration also:

  • Adds labels JSONB NOT NULL DEFAULT '{}' columns to gather_jobs and helm_sync_jobs
  • Migrates existing artifact names to unified host/path format (github.com/owner/repo, endoflife.date/product)
  • Creates partial unique indexes per source type for global job deduplication
  • Curates EOL products from ~431 to ~65 active products

Upstream Release Metadata

The upstream_releases.metadata JSONB column stores source-specific metadata per release:

Source TypeMetadata KeysDescription
github_releasesrelease_name, author, target_branch, assets_countGitHub release metadata
helm_repositoryapp_version, description, deprecated, kube_version, home, sourcesHelm chart metadata
oci_registrydigest, platformsContainer manifest metadata
endoflife_datecycle, eol, lts, support_end, latest_release_date, link, codename, extended_supportEOL lifecycle metadata

Gather Job Sync State

The gather_jobs.sync_state JSONB column (added by migration 038) tracks incremental sync progress for GitHub gather jobs:

KeyTypeDescription
full_sync_completebooleanWhether the last full sync fetched all releases without hitting the page limit
releases_fetchedintegerNumber of releases fetched in the last run
last_full_sync_atstring (RFC3339)Timestamp of the last full (non-incremental) sync

The dispatcher uses this state to inject an _incremental_since hint into the agent’s source config, allowing the GitHub gatherer to stop pagination early when it reaches already-known releases. Sync state is reset when a gather job is updated or its releases are cleared.

Source Types (Gatherers)

The agent supports four data source types for upstream releases:

Source TypeImplementationArtifact Name FormatDescription
github_releasespkg/gatherer/github.gogithub.com/owner/repoFetches releases from GitHub repositories
helm_repositorypkg/gatherer/helm.gohost/path/chart-nameFetches chart versions from Helm repositories
endoflife_datepkg/gatherer/endoflife.goendoflife.date/productFetches EOL data from endoflife.date API
oci_registrypkg/gatherer/oci.goregistry/imageFetches tags from OCI container registries

Version Extraction Methods

The agent supports three parsing strategies for scrape jobs, plus a manual entry mode:

TypeUse CaseExample Expression
YQYAML files (Chart.yaml).version
JQJSON files (package.json).version
RegexAny text fileversion:\s*([\d.]+)
ManualUser-entered versions (no agent needed)- (placeholder)

Manual scrape jobs use placeholder values (repository_url="manual://", target_file="-", parse_expression="-") and are never assigned to agents. Versions are set via the POST /scrape-jobs/{id}/set-version API endpoint.

Rule Types

TypeDescriptionParameters
days_behindAlert if version is N days oldmax_days
majors_behindAlert if N major versions behindmax_majors
minors_behindAlert if N minor versions behind (counts actual releases)max_minors

All rules support the stable_only parameter to ignore prereleases.