Observability & audit

How it works

Greenlight separates two things that often get conflated. The audit log is the record of every action that affected what an app can do — who took it, when, against what, and why. Runtime observability is the live picture of how apps and integrations are performing: logs, metrics, errors, latency. Both are surfaced in the IT dashboard; the underlying stores are different so they scale independently.

What’s audited

Every action that affects an app’s permissions, code, configuration, or state writes an audit event. The set of audited categories is fixed and visible in the dashboard.

Category	Examples
Platform actions	App registered, integration registered from the catalog or as a custom integration, integration granted to an app, policy added or changed, role assigned, integration or credential soft-deleted, platform release observed or applied (including a failed or canary update run), an update started by hand from Internal Health, a node-count change requested on Internal Health’s Cluster capacity card and whether it applied, a declined capacity ask, scheduled container-image retention (what a prune removed, what it kept and why, and whether the run was a dry run), and each control-plane incident Greenlight observed and its recovery if it saw one (a container killed for running out of memory, a service with no replica ready, background work that stopped progressing, a cloud dependency that degraded)
Agent actions	Pull request opened, Knowledge proposal submitted, proxy call made, policy bundle read, deployed app data inspected
IT actions	App killed or restored, integration credential rotated, user role changed, a crashed control-plane container’s captured log downloaded, a failed scheduled background job’s kept log opened
Auth events	Dashboard logins, /auth/check denials, token issuance, each agent sign-in grant and the person’s Allow or Cancel on the consent page (auth.agent_authorization.created, .consented, .declined), and OAuth authorization-server failures (auth.authorize.failed / auth.token_exchange.failed) with a distinct reason for each rejection (bad client, PKCE mismatch, code replay, deactivated user, …); user passthrough connections (user_connection.connect.succeeded, user_connection.connect.refused with the account that was refused and why, user_connection.revoke.succeeded with who revoked and why), never carrying a token

Reading data through the data broker is also audited at the per-call level, with the bound user attached.

Notifications are audited too, on every destination they go out on. A permission request or a request to connect a new system is emailed to your administrators and, when your org has registered a Slack webhook for it, posted to that channel, and each leg records its own outcome. An app that keeps crashing works the same way: Greenlight marks it Degraded and posts one health alert to Slack. It emails the app’s owner and your administrators too, but only once an admin turns on Email health alerts in Settings → Notifications, which is off to start with. It sends one alert per crash loop, not one a minute, and the alert links to the app’s page instead of carrying any of its logs. If GitHub stops running your organization’s pipelines because of Actions billing, your administrators get one alert for the whole organization — not one per app — linking to GitHub’s billing settings, delivered the same way. Greenlight also watches for behavior that is not a crash but still needs a person, and sends a health alert the same way: an app answering more than 10% of its requests with server errors, which goes to the owner and your administrators; and two signals that go to your administrators only — an app calling its integrations more than five times as often as usual over an hour, or an app the data proxy blocked from an integration it has no grant for. Each sends one alert for as long as the condition lasts, and records app.anomaly.detected when it starts and app.anomaly.resolved when it clears. When Greenlight can’t deliver one — nobody it should reach is active, the install has no email sender configured, the Slack webhook has been revoked, or the channel is archived — it records that in the audit log with the reason, the same way it records a message that went out. So “did anyone actually hear about that request?” is a question the audit log answers, rather than one that needs someone to go reading server logs.

Audit event shape

Every event has the same shape, regardless of category:

{
  "id": "evt_01HQX5MWGRTYBJ7C7C7C7C7C7C",
  "timestamp": "2026-03-12T14:08:11.412Z",
  "category": "knowledge",
  "event": "knowledge.proposal.accepted",
  "actor": {
    "type": "user",
    "actor_id": "usr_jane_doe",
    "actor_channel": "dashboard"
  },
  "target": { "kind": "knowledge_entry", "id": "kn_pf2", "scope": "app", "app_id": "app_k9x2m3p" },
  "diff": { "version": [3, 4], "title_changed": false, "body_chars_added": 412 },
  "reason": null
}

actor.type is one of user, agent, or system. An agent acting on a person’s behalf still attributes to that person — the delegated user id sits in actor_id — while actor.type: agent and actor_channel: mcp make it clear the proximate cause was the agent rather than the person clicking in the dashboard. Platform-internal actors with no delegated user (the updater, scheduler, queue workers) use system.

When an Org Owner is logged in as another user, every event is attributed to the real Org Owner — not the person they’re viewing as — and tagged with who they were viewing as, so “Jane did X while viewing as Bob” is always distinguishable from Jane acting as herself.

The structured form is consistent across event types: a monotonic time-ordered identifier, the actor (type, delegated user id, channel), the target object, a structured diff for state changes, and the reason text for the actions that require one (an env-var change, a co-owner added or removed, an integration grant requested). Auditors don’t have to interpret different shapes for “app paused” vs “Knowledge accepted” vs “integration revoked” — the wrapping is the same.

Data access is audited per call

A proxied integration writes one audit record per call: the app, the acting user when one is present, the integration, the upstream URL, the HTTP method, the response code, and the latency. The credential value never appears — the record names which integration credential was used, not its contents. Because the credential is substituted inside the broker and never reaches the app, attribution is per user, not per shared credential: a service_account integration shows the bound user behind each call, and a user_delegated integration records the same user whose token was exchanged upstream.

When a call shows System instead of a person

Not every proxied call has a person behind it. Background work — timers, queue consumers, scheduled jobs, startup tasks — genuinely has no user, so System is the honest answer rather than a missing value. But the same System also appears when an app does serve a signed-in person and simply doesn’t tell Greenlight who: attribution rides a header the app forwards on calls it makes while handling a user’s request, and an app whose code predates that convention won’t be forwarding it.

The audit page’s User attribution panel separates the two. For each app with proxy traffic in the last week it shows how many calls named a user, and — crucially — whether that app served signed-in users at all in the same period. An app with no attributed calls and no signed-in users is doing background work and is behaving correctly; an app with no attributed calls that did serve people is the one worth fixing. The fix is in the app, not the platform: its agent forwards the header the next time it touches the code, and coverage improves from there.

Injected integrations — the cases where a database wire protocol or a request-signing scheme can’t ride an HTTP proxy, so the app connects directly with an IT-managed credential — are audited at the grant level (who was granted access to what) rather than per call, since those calls don’t transit the broker. The data broker page describes which integrations are proxied vs injected and why.

Local development is audited too

When a builder works locally, the bundled greenlight CLI pairs once over its MCP session and then runs the app with greenlight run. That loop is fully audited:

• Session pairing records when a CLI session was approved — the start of the local-dev trail for that builder.
• Session revocation closes it. A builder can end any one of their own CLI sign-ins, and IT can end anyone’s; either way the event names the session, whose it was, and whether the person ended their own or an administrator stepped in. Ending a sign-in also cuts that builder’s local data access, which is audited as its own event.
• Proxied calls from the laptop land in the same per-call data-access log as deployed traffic, attributed to the same user, because local greenlight run reaches the same broker URL the deployed app uses with a short-lived, app- and user-scoped token. No credential is ever on the laptop.
• Data inspection — reading a deployed app’s database, or querying a connected database the builder is granted — runs inside the control plane and returns results only. Each inspection writes its own audit event under the data_inspection category, naming the app or builder and what was inspected; a connected-database query is recorded by a fingerprint of the statement, not its text. Queries against a per-user connected database also record the database’s name for the developer when their token establishes it, including failed queries. Credentials are never returned and never logged.

No environment value or credential ever passes through the agent or MCP. The local-dev events make a builder’s exploration of production data as legible to IT as a deployed app’s runtime traffic.

Working the log

Open Audit for the org-wide view. Filters by event, app, actor, and time range are fields in the filter bar — there is no separate query language to learn, and everything composable through the UI is composable through the audit API the same way.

Useful day-to-day filters:

• Agent actions per user — who’s working with their agent the most and what they’re doing.
• Integration grants and revokes — the source of truth for “who got access to what.”
• Data inspection by app — every time a builder read a deployed app’s database from a local session.
• Kill/restore events — every incident-style action.

Runtime logs and metrics

Each app’s stdout and stderr are captured per pod and surfaced live in the Monitoring tab of the app’s detail page, scoped per app per environment, with search and time-range filtering inline. New lines appear as the app writes them. If the live connection drops, the view keeps refreshing every few seconds from the last line it showed and reconnects on its own. Default retention is configurable per org.

Per-app metrics cover container CPU, memory, and restarts out of the box, rendered as history charts with a selectable time range alongside the current snapshot, plus HTTP request rate and 4xx/5xx error rate read from the ingress controller; latency percentiles and per-upstream integration call rates extend this as the platform’s collection points light up. Each app also shows what it reserves beside what it uses: an app that declares a compute size holds that capacity for as long as it’s deployed, spent or not, so the unused remainder is shown next to the reading rather than left to be worked out. There is no separate APM agent to install — Greenlight collects from the platform’s natural seams (the ingress controller, the data broker, the Kubernetes runtime) and stitches everything together by app id.

Greenlight also answers the adoption question — is anyone actually using this app? Because every authenticated request passes through the platform’s SSO check, Greenlight counts per-app daily active users, request volume, and last-active time for every app automatically, with no instrumentation in the app itself. Only aggregates are kept: IT sees counts and trends, never a browsable history of which person opened which app when.

When a deployed app misbehaves, the agent’s first read is getAppDiagnostics: one snapshot of pod status, probe results, resource pressure, the env-var names with their status (values masked), dependency reachability, and whether the pod runs the latest merge. The app’s owner, its editors, and IT see the same snapshot in the app’s Monitoring tab. From there it reads logs and metrics and, for the data behind a bug, the read-only inspectAppDb tool described above. There is no shell into a running pod: a shell would act as the workload rather than the developer and bypass the pull-request path, so the logs, metrics, and inspection surfaces cover the troubleshooting need instead.

Greenlight watches itself the same way. Internal Health answers two questions in two places: whether the control plane is healthy now, from the newest reading (and it stops claiming health once readings stop arriving), and what happened earlier, from 30 days of control-plane history kept in the install’s own database. The history charts each Greenlight service’s memory and CPU against its replicas’ own limits, its restarts and its readiness, one service at a time, then cluster capacity, background work, request latency and errors, and database connections, with incidents and release changes marked on the same timeline. An interval nobody observed is shown as a gap rather than as a quiet one, and choosing a range never changes the current status. Recorded incidents are kept for 90 days after they resolve, longer than the metrics around them.

The dashboard isn’t intended to replace a full log aggregator. Customers who want long-term log analytics ship the cluster’s logs to their own aggregator using the cloud’s native log shipping (Azure Monitor, Cloud Logging, CloudWatch) — Greenlight doesn’t sit in that path.

Failure logs sent to Shift

Some failures leave a log that Shift needs to support you, and Greenlight can send that log to Shift’s fleet monitor on its own channel, separate from the health report. Each log is redacted on your install before it leaves, so secrets and tokens the redactor recognizes never cross. Each failure is sent at most once, and the monitor rejects anything over the size cap rather than cutting it. Shift keeps these logs for 90 days, and only Shift’s own operators can read them. Every accepted send is recorded in your audit log as fleet.failure_log.sent, with the kind of failure and how many bytes were sent, never the text.

These logs are sent today:

• Failed updates. When a Greenlight update run fails — including one stopped by its time limit — or applies its release but fails one of its closing checks, the run’s log is sent once, a few minutes after the run ends. The log holds the infrastructure plan and apply output, the application rollout, and the installer’s own output. Up to 2 MiB is sent; a longer log keeps its start and its end. Successful runs, previews, and checks that found nothing to apply send nothing. This stream needs no setting: Shift needs it to fix a failed update, and it describes Greenlight’s own infrastructure rather than your apps or data.
• Failed pipeline runs. When one of your apps’ pipeline runs fails, Greenlight sends one log for the run about a minute later: each failed check’s details, the build’s CI log tail, and the crash snapshot of a deploy that failed, one section per check. Up to 1 MiB is sent, shared fairly between the failed checks, each keeping its newest output. Passed and cancelled runs send nothing. This stream needs no setting: Shift needs it to help fix a failed build or deploy. The Feedback page’s Pipeline failures tab shows whether each failed run has reached Shift.
• Control-plane incidents. When a Greenlight service in your cluster crashes — the async worker running out of memory, say, or ingress, certificates or the screenshot renderer exiting with an error — Greenlight captures the crashed container’s last output for the incident, as Internal Health already shows you. When background jobs give up after their last retry, the incident’s capture is those jobs’ last errors. Each capture is sent once, within the minute it is taken, up to 1 MiB, and a recovery Greenlight sees later is sent after it without the log. Incidents with no captured log send nothing. This stream needs no setting: these logs describe Greenlight’s own services, not your apps or data. An ingress or certificates log can name your apps’ hostnames, routes and the addresses Greenlight routes them to, because those are what Traefik and cert-manager work on; it never carries request content, and credentials are redacted like everywhere else.
• Failed scheduled background jobs. Greenlight’s control plane runs its own work on a schedule — the minute-by-minute health snapshot, the fleet report, the nightly clean-ups. When a run fails, Greenlight keeps that run’s log on your install: what the job logged during the run, then its error and stack, redacted, up to 256 KiB, keeping its start and its end. A job that keeps failing sends Shift one log for each different error, at most five, until it next succeeds, so a job failing every minute with one error sends one log, not one a minute. Errors are compared with their times and IDs removed. This stream needs no setting: these jobs are Greenlight’s own periodic work, not your apps or data. Background work done for one of your apps or orgs (webhook events, notification emails, screenshots) keeps its log on your install and never sends it.
• App crashes. When one of your apps’ containers exits abnormally, Greenlight keeps its last output before the crash, redacted, up to 256 KiB, for 30 days, and lists it on the Feedback page’s App crashes tab. It reaches Shift only if an org admin turns on Send app crash logs to Shift automatically in Settings (off by default), in which case each new crash is sent as it is recorded (or, when its log could not be read yet, up to 10 minutes later so a retry can fill it), or chooses Send to Shift for specific crashes (held the same way while a log could still be filled; choosing it for a crash whose send failed tries again). Turning the setting off stops crashes that were queued but not yet sent. Both the setting change (org.crash_log_auto_share.changed) and each manual send (app_crash.share.requested) are audited.
• App health alerts. Every crash loop is listed on the Feedback page’s Health alerts tab for 30 days, whether or not a notification went out. An alert reaches Shift only when an org admin selects it and chooses Send to Shift; there is no automatic setting. It carries the app, how many restarts in what window, and whether Slack or email was notified — no log. Each send is audited as health_alert.share.requested.

Retention and exports

Audit events live in the control plane’s audit store and stay queryable for as long as they are kept — there’s no cold-archive tier to re-fetch from; search and exports work uniformly over the whole range.

How long they are kept is your choice. Under Settings → Policies → Audit log retention, IT picks 90 or 180 days, 1, 2 or 7 years, or Keep indefinitely. A new install starts at 90 days; an install that already had audit history when the setting arrived starts at Keep indefinitely, so upgrading never deletes anything. Each night Greenlight deletes events older than the window, and records that it did (audit.retention.pruned, with how many and the cutoff). The Audit page shows the window in force and the oldest event still held.

More detail: choosing a window

• Match your auditor’s look-back. SOC 2 and ISO 27001 reviews commonly ask for 12 months of evidence; pick at least 1 year if you rely on this log for them.
• Shortening is permanent. The page warns before you save a shorter window: everything older is deleted at the next nightly run and cannot be recovered.
• Changes are audited. org.audit_retention.updated records who changed it, from what, to what.
• Only the Audit page loses history. The rest of the dashboard keeps its own record: an integration’s registered by, a Knowledge entry’s edit history, the date token usage reporting started, the last Slack delivery outcome, and how long an app has waited to delete all stay as they were.
• Ninety days is the minimum, because Greenlight’s cost and usage reports read audit events back more than a month.

CSV/JSON export for any time range and a SIEM webhook (push to Splunk, Datadog, or any HTTPS endpoint, HMAC-signed, with cursor-based replay) are coming soon. The export patterns they’re designed for: a quarterly export for SOC 2 or ISO evidence files, a targeted export filtered by user, app, or integration for an internal investigation, and a continuous feed into a customer-side SIEM.

Tamper resistance

Today the log is append-only at the storage layer — nothing edits an event, and the only deletion is the retention window above — with access restricted to admins through the dashboard and the audit API, backed by the control-plane database’s point-in-time recovery. The next hardening step mirrors every event into a WORM (write-once, read-many) object store with a time-based immutability lock and per-entry HMAC signing — the storage layer refuses overwrites within the retention window, and the HMAC lets a reviewer detect any altered field, so the evidence becomes tamper-evident rather than append-only by convention.

What’s not in the log

Request and response bodies for runtime traffic are not in the audit log. Audit records reference an integration call and capture the call’s metadata (URL, method, response code, latency, bound user), but they do not store payloads. The same holds for data inspection: an inspection event records that an app’s database was read and by whom, never the rows or the credential. Apps that need payload-level evidence — an SAR fulfillment trail, say — should write that into their own dedicated store; Greenlight’s audit log is not designed to be a payload archive.

Next

Governance & policyThe actions that emit the most important audit events.

Data brokeringHow proxied and injected integrations differ — and how each is audited.

Kill switch & restoreThe IT workflow that generates the highest-signal events.

ComplianceHow the audit log feeds SOC 2, HIPAA, and friends.