ARCHIVED OFFICIAL INCIDENT

Elevated 404s for GitHub App s2s tokens in git/API

GitHub · Last saved provider status: resolved

Current service report →
This is historical evidence.

The last saved event status is not the current service status. Source lists can be incomplete, and disappearance from a source does not confirm recovery. Open the current report or the official source for newer evidence.

Saved event details

Provider status
resolved
Provider impact
none
Provider record created
27 Oct 2026, 08:00:00 UTC
Provider record updated
27 Oct 2026, 20:21:53 UTC
Explicit provider start
27 Oct 2026, 20:21:53 UTC
Explicit provider end
27 Oct 2026, 08:00:00 UTC

The provider's end time precedes its start time. These reported timestamps do not establish a valid event duration.

Affected components reported by the provider

The provider explicitly listed no affected components in this saved revision.

These associations describe this event’s reported scope. They do not establish current component availability or verified dependencies.

A record creation time is not necessarily an outage start. Unreported times remain unavailable. We do not calculate downtime from collection times.

Provider updates in saved revisions

Latest first. We show up to 100 distinct updates from the latest 20 saved content revisions. Revised wording at the same provider time is retained separately.

  1. resolved

    Between 0655 and 0801 UTC on 10/27, customers experienced increased failure rates for API and git requests authenticated with server-to-server tokens for GitHub Apps (excluding Actions tokens). During this window, authenticated git operations mistakenly returned 404 for 1.24% (2.6x) of requests on average and 2.49% (5.3x) at peak. These failures were caused by a scheduled maintenance event where GitHub migrated 7 tables between two MySQL clusters in our IAD and Azure Central US datacenters. One of the affected tables stores installation records for GitHub App server-to-server tokens (excluding Actions). We've successfully completed several of these migrations in recent months without notable customer impact. However, in this case, the migration coincided with a scheduled cleanup job on the installation records table, which deleted approximately 59 million records from 0444-0617 UTC. As a result, the gh-ost tooling used in the migration experienced inter-cluster replication delays (up to 1 hour at peak). At 0655 UTC, we begin transitioning reads of the target tables to the new MySQL cluster, with 100% of reads cutover at 0721 UTC. Customer impact began, but was not detected, throughout the read cutover and persisted until completion of the write cutover at 0801 UTC. We are updating our processes and automation to prevent a repeat of this incident, including disabling overlapping cleanup jobs prior to these migrations and blocking the cutover of reads when replication delays are high.

    Seen in a saved revision at 27 Oct 2026, 20:23:07 UTC

Cookie preferences

With your permission, we use Google Analytics and Umami to understand service queries. These tools use cookies or browser storage. Optional analytics stay off until you accept.

Cookies and storage details