Incident with Webhooks

Gespeicherte Ereignisdetails

Anbieterstatus

resolved

Anbieterauswirkungen

minor

Anbieterdatensatz erstellt

13 Aug 2026, 14:45:40 UTC

Anbieterdatensatz aktualisiert

31 Aug 2026, 20:37:37 UTC

Ausdrücklicher Anbieterbeginn

13 Aug 2026, 14:45:40 UTC

Ausdrückliches Anbieterende

13 Aug 2026, 15:36:34 UTC

Vom Anbieter gemeldete betroffene Komponenten

• Webhooks 4230lsnqdsld
• Git Operations 8l4ygp009s5s
• Pull Requests hhtssxt0f5v2
• Issues kr09ddfgbfsf
• Packages st3j38cctv9l

Zuordnungen beschreiben gemeldeten Ereignisumfang, keine aktuelle Komponentenverfügbarkeit oder verifizierten Abhängigkeiten.

Die Erstellung eines Datensatzes ist nicht zwingend der Ausfallbeginn. Nicht gemeldete Zeiten bleiben unbekannt. Aus Erfassungszeiten berechnen wir keine Ausfalldauer.

Anbieterupdates in gespeicherten Revisionen

Neueste zuerst. Bis zu 100 verschiedene Updates aus den letzten 20 gespeicherten Revisionen. Geänderte Formulierungen bei gleicher Anbieterzeit bleiben separat.

1. resolved13 Aug 2026, 15:36:34 UTC

   Between 14:24 and 14:53 UTC on 13 August 2026, a routine background job to delete an organization overwhelmed a key shared database, causing multiple GitHub services to briefly return elevated errors and slower responses. Most affected was the webhook management API, with smaller impact to Git operations, pull requests, issues, packages, sign-in, and Copilot. Impact cleared on its own at about 14:53 UTC once the job finished; we resolved the incident at 15:36 UTC. <br /><br />Affected users may have experienced a brief increase in errors and slower responses, primarily when creating, listing, or updating webhooks, with smaller impacts to pull requests, issues, packages, and Git operations. Failures peaked at about 1% for several minutes around 14:37 UTC. <br /><br />To prevent future incidents, we've already shipped an update that turns on the safer deletion path for organizations, along with caps on deletion holds on databases. Building on these changes, we're auditing all bulk deletion and cleanup jobs that write to shared databases to prevent similar issues in future.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
2. monitoring13 Aug 2026, 15:33:54 UTC

   We have temporarily disabled a background job which caused the impact. At this time the impact is fully mitigated.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
3. monitoring13 Aug 2026, 15:33:02 UTC

   The degradation affecting Git Operations, Issues, Packages, Pull Requests and Webhooks has been mitigated. We are monitoring to ensure stability.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
4. investigating13 Aug 2026, 14:58:49 UTC

   We are currently investigating a brief degradation of service for Git operations (specifically pushes), issues, pull requests, package registry, and webhooks between 14:32 and 14:46 UTC. We have identified the source of the degradation and are investigating mitigation strategies to prevent recurrence.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
5. investigating13 Aug 2026, 14:56:45 UTC

   Packages is experiencing degraded performance. We are continuing to investigate.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
6. investigating13 Aug 2026, 14:46:52 UTC

   Git Operations is experiencing degraded performance. We are continuing to investigate.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
7. investigating13 Aug 2026, 14:46:23 UTC

   Issues is experiencing degraded performance. We are continuing to investigate.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
8. investigating13 Aug 2026, 14:46:11 UTC

   Pull Requests is experiencing degraded performance. We are continuing to investigate.

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC
9. investigating13 Aug 2026, 14:45:40 UTC

   We are investigating reports of degraded performance for Webhooks

   In gespeicherter Revision gesehen um 06 Oct 2026, 13:58:39 UTC