Incident with several GitHub Services
GitHub · 마지막 저장 제공업체 상태: resolved
공식 출처 문구는 원어로 표시됩니다.
마지막 저장 사고 상태는 현재 서비스 상태가 아닙니다. 출처 목록은 불완전할 수 있고 소멸은 복구 확인이 아닙니다. 최신 증거는 현재 보고 또는 공식 출처를 확인하세요.
저장 사고 상세
- 제공업체 상태
- resolved
- 제공업체 영향
- critical
- 제공업체 기록 생성
- 13 Sep 2026, 09:16:11 UTC
- 제공업체 기록 갱신
- 15 Sep 2026, 21:47:16 UTC
- 제공업체가 명시한 시작
- 13 Sep 2026, 09:16:11 UTC
- 제공업체가 명시한 종료
- 13 Sep 2026, 10:44:55 UTC
제공업체가 보고한 영향받은 구성 요소
- Actions
br0l2tvcx85d - API Requests
brv1bkgrwx7q - Pull Requests
hhtssxt0f5v2 - Issues
kr09ddfgbfsf - Pages
vg70hn9s2tyj
연결은 사고 보고 범위이며 현재 구성 요소 가용성이나 검증 의존 관계를 입증하지 않습니다.
기록 생성 시각이 반드시 장애 시작 시각인 것은 아닙니다. 보고되지 않은 시각은 알 수 없음으로 유지하며 수집 시각으로 중단 시간을 계산하지 않습니다.
저장 수정본의 제공업체 업데이트
최신순입니다. 최신 저장 내용 수정본 20개에서 최대 100개의 서로 다른 업데이트를 표시합니다. 같은 제공업체 시각의 문구 변경도 별도 보존됩니다.
- resolved
On September 13, 2026, between 08:43 and 10:44 UTC, GitHub experienced degraded availability across approximately 28 services, including Issues, Pull Requests, Actions, Codespaces, Pages, Notifications, Code Scanning, Git LFS, and new account signup. At peak, 8.8% of requests to create GitHub App installation access tokens failed. Token issuance for Actions workflows was also affected, impacting approximately 4% of workflows during the incident time frame. Creating issues through the web interface failed for about 96% of attempts, and signup failures were above 90%. <br /> <br />The cause was an internal data-cleanup job that began writing to a shared database cluster at 07:33 UTC. That cluster stores permission data read on nearly every authenticated request. The safeguard that was pacing the background job watched only one health signal — how far the database replicas were lagging — and that signal stayed low the whole time. It did not account for the load building on the primary itself, so the job kept writing while the primary quietly ran toward its limit. <br /><br />When the primary ran out of available connections, requests that needed it could not complete. First, there was no quick timeout on these database calls, so request handlers waited on the stalled database instead of failing fast, and the shared request-handling capacity degraded into site-wide errors. Second, a retry loop around token creation kept re-sending the writes that were already failing, which held the database saturated rather than letting it recover. <br /><br />Monitoring declared the incident at 08:50 UTC, but due to the broad impact and amplification from token creation, it took time to identify the source of the load. First responders mitigated by shedding internal load and pausing the job, and all services recovered by 10:44 UTC. <br /><br />To prevent recurrence, we are rate-limiting background jobs against shared, customer-serving databases by default, and adding automatic pausing and paging on primary-server load rather than replication lag alone. We are also surfacing running background work directly alongside database health signals so responders can see and pause it without leaving those dashboards, bounding retries in the token-issuing path, and adding request-level timeouts so one unhealthy database cannot consume shared web server capacity. In addition, we are breaking apart this database cluster to remove the single point of failure. We will be moving various service-specific data, including the authorization data, out of this shared cluster in the next two weeks.
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC - investigating
Pull Requests is experiencing degraded performance. We are continuing to investigate.
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC - investigating
We have reduced load on this cluster with internal load-shedding and are seeing signs of recovery but continue to monitor
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC - investigating
We're seeing increased database replication delays on collab which is causing increased error rates in authorization endpoints and follow-on increased error rates across the system - we are investigating
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC - investigating
Actions is experiencing degraded performance. We are continuing to investigate.
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC - investigating
We are investigating reports of degraded availability for API Requests, Issues, Pages and Pull Requests
저장 수정본에서 확인 06 Oct 2026, 13:58:39 UTC