Google suffers Sunday outage, impacts Cloud, YouTube, Gmail and more
Google Cloud experienced widespread issues on Sunday, June 2, impacting the search giant's own services, as well as that of its cloud customers.
The intermittent outage, which has since been resolved, was blamed on "high levels of network congestion."
Networking not working
Google services like YouTube, Nest and Gmail, as well as Cloud customers like Snapchat, Shopify, Vimeo and Discord, were impacted by the problem, which began around 12:15 Pacific time.
"We are experiencing high levels of network congestion in the eastern USA, affecting multiple services in Google Cloud, G Suite and YouTube. Users may see slow performance or intermittent errors," the company said on its status page at the time.
While the congestion occurred in the US, its impact was felt globally and was described by network monitoring company ThousandEyes as a "large scale" outage
The problem was resolved as of 4:00 pm Pacific time, with Google promising to "conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a detailed report of this incident once we have completed our internal investigation. This detailed report will contain information regarding SLA credits."
In a statement, the company apologized for the inconvenience and thanked customers for their "patience and continued support." It added: "Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better."
The outage which, among other things, meant that Nest users could not control their thermostats, comes after several major disruptions have impacted the largest cloud companies in recent years, highlighting the difficulty of building a resilient service, even with enormous resources.
Just last year saw Google Cloud go down due to a BGP error, Microsoft Azure be knocked out by a lightning strike, and Amazon Web Services be disrupted by a “power event.”
Update: Google's VP of 24x7, Benjamin Treynor Sloss, said in a blog post: "In essence, the root cause of Sunday’s disruption was a configuration change that was intended for a small number of servers in a single region. The configuration was incorrectly applied to a larger number of servers across several neighbouring regions, and it caused those regions to stop using more than half of their available network capacity. The network traffic to/from those regions then tried to fit into the remaining network capacity, but it did not. The network became congested, and our networking systems correctly triaged the traffic overload and dropped larger, less latency-sensitive traffic in order to preserve smaller latency-sensitive traffic flows, much as urgent packages may be couriered by bicycle through even the worst traffic jam.
"Google’s engineering teams detected the issue within seconds, but diagnosis and correction took far longer than our target of a few minutes. Once alerted, engineering teams quickly identified the cause of the network congestion, but the same network congestion which was creating service degradation also slowed the engineering teams’ ability to restore the correct configurations, prolonging the outage. The Google teams were keenly aware that every minute which passed represented another minute of user impact, and brought on additional help to parallelize restoration efforts."
source datacenterdynamics
Industry: Data Centre
Latest Jobs
-
- IAM Consultant- OKTA
- Germany
- upto €90,000 plus benefits
-
I am looking for an experienced IAM process Manager to help drive forward a series of IAM implementation for a global Manufacturing business, Ideally you will be skilled with Okta and have knowledge of PAM Solutions, You will be responsible for: Driving the design and continuous improvement of complex IAM solutions in close collaboration with business partners Consult on the optimisation of IAM processes and design proper IT-based solutions to meet availability and quality targets Define technical specifications for SW-development (standards, design patterns, test cases, scenarios) and manage the life cycle of designed solutions Actively scan for relevant innovations and new technologies to identify further potential for improving IAM solutions and processes using OKTA Analyse new features of the regular Okta releases We are looking for someone with strong IAM experience as an Architect, Analyst, Technical Engineer, or similar role in the Identity and Security domain Experience with relevant certifications in development/administration, design and configuration of the Okta IAM platforms Familiar with LCM - joiners, movers, leavers, application federation - SAML, OIDC, SCIM and many other IAM terms Good mix of competences in IAM business process and project management concepts and tools e.g., ServiceNow, Jira, PRINCE2, SCRUM (agile)
-
- SAP Security Consultant
- France
- upto €70,000 plus benefits
-
I am looking for an experienced SAP Security Consultant. The ideal candidate will have a strong understanding of SAP security concepts and be able to apply them to real-world scenarios. ideally you will also have experience with Securitybridge or Onapsis, or a similar SAP security tool. Your responsibilities will include: Reviewing and auditing SAP security settings and controls Identifying and remediating security vulnerabilities Implementing security best practices Educating users on SAP security Experience experience in SAP security Experience with Securitybridge or Onapsis, or a similar SAP security tool would be very advantageous Strong understanding of SAP security concepts Excellent problem-solving and analytical skills Excellent communication and presentation skills Fluent in French & English
-
- Post Grad MSc Cyber security - Junior Cyber Risk Analyst wanted. UK
- United Kingdom
- Entry role
-
The perfect start to your new Cyber Security Career. Post Graduate Cyber Risk Analyst Wanted. Are you are fresh from earning your Cyber Security MSc and eager to start your career in Cyber Security? We are looking for a recent post graduate to join a forward thinking Cyber Security Consultancy for the ideal entry role into Cyber Security. Whilst employed industry experience is not expected, as full training and support will be provided, a history of recent education in Cyber Security / Cyber Risk is essential. We are looking for someone with an inquisitive mind, who is confident to ask the right questions and who isn't afraid to challenge the status quo. Superb communication skills are a must (in person, written and verbal) This is a UK based role that is remote first with monthly travel (1-2 a month) to meet with the team and in time to meet clients. If you aren’t available to travel this isn’t the opportunity. We are unable to provide VISA sponsorship as there will be a requirement to achieve Security clearance If you're adaptable, open to fresh perspectives, and excited to be part of a forward-thinking team and looking for an opportunity to help make a difference in a Cyber consulting role, this opportunity is for you. For more information apply here……
-
- Microsoft Exchange Contractor | London | OUTSIDE IR35 | SC Cleared
- London
- OUTSIDE IR35
-
Microsoft Exchange Contractor | London | OUTSIDE IR35 | SC Cleared • We require someone that has experience of migration exchange from windows server 2012 to 2019. • In depth understand of On-Prem exchange server management and deployment. • Experience migrating On-Prem exchange servers from 2012 upwards. • Secure Email Gateway experience essential Due to the nature of the requirement the individual must be commutable to London 2-3 days a week.