Google suffers Sunday outage, impacts Cloud, YouTube, Gmail and more
Google Cloud experienced widespread issues on Sunday, June 2, impacting the search giant's own services, as well as that of its cloud customers.
The intermittent outage, which has since been resolved, was blamed on "high levels of network congestion."
Networking not working
Google services like YouTube, Nest and Gmail, as well as Cloud customers like Snapchat, Shopify, Vimeo and Discord, were impacted by the problem, which began around 12:15 Pacific time.
"We are experiencing high levels of network congestion in the eastern USA, affecting multiple services in Google Cloud, G Suite and YouTube. Users may see slow performance or intermittent errors," the company said on its status page at the time.
While the congestion occurred in the US, its impact was felt globally and was described by network monitoring company ThousandEyes as a "large scale" outage
The problem was resolved as of 4:00 pm Pacific time, with Google promising to "conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a detailed report of this incident once we have completed our internal investigation. This detailed report will contain information regarding SLA credits."
In a statement, the company apologized for the inconvenience and thanked customers for their "patience and continued support." It added: "Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better."
The outage which, among other things, meant that Nest users could not control their thermostats, comes after several major disruptions have impacted the largest cloud companies in recent years, highlighting the difficulty of building a resilient service, even with enormous resources.
Just last year saw Google Cloud go down due to a BGP error, Microsoft Azure be knocked out by a lightning strike, and Amazon Web Services be disrupted by a “power event.”
Update: Google's VP of 24x7, Benjamin Treynor Sloss, said in a blog post: "In essence, the root cause of Sunday’s disruption was a configuration change that was intended for a small number of servers in a single region. The configuration was incorrectly applied to a larger number of servers across several neighbouring regions, and it caused those regions to stop using more than half of their available network capacity. The network traffic to/from those regions then tried to fit into the remaining network capacity, but it did not. The network became congested, and our networking systems correctly triaged the traffic overload and dropped larger, less latency-sensitive traffic in order to preserve smaller latency-sensitive traffic flows, much as urgent packages may be couriered by bicycle through even the worst traffic jam.
"Google’s engineering teams detected the issue within seconds, but diagnosis and correction took far longer than our target of a few minutes. Once alerted, engineering teams quickly identified the cause of the network congestion, but the same network congestion which was creating service degradation also slowed the engineering teams’ ability to restore the correct configurations, prolonging the outage. The Google teams were keenly aware that every minute which passed represented another minute of user impact, and brought on additional help to parallelize restoration efforts."
- IT Cyber Security Recruitment Consultant – Hot Desk
- Dependent on Experience
We are looking for an IT Cyber Security Recruitment Consultant – Hot Desk who has cybersecurity recruitment experience, with a track record of success. Most of the roles you will be required to recruit for will be within the salary region of £50k - £300k, experience placing candidates at this level is desirable. Responsibilities: To provide a consistent, high-quality level of service to new and current clients in order to build a long term working relationships with clients. Detailed, consultative approach to calling passive candidates in order to profile them in detail, against set criteria of skills and experience given to you by your client Achieve and exceed sales targets. Self-manage your daily tasks in order to make sure that the following day will be as successful as possible Have a structured approach with a solution selling ability as the sales cycles are not as quick.
- Principal Mechanical Engineer
- Up to £90,000 Base + Bonus
Principal Mechanical Engineer Location: London Salary: Up to £90,000 Base + Bonus A Principal Mechanical Engineer is needed for a state of the art, London based Data Centre provider. The Principal Mechanical Engineer will be responsible for all of the Mechanical components (support, development/design etc.) within our clients Data Centre’s. Other responsibilities include but not limited to; Commissioning, approving, design & review/improvement of new data centre infrastructure Commercial’s (Contract negotiation, project finances etc.) Project management Training/Development of other staff General engineering tasks Requirements HND / Degree in Engineering or equivalent. Must have current/recent experience (ideally in a senior position) within a mechanical/electrical position ideally within a DC or Consultancy background Candidates must be UK based and unfortunately, our client are unable to provide sponsorship Ref: PG7608 (M&E Jobs, Mechanical & Electrical Jobs, Engineering Jobs, Data Centre Jobs, Data Center Jobs)
- Carrier Service Manager / Access Delivery manager
- Up to £70,000 Base + 10% Bonus + Benefits
An Access Delivery Manager (Carrier Service Manager) is required for this Global services provider, to be responsible for vendor management across the European region and to manage the purchasing and delivery of 3rd party Access. Your key responsibility will be to conduct regular service reviews with key providers across the region, provide performance feedback, prepare & present vendor balanced scorecards, drive action plans for improvements and drive the vendor profile management strategy. Alongside the access management, you will act as the services manager to the 3rd party suppliers ensuring that the services being offered are being delivered as sold. If you are interested in this position, you will need the background from a telecoms provider where you have had experience of dealing with 3rd party suppliers, negotiating access services. It is key that you have a relationship with international carriers. Ref RA7293 (Telecoms Jobs, Telecommunications Jobs, Telecommunications Jobs in London)
- Cloud Sales Specialist
- Up to £100,00 Base + Double OTE
DCL are currently working on behalf one of the fastest growing service providers in London who are on the look out for a Cloud Sales Specialist. The Cloud Sales Specialist will be responsible for selling (opening and closing new business opportunities new business) and being the SME in all things Cloud providing support to other members in the sales team. Preference will be given to the Cloud Sales Specialist who possesses Exceptional knowledge of Cloud Technology (Public / Private / Hybrid.). Proven sales experience of identifying and closing new business within the Cloud market. Must be currently selling into the enterprise market. Consistency on tenure in current and past roles. New business background is a must In return you will be working for a successful, growing SME organisation with excellent sales support from pre-sales, post-sales, project management, service management, bid management, pricing and customer service. Reference Number: BD7588 (Cloud Sales Jobs, Cloud Computing Jobs, Cloud Computing Sales)