Why Google’s Infrastructure Is the Foundation for Everything

When YouTube loads instantly for hundreds of millions of users simultaneously, most people don’t think about the underlying infrastructure making that possible. According to KC Ayyagari, this is by design. Google has spent roughly 18 years building its own hardware, software, data centers, and networking stack — all tuned for speed, reliability, and scale. That same infrastructure is now exposed to developers and businesses as Google Cloud managed services.

Google’s Network Scale

During the masterclass, Ayyagari cited that Google carries approximately 25 percent of the world’s internet traffic. He explained a key differentiator in how Google routes that traffic: while most cloud and network providers use a hot potato architecture — handing packets off to the nearest external node as quickly as possible — Google uses a cold potato architecture. This means Google retains packets within its own faster private network for as long as possible before handing them off to the nearest ISP, reducing latency and improving reliability.

Machine Learning Applied to Data Center Cooling

Google also shared a concrete example of using its own ML capabilities internally: managing data center cooling. By training a model on data center telemetry to predict and control optimal temperatures, Google reportedly reduced cooling-related energy costs by 40 percent. This is a practical illustration of applied ML — not a research project, but a production system managing real infrastructure at scale.

Google Cloud: 100+ Products Across Key Domains

Google Cloud (cloud.google.com) provides over 100 products spanning:

• Machine Learning & AI — AutoML, AI Platform, pre-built APIs
• Big Data — BigQuery, Dataflow, Pub/Sub
• Compute — Compute Engine, Kubernetes Engine (GKE)
• Networking — Cloud CDN, VPC, Cloud Interconnect
• Databases — Cloud Spanner, Cloud SQL, Firestore

Ayyagari described the portfolio as “Lego pieces” — modular services that developers assemble based on what they’re building. The platform doesn’t prescribe a single architecture; instead it offers components that can be composed to build anything from a simple web app to a globally distributed ML pipeline.

Google’s ML Ecosystem: Something for Every Role

One of the clearest takeaways from the session was that Google has segmented its ML offering by user persona, not just by use case:

What Is Google Cloud AutoML?

AutoML sits in a useful middle ground: you bring your own labeled data, but you don’t need to design or tune a neural network architecture. Google handles model selection, hyperparameter tuning, and training on its infrastructure, using techniques like transfer learning — starting from a pre-trained model and fine-tuning it on your specific dataset.

The typical AutoML workflow still follows the general ML lifecycle:

1. Data acquisition — Collect relevant labeled examples
2. Data exploration — Understand distributions, class balance, quality
3. Data preparation — Clean, format, and split data
4. Feature engineering — Select or transform input variables (partially automated in AutoML)
5. Model selection & training — Handled automatically by AutoML
6. Prediction / serving — Deploy the trained model to an endpoint

The key advantage AutoML offers is compressing steps 4 and 5. You don’t need to choose between ResNet, EfficientNet, or BERT — AutoML runs a neural architecture search or applies transfer learning from appropriate base models and returns a trained, deployable model.

Google Cloud AutoML Services: An Overview

At the time of the masterclass (late 2019), Google’s AutoML portfolio covered four primary domains:

AutoML Vision

Train custom image classification or object detection models. Upload labeled images, trigger a training job, and receive a model that can classify new images into your custom categories — without writing any training code.

AutoML Video Intelligence

Extend image-based concepts to video — classify video segments or track objects across frames using custom training data.

AutoML Natural Language

Build custom text classification models. Useful for sentiment analysis, content tagging, or routing — on domain-specific text that generic APIs don’t handle accurately.

AutoML Translation

Customize Google’s translation models using your own bilingual sentence pairs — especially valuable for industry-specific or technical terminology where generic models underperform.

AutoML Tables

Apply AutoML to structured (tabular) data — the most common data type in enterprise workflows. Upload a CSV or connect a BigQuery table, select the target column, and AutoML Tables trains and evaluates multiple model types to find the best performer for regression or classification tasks.

The Role of BigQuery in the ML Pipeline

Ayyagari also touched on BigQuery — Google’s serverless, highly scalable data warehouse — as a core part of the data pipeline feeding AutoML and other ML workflows. BigQuery allows teams to run SQL queries across petabyte-scale datasets in seconds, making it a practical starting point for data exploration and preparation before training begins. AutoML Tables integrates directly with BigQuery, allowing you to train a model from a BigQuery table without exporting data.

Why AutoML Matters for Speed-to-Market

A recurring theme in Ayyagari’s session was time. ML model development typically involves expensive iteration cycles: collect data, engineer features, train, evaluate, adjust, repeat. For startups and product teams without dedicated ML engineers, this loop is prohibitively slow.

AutoML accelerates the experimentation cycle. Teams can validate whether a custom ML model improves their product — using real business data — before committing to a full ML engineering investment. The tradeoff is reduced control over model internals, which may matter less at early stages than shipping something that works.

“Machine Learning is all about experimentation. The moment you do any experimentation, you have to come to the market quickly. If you keep wasting your resources on training, the data, it’s of no use. And this is the problem we are trying to solve with AutoML.”

— KC Ayyagari, Startup Ecosystem Manager – India, Google Cloud for Startups (TechSparks 2019)

Key Takeaways

• Google Cloud exposes the same infrastructure Google uses internally as managed services with SLAs.
• Google’s cold potato network architecture prioritizes retaining traffic within its own low-latency network before handing off to external ISPs.
• The Google Cloud ML portfolio spans the full spectrum from pre-built APIs to custom model training, covering every technical persona from app developer to ML researcher.
• AutoML uses transfer learning to train custom models on your data without requiring ML coding expertise.
• AutoML services cover vision, video intelligence, natural language, translation, and tabular data.
• BigQuery serves as a natural data source for AutoML Tables and structured ML workflows.
• The primary value proposition of AutoML for engineering teams is dramatically reducing the time required to validate and ship ML-powered features.

Note: Google’s AutoML product portfolio and naming conventions have continued to evolve since 2019 under the Vertex AI umbrella. Check the official Vertex AI documentation for current product names, capabilities, and pricing.

Serverless deployment isn’t just for startups anymore.

Mainstream companies have revved up DevOps velocity and slashed IT operations overhead with the addition of serverless technologies to underpin new apps. These companies, which base modern apps on AWS Lambda, OpenWhisk and Google Cloud Functions, say that serverless deployment abstracts the underlying cloud infrastructure completely from app developers, and lets DevOps teams focus on business logic and application code instead of infrastructure management.

But while serverless deployment can pay off, so far, traditional companies have found it easiest to adopt serverless infrastructure when a new project begins. Consumer robot maker IRobot Corporation tried a serverless approach on AWS Lambda in 2015, when the Bedford, Mass., maker of the Roomba vacuum cleaner was unhappy with a turnkey cloud service provider for its IoT application.

The company found that the turnkey platform wasn’t scalable or extensible enough when it added sensors to its consumer products and so it decided to replace that platform. Starting fresh gave IRobot an opportunity to rethink its infrastructure and relaunch the cloud robotics application on AWS, where it now uses more than 30 of the cloud provider’s services, including AWS IoT Services and Lambda. The serverless deployment for this project falls to a DevOps team separate from the one that handles IRobot’s existing applications.

“It takes single-digit operations headcount to run, and basically runs itself,” said Ben Kehoe, cloud robotics research scientist at IRobot. “It’s been a very successful serverless story, but it’s not one of a transition from a legacy system.”

The cloud robotics project is well-suited to serverless deployment because AWS IoT uses event-driven communication between services, which dovetails nicely with Lambda functions as a service, Kehoe said.

Serverless deployment also meant IRobot could redeploy the cloud robotics project quickly, and focus on how to make it work for customers, rather than how to build the underlying cloud infrastructure.

“It meant that, as an organization, we didn’t have to develop the expertise in scalable, elastic cloud application development,” Kehoe said. “We were able to completely leapfrog that and work primarily in the [business] problem domain.”

For app developers, this statistic can seem especially grim. No one wants their hard work to end in abandonment, but it’s hard to figure out where you’re missing the mark. This is where user feedback and user evaluation comes in. But how do you know which technique to use?

Many app developers prioritize user feedback over other evaluation techniques, such as heuristic evaluation and A/B testing. But is user feedback the only method you should use? Is there such a thing as too much user feedback?

What follows is everything you need to know about the assessment methods available to you and how they can give you a fuller picture of your app’s user experience.

How much user feedback is too much?

The top innovators — like Steve Jobs and Henry Ford — would argue that user feedback is unnecessary. From their perspective, user feedback stifles creativity and doesn’t even lead to products users actually want.

As a quote commonly attributed to Ford says, “If I had asked people what they wanted, they would have said faster horses.” According to Jobs, “It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.”

Ford and Jobs created innovative new products for which there was no initial demand, but those products quickly became indispensable. If you are building an app that is completely unprecedented, user feedback won’t be of much use. All it will do is stifle innovation.

It’s unlikely your app is truly revolutionary, which means there is a place for user feedback in evaluating its strengths and weaknesses. But there’s also an important lesson here: Users can’t tell you what your next product should be. They cannot present you with fully-formed app ideas.

Prototyping and user feedback will provide incremental benefits to your app by improving its functionality. For most app developers, the question is not whether user feedback should be eschewed, but how much feedback is useful.

There is a danger in allowing projects to become user-led. Few users are knowledgeable enough to know whether their needs and wants are practical. Even when surveying a large sample of users, there is a chance those surveyed will not represent the needs of all users.

It’s important to be deliberate in how you collect user feedback, and how much of it you get.

Many companies fall into some common pitfalls when soliciting user feedback. Here are some tips on getting good data you can convert into action.

Don’t ask users what they want. This is too generic a line of inquiry that often results in feedback that is both overly specific and not entirely accurate. Instead, pursue more fruitful lines of questioning, such as:

• What moves users emotionally?
• What features do users find frustrating or difficult to use?
• What features do they most enjoy? Why?
• What problems does the app solve for its users? What doesn’t it solve?

Avoid closed questions. Don’t ask questions that can be answered with a simple “yes” or “no.” This doesn’t start a conversation or give any meaningful insight because it doesn’t get to the why.

Remember: One opinion is not data. One user’s opinion shouldn’t be enough for you to change your app. Look for trends by clustering your data samples. If 100 users have difficulty with a particular feature, that carries more weight than if just one user struggles.

Look at user behavior patterns. Actions speak louder than words. If your app has a feature users claim is useful but don’t actually use, it’s not useful to them. Examine what your users do, not merely what they say.

Other ways to evaluate your app

While user feedback has its place, it is not the only way to evaluate your app. Heuristic evaluation and A/B testing can be valuable to the product development process.

Heuristic evaluation. “Heuristic” is a fancy word for “trial-and-error” or “hands-on.” As part of heuristic evaluations, user-experience specialists dig into the interface of an app to identify usability issues. This is an efficient and effective way to get an overview of app UX and find the main problems that are affecting its usability.

Heuristic evaluators need to understand the end user’s needs and goals so they can evaluate the app from the user’s perspective. It is helpful to develop user personas, so evaluators know who is using the app and why.

A/B testing. Some developers view A/B testing as a waste of time, and would rather focus on developing and tweaking their app. This is unfortunate, as A/B testing allows you to move from making technical and design choices based on hunches to making them based on data.

Everything from the layout of the user interface to the color of the buttons can be subjected to A/B testing. Google, for example, tested what shade of blue to use for its text links. They discovered that one particular shade made people more likely to click, which Google converted into an extra $200 million in ad revenue. It’s unlikely Google would have seen such benefits had it asked its users what color its links should be.

Different evaluation methods serve different purposes, but used together, user feedback (solicited in the right way), heuristic evaluation, and A/B testing will give you a full picture of your app’s user experience and, most importantly, how to improve it.