At GitLab, we are focused on improving developer productivity and satisfaction. And GitLab 13.6 has all the right ingredients to help you achieve all that and more! We hope that you find these top features, and the 60+ new features and improvements packed in this release, useful.

Improved ease of use and automation for efficiency
To make it easy for you to get started with GitLab CI/CD with Amazon Web Services (AWS), Auto DevOps has now been extended to support AWS, so you can now Auto-Deploy to Amazon EC2 using Auto DevOps without using Kubernetes (as previously required by Auto DevOps).

Docker Hub has enforced rate limits on docker pull requests. We have mitigated the impact for our SaaS and self-hosted users and have shared ways to monitor the limits with Prometheus in your environments. We want all our users to stay safe with their CI/CD pipelines and Kubernetes clusters. We are moving the Dependency Proxy to Core available for everyone.

Listening to the community’s feedback to have a more descriptive default branch, Group owners now have more flexibility in configuring a custom default initial branch name for new repositories as opposed to the master branch. Speaking of defaults, the Static Site Editor can use a default merge request template across a project, reducing the need to navigate to the merge request after submission to update the description.

Improved visibility for faster decision making
You cannot fix what you cannot find. With 13.6, we’ve made improvements to several dashboards and reports to aid you with faster decision making.

With the code quality severity included within the merge request and the Full Code Quality Report, you can now quickly determine which code quality violations are critical to resolve before merging. Thanks for the community contribution with Code Quality Report, Vicken Simonian!

We’ve made updates to the Project Security Dashboard to include the results of the latest run pipeline security scan and also a dynamic vulnerability trend chart to help you stay on top of the real time and historical vulnerability trends. We’ve also added the fuzz testing results in the merge request along with the other security results and improved the readability of this report by adding the source file name and line number to help you quickly find the exact crash location and fix it.

GitLab Self Managed Administrators can now see their organization’s usage trends for popular features such as users, projects, groups, issues, and pipelines over the last 12 months.

Improved extensibility for a seamless workflow
We aim to play well with other popular tools you may be using in your environment so that you have a seamless experience, even when you use only a few parts of GitLab. With 13.6, to enable easy access and collaboration from within VS Code, we’ve improved our extension with VS Code to insert Snippets, view and comment on merge requests and issues directly from VS Code rather than switching to the GitLab interface.

GitLab integrations can now be configured at a group level in addition to instance and project level – helping group owners manage integrations with ease.To enable you to grow beyond the 10GB per project storage limit, we recently introduced an add-on to purchase additional storage for your group or personal name space. In addition to Dependency Proxy, we’ve also moved Tracing to Core as part of this release.

These are just a few highlights from the many new features and performance improvements described below. If you’d like to preview what’s coming in next month’s release, check out our Upcoming Releases page as well as our 13.7 release kick off video series where the Product Managers highlight key features coming in the next release.

The following sections walk through each phase of a typical DevOps lifecycle and explain how real-time messaging adds value at each step.

1. Plan

Your messaging platform is the system of record for team conversations related to planning new products and features. Teams collaborate on ideas, debate technical approaches, and reach decisions — all within a searchable, persistent history that keeps stakeholders informed and aligned.

• Document sharing: Roadmaps, diagrams, and presentations can be shared directly in a channel, reducing context-switching to email or file servers.
• Threaded conversations: Threaded replies keep planning discussions organized and easy to reference later.
• Planning tool integrations: Integrations with tools such as Jira let teams surface issue details, create tickets, and update statuses without leaving the messaging interface.
• Video conferencing: When a text conversation needs to become a voice call, integrations with platforms such as Zoom or Webex make it simple to start a meeting from within the channel and return to the chat thread afterwards.

2. Code

As developers write code, they naturally turn to each other for collaboration and support. Whether the team is co-located or fully remote, a shared virtual workspace enables real-time problem-solving and peer review.

• Code snippet sharing: A messaging platform with language syntax highlighting lets developers paste a code block directly into a conversation and see it rendered with correct formatting, making reviews faster and clearer.
• Source control integration: Integrations with platforms such as GitLab or GitHub allow developers to receive pull request notifications, review diffs, and merge code without leaving their messaging workspace.
• Async collaboration: Persistent message history means remote team members working across time zones can pick up a conversation exactly where it left off.

3. Build

Integrating your messaging platform with your build system makes the build process transparent and actionable without requiring engineers to switch between tools.

• Slash commands: Custom slash commands let team members trigger a build, check its status, or retrieve artifact information directly from the messaging UI.
• CI tool plugins: Plugins for tools such as Jenkins expose job status, build logs, and failure alerts inside the team channel, so engineers are notified immediately when a build breaks.
• Automated notifications: Build success or failure messages posted to a dedicated channel ensure the right people are informed without manual follow-up.

4. Test

The testing phase requires close collaboration as teams validate new releases, identify defects, and fix bugs under time pressure. A well-integrated messaging platform can serve as a centralized testing hub.

• Webhooks: Use webhooks to connect CI pipelines and bug-tracking tools to your messaging platform, automatically posting test results, failure alerts, and bug reports to the appropriate channel.
• Triggered test runs: Teams can trigger automated test suites from within the messaging interface using bot commands or slash commands, removing the need to access a separate CI dashboard.
• Real-time visibility: Everyone involved — developers, QA engineers, and product managers — can watch test results appear in real time, reducing the communication lag between finding and fixing a defect.

5. Release

At release time, the messaging platform acts as a real-time broadcaster, ensuring that developers, operations engineers, and business stakeholders all know when a build has shipped and whether it was successful.

• Automated release notifications: Integrating release packaging tools such as Jenkins into your release workflow enables automatic notifications to the right channels when a release is packaged and ready.
• Cross-team transparency: Developers can accelerate follow-on work knowing a release has landed, while ops engineers can prepare for the incoming deployment without waiting for a manual handoff message.
• Audit trail: The persistent message history creates an informal audit trail of who was notified and when, which can be useful during post-release reviews.

6. Deploy

Following code to its final destination in production, the DevOps team needs visibility into which servers received the deployment and any relevant metrics, so they can troubleshoot issues quickly.

• Deployment tool integrations: Whether your team deploys using Puppet, Kubernetes, AWS services, or another solution, your messaging platform can receive automated deployment event notifications and surface them in a dedicated deployment channel.
• Rollout tracking: Automated messages can report the progress of a rolling deployment — for example, how many nodes have received the update — giving the team real-time confidence in the rollout.
• Rapid incident response: If a deployment triggers an error, the alert can surface immediately in the same channel where the deployment was announced, collapsing the gap between detection and response.

7. Monitor

Monitoring is where real-time messaging arguably delivers its greatest operational value. Posting alerts, health metrics, and incident updates directly into a messaging channel transforms how teams respond to production issues.

• Monitoring tool integrations: Tools such as Nagios, Datadog, or PagerDuty can be configured to post alerts and performance data directly to designated channels, making application health visible across the organization — not just to the on-call engineer.
• Incident response channels: When an outage occurs, a dedicated incident channel can be created instantly to bring together the right engineers, share diagnostic data, and coordinate a resolution — all with a preserved log of the incident conversation for the post-mortem.
• Historical data: Posting periodic metric summaries to a channel gives both engineers and business stakeholders a shared view of system health trends without requiring access to a separate monitoring dashboard.

Choosing the Right Integrations

The tools mentioned above — Jira, GitLab, Jenkins, Puppet, Kubernetes, Nagios, Zoom — are illustrative examples of a broader ecosystem. When evaluating a messaging platform for your DevOps environment, prioritize the following capabilities:

• A rich integration marketplace or support for custom webhooks and slash commands
• Persistent, searchable message history
• Granular channel and notification controls to prevent alert fatigue
• Support for code syntax highlighting
• Enterprise-grade security and compliance features appropriate for your industry

When a messaging platform is woven into every stage of the DevOps lifecycle, it becomes far more than a chat application. It becomes the connective tissue that keeps distributed teams aligned, accelerates incident response, and provides the transparency that continuous delivery demands.

Originally sourced from sdtimes.com (2019).

Editorial note: The original source for this article was a third-party market-research report teaser (circa 2019) that masked all financial figures with placeholder values. No revenue, CAGR, or market-size numbers are reproduced here because none were verifiable. The vendor profiles and market structure below are based on publicly known information about each company’s deployment automation offering.

What Is Deployment Automation?

Deployment automation is the practice of using tooling and scripting to move application artifacts—binaries, container images, configuration files—through a defined pipeline of environments (development, staging, production) with minimal or no manual intervention. It is a foundational component of Continuous Delivery (CD) and sits downstream of Continuous Integration (CI), which validates code quality.

Core capabilities that define a deployment automation platform include:

• Pipeline orchestration – defining the ordered sequence of build, test, and release stages.
• Environment management – provisioning or targeting infrastructure for each stage.
• Rollback and approval gates – automated or human-in-the-loop controls before production promotion.
• Artifact management – versioning and storing release artifacts for traceability.
• Audit and compliance logging – capturing who deployed what, when, and where.

Key Vendors in the Deployment Automation Market

The market is served by a mix of large cloud platform providers, dedicated CI/CD specialists, and infrastructure-automation vendors. The following companies are widely recognized as significant players.

Microsoft – Azure Pipelines / Azure DevOps

Microsoft integrates deployment automation into its Azure DevOps suite. Azure Pipelines supports YAML-based pipeline definitions and integrates natively with Azure cloud services, GitHub repositories, and third-party registries. It covers both Windows and Linux build agents and supports deployment to Kubernetes, Azure App Service, virtual machines, and on-premises targets via deployment groups.

JetBrains – TeamCity

JetBrains’ TeamCity is a mature CI/CD server with strong support for build chaining and artifact dependencies. It is popular in Java and .NET shops and integrates with a wide range of version-control systems and cloud providers. TeamCity can be self-hosted or run as a cloud-managed service.

Octopus Deploy

Octopus Deploy is purpose-built for the CD portion of the pipeline. Unlike general-purpose CI tools, it focuses specifically on release management, environment promotion, and deployment orchestration. It integrates with most CI systems (TeamCity, Azure Pipelines, Jenkins, GitHub Actions) as the release delivery layer, and supports runbooks for operational tasks beyond deployments.

GitLab Inc. – GitLab CI/CD

GitLab provides an end-to-end DevOps platform where the source repository, CI pipeline, container registry, and CD pipeline exist within a single application. GitLab CI/CD uses a .gitlab-ci.yml file at the repository root to define pipeline stages, including deployment jobs. GitLab Environments and Deployments provide visibility into where each commit has been deployed.

AppVeyor

AppVeyor is a cloud-based CI/CD service with particular strength in Windows build support—a differentiator in an ecosystem where Linux-first tools historically dominated. It is frequently used for open-source projects that need cross-platform build and deployment pipelines.

Atlassian – Bitbucket Pipelines / Bamboo

Atlassian offers two deployment automation paths. Bitbucket Pipelines is a cloud-native CI/CD service built into Bitbucket repositories, using a YAML configuration file and Docker-based build environments. Bamboo is Atlassian’s self-hosted CI/CD server, which integrates closely with Jira and Confluence for project tracking and documentation.

DeployBot

DeployBot specializes in code deployment from Git repositories to servers via FTP, SFTP, or SSH, and to cloud platforms such as AWS, Heroku, and others. It targets smaller teams and straightforward web-application deployment scenarios.

CircleCI

CircleCI is a cloud-first CI/CD platform known for its fast build execution and Docker-native pipeline model. Its orbs ecosystem provides reusable, shareable pipeline configuration components that simplify integration with deployment targets such as AWS, GCP, Kubernetes clusters, and Heroku.

Amazon Web Services – CodePipeline / CodeDeploy

AWS provides deployment automation through a family of services: CodeCommit (source), CodeBuild (build), CodeDeploy (deployment agent), and CodePipeline (orchestration). CodeDeploy automates application deployments to Amazon EC2 instances, on-premises servers, Lambda functions, and Amazon ECS services, with built-in support for blue/green and in-place deployment strategies.

Codeship

Codeship (now part of CloudBees) offers both a hosted CI/CD service and a Docker-based pipeline product. It focuses on developer simplicity and fast onboarding, with integrations for major cloud deployment targets.

Stackify – Retrace / Prefix

Stackify’s tooling sits at the intersection of deployment automation and application performance monitoring (APM). Its Retrace product tracks deployments as events correlated against performance and error data, helping teams assess the impact of each release on application health.

CloudBees ElectricFlow (now CloudBees CD)

ElectricFlow, rebranded as CloudBees CD, targets enterprise-scale deployment pipelines. It provides a visual pipeline editor, release orchestration across multiple teams and environments, and integration with enterprise systems such as ServiceNow for change management approval workflows.

PDQ Deploy

PDQ Deploy is an IT operations tool focused on software deployment to Windows endpoints in a managed corporate network. It is more aligned with IT operations and patch management use cases than with application release pipelines, but occupies a recognized niche in the broader deployment automation category.

Chef (Progress Chef)

Chef approaches deployment automation through infrastructure-as-code. Chef Infra manages server configuration and application deployment through Ruby-based recipes and cookbooks. Chef Habitat extends this to application packaging and runtime management, enabling deployable, self-contained application artifacts that carry their own configuration and lifecycle scripts.

Market Segments: By Product Type

Deployment automation tools are commonly segmented by their delivery model and primary use case:

• Cloud-hosted (SaaS) CI/CD platforms – CircleCI, AppVeyor, Bitbucket Pipelines, GitLab.com, GitHub Actions. No infrastructure to manage; pricing is typically usage- or seat-based.
• Self-hosted CI/CD servers – TeamCity, Bamboo, Jenkins (open-source), CloudBees CD. Deployed on-premises or in a private cloud; suited for organizations with strict data residency requirements.
• Cloud-native deployment services – AWS CodeDeploy, Azure Pipelines, Google Cloud Deploy. Tightly integrated with a specific cloud provider’s compute and storage services.
• Release orchestration platforms – Octopus Deploy, CloudBees CD. Focused specifically on coordinating multi-environment, multi-team releases rather than build execution.
• Infrastructure-as-code deployment tools – Chef, Ansible, Puppet. Treat deployment as a configuration management problem and can manage both infrastructure state and application deployment.

Market Segments: By Application and End Industry

Adoption patterns vary significantly by industry:

• Software and technology companies – the earliest and deepest adopters, driven by the need to ship product updates continuously.
• Financial services – growing adoption, tempered by regulatory requirements around change management and audit trails; tools with strong approval-gate and compliance logging capabilities are preferred.
• Retail and e-commerce – deployment automation is critical for seasonal scaling and frequent feature releases tied to marketing campaigns.
• Healthcare – adoption is increasing but constrained by HIPAA compliance requirements and validation obligations for clinical software.
• Manufacturing and industrial – emerging use case as OT/IT convergence and IoT edge deployments require automated software rollout to distributed device fleets.

Regional Market Dynamics

North America

North America—led by the United States—is home to the majority of the market’s largest vendors and the highest concentration of enterprise DevOps adopters. The density of cloud-native software companies, large SaaS vendors, and mature enterprise IT organizations drives sustained demand. Regulatory environments such as SOC 2, FedRAMP, and HIPAA create additional demand for deployment audit and access-control features.

Europe

Europe is a significant and maturing market. GDPR compliance has increased attention to data residency and access logging, favoring self-hosted or EU-region cloud deployments. Germany, the United Kingdom, and France are the largest European markets for enterprise DevOps tooling.

Asia-Pacific

The Asia-Pacific region represents high growth potential. China has a large and rapidly maturing domestic software industry, while India’s IT services sector and rapidly growing product-company ecosystem are significant demand drivers. Southeast Asian markets—Singapore, Indonesia, Vietnam—are also seeing accelerated DevOps adoption as digital transformation investments increase.

South America, Middle East, and Africa

These regions are earlier-stage markets for formal deployment automation tooling. Cloud adoption is growing in Brazil, the UAE, and South Africa, which tends to pull CI/CD adoption with it as organizations move workloads to AWS, Azure, or GCP and leverage the native deployment services those platforms provide.

Key Drivers of Deployment Automation Adoption

• Increasing release frequency – competitive pressure forces organizations to deploy more often, making manual deployment processes a bottleneck.
• Cloud migration – moving workloads to cloud platforms creates a natural on-ramp to cloud-native CI/CD tooling.
• Microservices and containerization – breaking monolithic applications into services multiplies the number of independently deployable components, making automation essential.
• DevSecOps requirements – embedding security scanning and compliance checks into the pipeline requires the kind of structured, repeatable process that deployment automation provides.
• Remote and distributed engineering teams – automated pipelines provide a consistent, auditable deployment process regardless of where individual team members are located.

Choosing a Deployment Automation Platform

No single tool dominates every use case. A practical selection framework should consider:

1. Hosting preference – SaaS vs. self-hosted vs. cloud-provider-native.
2. Existing toolchain – integration with your version-control system, artifact registry, and target infrastructure.
3. Team size and scale – enterprise platforms such as CloudBees CD and Octopus Deploy support complex multi-team, multi-environment governance; simpler SaaS tools are faster to adopt for smaller teams.
4. Compliance requirements – approval workflows, audit logs, and RBAC controls vary significantly between products.
5. Operating system targets – Windows-heavy environments may favor Azure Pipelines, AppVeyor, or Octopus Deploy over Linux-first tools.
6. Community and ecosystem – a large plugin or integration ecosystem reduces custom scripting effort.

Summary

The deployment automation market encompasses a broad range of tools—from cloud-native SaaS platforms and dedicated release orchestrators to infrastructure-as-code engines and cloud-provider-bundled services. Major vendors including Microsoft, GitLab, Atlassian, Octopus Deploy, CircleCI, AWS, and Chef each address different points on the spectrum from simple web-app deployments to complex enterprise release management. Organizations evaluating this space should prioritize fit with their existing toolchain, compliance obligations, and deployment target environment over brand recognition or headline market-share figures.

The Cloud Security Problem

Protecting critical data and applications is difficult under any circumstances, but the challenge intensifies when workloads run in the cloud. Most organizations today have moved a significant portion of their infrastructure to cloud providers—and with that shift comes a fundamental security tension: your team is responsible for securing workloads and applications, but you do not have full control over the underlying environment.

This tension is formalized in the Shared Responsibility Model, which broadly assigns responsibility for security of the cloud (physical infrastructure, hypervisors, networking hardware) to the cloud provider, while placing responsibility for security in the cloud (operating systems, applications, data, access controls) squarely on the customer. Confusion about this boundary is routinely exploited by attackers.

Why Traditional Security Tools Fall Short in the Cloud

Security tooling designed for static, on-premises data centers was built with a fixed network perimeter in mind. It relies on stable IP addresses, subnets, and Layer 2 network controls to define trust boundaries and enforce policy. Cloud environments break almost every one of these assumptions:

• Dynamic IP addressing: Cloud workloads frequently change IP addresses as instances are created, scaled, or replaced.
• No Layer 2 access in IaaS: Infrastructure-as-a-Service platforms abstract away the data-link layer, making port-level and MAC-address-based controls unavailable to tenants.
• Elastic subnets: Cloud subnets are intentionally large to accommodate auto-scaling, which means workloads are grouped more loosely than in on-premises environments and are more exposed to lateral movement within the same subnet.
• Ephemeral workloads: Containers and serverless functions may live for seconds or minutes, making agent-based or IP-tracked policies impractical.

Despite these realities, many organizations continue to rely on hardening the network perimeter, adding more threat-detection tooling, and writing subnet-based firewall rules. These controls have value, but they do not prevent an attacker who has already gained a foothold from moving laterally—and in cloud environments, lateral movement is easier precisely because subnets are broader.

Zero Trust as the Correct Security Model for the Cloud

Zero trust is a security philosophy that treats all internal network communication as untrusted by default, regardless of origin. Instead of assuming that traffic inside a network boundary is safe, zero trust requires every service-to-service or application-to-application communication to be explicitly authorized before it is permitted.

Key principles of zero trust relevant to cloud workloads:

• Least privilege: Each workload or service is granted only the specific communication rights it needs—nothing more.
• Continuous verification: Authorization is checked on every communication attempt, not just at initial connection.
• Micro-segmentation: Security perimeters are drawn tightly around individual workloads or application components, not around broad network segments.

Zero trust is not a product—it is an architecture. The challenge is implementing it in a way that is portable across environments.

The Core Problem: Network-Coupled Security Policies

When security policies are defined using network constructs—IP addresses, subnets, port ranges, VLANs—they become tightly coupled to the specific environment where they were written. Moving a workload to a different cloud region, a different provider, or even a different VPC can invalidate every policy rule that references a specific address or subnet.

The operational cost of maintaining network-based policies at cloud scale is significant:

• Security engineers spend time mapping changing network addresses to application context.
• Migrations require policy rewrites or exception lists that introduce risk.
• Granularity is lost because large cloud subnets force coarse-grained rules.
• Developers must translate application intent into network constructs they may not fully understand.

The Solution: Decouple Workload Protection from the Network

The architectural answer is to decouple security policy from the underlying network infrastructure entirely. Instead of writing rules based on where a workload lives (its IP address or subnet), you write rules based on what the workload is—its identity.

Cryptographic Workload Identity

By deriving identity from the immutable properties of a workload—such as its binary hash, signing certificate, runtime attributes, or service account—security teams can issue cryptographic identities to applications and services. These identities travel with the workload regardless of where it is deployed.

With cryptographic identities in place:

• Security policies reference identities, not network addresses.
• When a workload migrates, scales, or restarts, its identity remains stable and its policies remain valid.
• Every communication attempt is authenticated against identity before it is permitted.
• Policies can be versioned and updated to tolerate software upgrades without requiring network reconfigurations.

Microperimeters That Follow Workloads

Identity-based policies allow security teams to create microperimeters—tight security boundaries drawn around individual applications or services. Because these perimeters are defined by identity rather than network location, they follow the workload wherever it is deployed: on-premises, in a public cloud, in a container cluster, or at the edge.

Every time an application attempts to communicate across a microperimeter boundary, the identity of the initiating workload is verified against policy. If it is not explicitly authorized, the communication is blocked—regardless of what subnet or network segment the workload is currently running in.

Benefits Beyond Security: Developer Velocity

Network-agnostic, identity-based security policies remove a significant source of friction in the software development lifecycle. When security policy is expressed in terms of application identity rather than network constructs:

• Developers can define security policy alongside application code, in the same language and toolchain they already use.
• Applications can be built and tested in any environment—local laptops, CI pipelines, staging clouds—without requiring network reconfiguration at each stage.
• Deploying to production (even if hosted by a third party) does not require security teams to rewrite or translate policies.
• The gap between development and operations security requirements narrows, which is the core promise of DevSecOps.

Practical Implications for DevSecOps Teams

Adopting portable, identity-based security policies requires changes to tooling, process, and culture:

• Tooling: Evaluate service mesh solutions, workload identity platforms, and policy engines that support cryptographic identity and are network-agnostic. Examples in the ecosystem include mutual TLS (mTLS) for service-to-service authentication, SPIFFE/SPIRE for workload identity, and open policy frameworks—though specific tool choices should be evaluated against your environment.
• Policy as code: Store security policies in version control alongside application code. Treat policy changes with the same review and testing discipline as application changes.
• Shift-left security: Give developers tooling to define and test security policy early in the development cycle, not as a gate at deployment time.
• Audit and observability: Identity-based systems generate rich, application-contextual audit logs. Invest in tooling to consume and alert on this data.

Summary

Cloud environments break the assumptions that traditional, network-coupled security tools depend on. Organizations that continue to rely on perimeter hardening and subnet-based controls are leaving themselves exposed to lateral movement and policy drift during migrations.

The path forward is to decouple workload protection from the underlying network by building security policy around cryptographic workload identities rather than IP addresses and subnets. Identity-based policies are portable—they follow workloads across environments, survive infrastructure changes, and enable genuine zero trust without requiring constant policy rewrites. They also reduce friction for developers, enabling faster and safer delivery pipelines that are the hallmark of mature DevSecOps practice.

No organization should have to trade security for the operational efficiency that cloud and container platforms offer. Identity-based, network-agnostic security policies make that trade-off unnecessary.

Overview

Amazon Web Services (AWS) has signed a Whole-of-Government procurement agreement with the Commonwealth of Australia, opening the full AWS cloud services portfolio to Federal, State, and Territory government agencies under a single, pre-negotiated framework. The deal also extends to public universities and government-controlled organisations.

What the Agreement Covers

The Whole-of-Government agreement is designed to remove the administrative friction historically associated with government technology procurement. Key elements include:

• Simplified procurement: Agencies can access any AWS service in any AWS Region at pre-agreed terms, eliminating the need to negotiate individual contracts for each engagement.
• Self-service access: A low-administration, self-service model replaces the traditional fixed price list that could take weeks to update, enabling faster experimentation and iteration.
• Enterprise Support: AWS Enterprise Support is included to provide guidance and best practices to agencies at all levels.
• Professional Services: AWS Professional Services is available for agencies undertaking complex cloud migration or modernisation projects.
• Training: AWS Training is bundled into the agreement to help agency staff build and maintain current cloud skills.
• Global reach: Agencies gain consistent access to AWS services across all of AWS’s Availability Zones and Regions worldwide without separate contract negotiations per Region.

AWS Partner Network (APN) Involvement

The agreement streamlines engagement with APN members as well. Consulting and technology partners within the AWS Partner Network can more readily offer their services to government agencies, helping to build government-specific solutions and supporting workload migrations to AWS. This is expected to accelerate the broader Australian public sector ecosystem of cloud-native delivery.

Security and Compliance: PROTECTED Certification

A significant enabler for the agreement is the PROTECTED certification granted to the AWS Asia Pacific (Sydney) Region by the Australian Cyber Security Centre (ACSC) in January 2019. This certification allows government agencies to migrate highly sensitive workloads and applications to AWS infrastructure in Australia, meeting the security requirements mandated for protected government data.

Because all participating agencies operate under the same agreement terms, even smaller agencies benefit from the same security capabilities and fault-tolerant infrastructure that were previously more accessible to larger departments with greater negotiating resources.

Building on Existing Public Sector Relationships

The Whole-of-Government deal builds on existing AWS agreements already in place with several Australian public sector organisations, including:

• Australian Taxation Office (ATO)
• Australian Bureau of Statistics (ABS)
• La Trobe University

The new framework creates a consistent baseline that these and other organisations can leverage without renegotiating terms from scratch.

Industry Reaction

Peter Moore, Managing Director of AWS Worldwide Public Sector Asia Pacific, commented on the agreement:

“This new Whole-of-Government procurement agreement with the Commonwealth of Australia greatly simplifies and streamlines government technology procurement by providing government agencies with rapid access to the full portfolio of AWS services in any AWS Region at pre-agreed upon terms. This paves the way for greater innovation by enabling agencies to build and deploy digital services to serve Australians more effectively, transform legacy operations, and accelerate scientific breakthroughs.”

— Peter Moore, Managing Director, AWS Worldwide Public Sector Asia Pacific

Key Takeaways for Public Sector Cloud Adoption

• Whole-of-Government agreements can significantly reduce procurement lead times and administrative overhead across public sector entities.
• Bundling enterprise support, professional services, and training into a single framework lowers the barrier for smaller agencies to adopt cloud at scale.
• Security certifications (such as ACSC PROTECTED) are a prerequisite for handling sensitive government workloads and should be confirmed before migration planning begins.
• Enabling APN partner access within the same framework multiplies delivery capacity without requiring additional procurement cycles.

The Data Engineering Bottleneck Slowing AI Adoption

Data integration and big data management solutions provider Attunity announced an analytics platform designed to automate streaming data pipelines for AI, machine learning, and data science initiatives. The core problem it addresses is a familiar one: data engineering teams cannot keep pace with the demand for real-time datasets that machine learning applications require.

Data integration and migration are often manual, complex, and time-consuming processes. By the time a dataset is assembled and delivered to data scientists, it is frequently outdated — undermining the value of the analysis before it even begins. Attunity’s approach targets this lag directly by combining efficient change data capture (CDC) at scale with automated data transformations running on Apache Spark.

Attunity for Databricks Unified Analytics Platform

The integration — Attunity for Databricks Unified Analytics Platform — provides continuous change data capture, delivery, and automated data refinement to produce analytics-ready datasets inside Databricks. Databricks’ Unified Analytics Platform unifies data processing and machine learning in a single environment, with Apache Spark as the high-performance processing engine underneath.

By pairing Attunity’s CDC pipeline automation with Databricks, organizations gain a path from raw transactional source data to clean, ML-ready datasets without writing custom integration code. This matters in practice because manual pipeline development is a significant time sink for data engineers — time that could otherwise be spent on higher-value work.

“Our ability to reach into virtually all enterprise systems and deliver real-time, analytics-ready and transactionally consistent data without coding is a powerful enabler for data scientists, allowing them to generate the business insights that accelerate corporate decision making and growth.”

Itamar Ankorion, Chief Marketing Officer, Attunity

Key Capabilities

Accelerated Time to Value

Using Apache Spark as the processing engine, Attunity automates data delivery and creates analytics-ready datasets in Databricks. Data engineers can build reusable, automated pipelines that reduce reliance on manual coding and expensive custom development resources, shortening the delivery cycle from source systems to data scientists.

Reduced Data Preparation Overhead

Attunity’s pipeline automation reduces the data preparation burden on data scientists. Rather than spending significant time wrangling raw data, data scientists can focus on analysis and model development. More agile pipeline generation means that changes to source data schemas or business requirements can be accommodated without rebuilding pipelines from scratch.

Real-Time Data with CDC at Scale

Attunity’s change data capture (CDC) technology provides the low-latency, high-throughput characteristics needed for large-scale machine learning datasets in the cloud. CDC captures only the changes made to source data — inserts, updates, and deletes — and streams those changes continuously, rather than running expensive bulk extractions on a schedule.

Multi-Cloud Support

Attunity supports Databricks deployments on both Microsoft Azure and Amazon Web Services (AWS), as well as a broad range of data lake, data warehouse, and streaming services available on those platforms. This gives organizations flexibility to adopt the platform within their existing cloud strategy without vendor lock-in to a single cloud provider.

Transactionally Consistent, ACID-Compliant Data

Attunity provides full support for the Databricks Unified Analytics Platform’s ACID capabilities. Transactions are delivered and applied in the order they were committed on the source system, ensuring that data scientists and ML models work from a transactionally consistent view of the data — a critical requirement for models where data ordering and integrity affect accuracy.

Why This Matters for DataOps

The Attunity-Databricks integration reflects a broader shift toward DataOps practices — applying DevOps-style automation, collaboration, and continuous delivery principles to data pipelines. Automating the movement and transformation of data reduces the manual coordination between data engineering and data science teams, shortens feedback loops, and makes it easier to keep ML models trained on current data.

For enterprises with data spread across siloed storage systems — on-premises databases, cloud data warehouses, streaming platforms — the ability to unify those sources into a single analytics environment without hand-coded ETL is a meaningful operational improvement. The combination of CDC for real-time ingestion and Spark for transformation automation addresses the two most time-consuming stages of the modern data pipeline.

Note: Attunity was acquired by Qlik in May 2019, shortly after this announcement. Organizations evaluating this technology should review current product branding and availability under Qlik.

The Most Common Misconception About DevOps

When organisations first explore moving to a DevOps model, a near-universal fear surfaces: “If a developer checks in a single line of code, is it going straight to production?” That fear is understandable, but it rests on a false premise. DevOps does not mandate that every commit triggers a live deployment.

The terms continuous integration (CI) and continuous deployment (CD) are often treated as binary switches — either you have them or you don’t — but in practice they are dials. Your team decides how far to turn each one based on risk appetite, tooling maturity, and the pace of change in your applications.

Defining “Continuous”: It Is Not the Same for Every Team

“Continuous deployment” means moving completed developer changes into a production environment where end users can work with them. The word deploy has a fixed meaning; the word continuous does not. Your organisation owns the definition of continuous that fits your context.

Automated deployment pipelines consume real resources — build agents, test infrastructure, environment capacity, and engineering attention. As a purely practical matter, teams have to decide how often it makes sense to marshal those resources. Common deployment cadences include:

• On every commit — maximum feedback speed, lowest increment size per release
• Nightly — balances feedback with infrastructure cost
• Weekly — common in organisations still building automation confidence
• Monthly — acceptable for applications with a very low rate of change

The Increment of Functionality Problem

The longer the interval between deployments, the more changes accumulate in any single release. This accumulated set of changes — sometimes called the increment of functionality — is directly correlated with deployment risk. A release containing three small changes is far easier to validate, roll back, and diagnose than one containing three months of accumulated work. This is one of the strongest structural arguments for shorter deployment cycles, even when that feels counterintuitive.

“Potentially Deployable” Does Not Mean “Must Deploy”

A nuance often lost in CI/CD discussions: the goal of a continuous deployment pipeline is to produce a potentially deployable artifact — not necessarily to deploy it immediately. Your pipeline can build, test, and validate a package against a production-like environment on every cycle without ever pushing it live until a human or a release gate approves it.

This approach — sometimes distinguished as continuous delivery versus continuous deployment — gives organisations the safety net of frequent validation without the perceived disruption of frequent live releases. The pipeline proves the code could go to production; a separate decision determines when it does.

The Hidden Cost of Infrequent Deployments

Choosing a longer deployment interval to reduce disruption trades one risk for another. The most significant hidden costs are:

• Delayed user feedback. Real production feedback only arrives after an actual deployment. The longer the gap, the longer your team is building without knowing whether the previous release met user needs.
• Expensive rework. When a developer receives feedback on a feature shipped weeks or months ago, context has been lost. Fixing it takes longer and risks destabilising newer work built on top of the original change.
• Compounding integration risk. As features pile up between releases, isolating the cause of a production issue becomes progressively harder.

A common pattern in organisations adopting DevOps is to start with monthly or weekly deployments to build confidence in their automation, and then gradually shift toward shorter cycles as the team realises the risks of large increments outweigh the perceived safety of infrequent releases.

How Integrated Should Your Integration Be?

Continuous integration means merging developers’ changes into a shared mainline frequently, rather than leaving branches in isolation for days or weeks. The fundamental goal is that all developers are working from the same codebase at all times, preventing the “integration hell” that occurs when long-lived branches must eventually converge.

Historically, the minimum bar for CI was simply that the application could still compile after a merge (the “don’t break the build” rule popularised by the Windows NT team, among others). Modern CI pipelines go much further, typically including:

• Automated unit tests run by the developer before or immediately after commit
• Static code analysis and linting
• Integration tests that exercise the merged application as a whole
• Contract tests between services in a microservices architecture
• Performance and stress tests (often deferred to a separate stage due to resource cost)

Developers can — and should — write automated unit tests that verify their own code in isolation. But bugs that only surface when components interact can only be caught during integration testing, after code is merged. This distinction matters when deciding where to place which tests in your pipeline.

Calibrating Your Integration Testing Strategy

The ideal is to run all integration tests against a full duplicate of the production environment on every commit. In practice, cost and environment complexity force most teams to make trade-offs. Two variables determine where tests end up in your pipeline:

• How easy it is to keep the integration environment in sync with production. Environment drift is a hidden tax on test reliability. If your integration environment frequently diverges from production, test results become less meaningful.
• How resource-intensive the tests are. Functional correctness tests are usually fast and cheap. Stress tests, user-acceptance tests, and full end-to-end tests can be slow and expensive to run on every commit.

A practical approach many teams use is a tiered test strategy:

1. On every commit: unit tests and fast integration smoke tests
2. On every build (nightly or per-branch merge): full functional integration tests
3. On release candidates: performance, stress, and user-acceptance tests

The important principle is that deferring a test increases the cost of fixing any defect it catches, because the developer must re-establish context from work done further in the past. As your pipeline becomes more efficient, the economics shift: it becomes cheaper to run more tests earlier, and teams naturally pull tests further left in the process.

The Natural Evolution of a CI/CD Programme

Organisations rarely reach an optimal CI/CD cadence on day one. A realistic progression typically looks like this:

1. Establish the pipeline. Get automated builds and basic tests running. Deploy infrequently (weekly or monthly) to build confidence in automation.
2. Increase deployment frequency. As confidence grows, the team notices that large increments of functionality are themselves a source of risk, and begins deploying more often.
3. Move tests left. As the pipeline matures, expensive tests that were deferred to the deployment stage are moved into the integration stage, or even into per-commit validation.
4. Reach a sustainable balance. The team finds a cadence that minimises overall risk — covering both deployment disruption and the risk of delayed feedback — and continues to adjust as the application and organisation evolve.

Key Takeaways

• CI and CD are spectrums, not binary states. Your team decides where on each spectrum to operate.
• Longer deployment intervals reduce disruption frequency but increase per-release risk and delay user feedback.
• “Potentially deployable” is a valid outcome of a CD pipeline — actual deployment can be gated by humans or automated release policies.
• The right integration testing strategy is determined by environment parity costs and test resource costs, not by ideology.
• As pipeline maturity increases, the economics of shifting tests earlier improve. Treat your CI/CD posture as something that evolves continuously alongside your team’s capabilities.

DevOps does not impose a one-size-fits-all deployment rhythm on your organisation. It gives you the tooling and practices to make deployment safe enough to do more frequently — and then leaves the decision about how frequently firmly in your hands.

Why Servers Are Sometimes Offline

Network-restricted servers exist for legitimate reasons:

• Security policy: Preventing outbound connections reduces the attack surface.
• Compliance: Regulations in finance, healthcare, and government often require isolated environments.
• Stability: Ensuring that a production environment never pulls unexpected package versions from the internet.

Whatever the reason, you need to bring all dependencies with your deployment artefact rather than fetching them at install time.

Core Strategy: Bundle Your Dependencies

The fundamental approach is to download all Python packages on a machine that does have internet access, bundle them alongside your application code, transfer the bundle to the offline server, and install from the local copy. The steps below walk through this process.

Step 1: Pin All Dependencies

Before anything else, generate a fully pinned requirements.txt from your working development or CI environment. Use pip freeze to capture exact versions:

pip freeze > requirements.txt

Pinned versions are critical for offline deployments because you cannot let pip resolve the latest compatible version at install time — there is no network to query.

Step 2: Download Packages as Wheel Files

On an internet-connected machine that matches the target server’s Python version and OS architecture, download all packages as pre-built wheel (.whl) files into a local directory:

pip download -r requirements.txt -d ./wheelhouse

The -d flag tells pip to store the downloaded files in the ./wheelhouse directory rather than installing them. Wheel files are platform-specific, so the download machine should ideally run the same OS and CPU architecture as the target server (e.g., Linux x86-64). If there is a mismatch, you can add the --platform, --python-version, and --abi flags to pip download to target the correct platform explicitly.

Step 3: Bundle and Transfer

Archive your application code and the wheelhouse directory together:

tar -czf myapp_deploy.tar.gz myapp/ wheelhouse/ requirements.txt

Transfer the archive to the offline server using whatever mechanism is available — SCP over a bastion host, USB drive, internal file share, or a deployment tool such as Ansible, Fabric, or a CI/CD pipeline that has internal network access.

Step 4: Install from the Local Wheelhouse

On the offline server, extract the archive and install packages using --no-index and --find-links to prevent pip from reaching out to PyPI:

pip install \
  --no-index \
  --find-links=./wheelhouse \
  -r requirements.txt

• --no-index disables all index lookups, including PyPI.
• --find-links tells pip to look in the specified local directory for package files.

If every required package is present in the wheelhouse, the installation will succeed without any network calls.

Setting Up a Private PyPI Mirror (for Scale)

If you manage many Flask microservices — or update them frequently — maintaining per-application wheelhouse bundles quickly becomes tedious. A better long-term solution is to host an internal PyPI mirror on a server that is reachable from your offline servers over the internal network.

Tools commonly used for this include:

• devpi — a full-featured PyPI server and proxy with caching.
• pypiserver — a lightweight, easy-to-deploy PyPI-compatible server.
• Nexus Repository Manager or Artifactory — enterprise artefact repositories that support PyPI among other formats.

Once your internal mirror is populated, configure pip on all offline servers to use it by default, either via the PIP_INDEX_URL environment variable or a pip.conf file:

# /etc/pip.conf or ~/.pip/pip.conf
[global]
index-url = http://internal-pypi.company.local/simple/
trusted-host = internal-pypi.company.local

Using Virtual Environments

Regardless of whether you use a wheelhouse or an internal mirror, always deploy Flask applications inside a Python virtual environment. This isolates the application’s dependencies from the system Python and from other applications running on the same host:

python3 -m venv /opt/myapp/venv
source /opt/myapp/venv/bin/activate
pip install --no-index --find-links=./wheelhouse -r requirements.txt

Containerised Deployments on Offline Servers

If your offline server runs Docker or another container runtime, you can build the Docker image on an internet-connected machine and transfer the image as a tar archive:

# On the build machine
docker build -t myapp:1.0 .
docker save myapp:1.0 | gzip > myapp_1.0.tar.gz

# Transfer myapp_1.0.tar.gz to the offline server, then:
docker load < myapp_1.0.tar.gz
docker run -d -p 5000:5000 myapp:1.0

This approach bundles the entire runtime environment — OS libraries, Python interpreter, and application code — into a single portable artefact, eliminating dependency concerns on the target host entirely.

Serving Flask in Production

Flask’s built-in development server is not suitable for production use. On an offline server you still need a production-grade WSGI server. Common choices include:

• Gunicorn — widely used, easy to configure, include it in requirements.txt so it is bundled in your wheelhouse.
• uWSGI — performant and feature-rich, though more complex to configure.
• Waitress — pure-Python, a good choice when compiling C extensions is not possible on the target platform.

Pair the WSGI server with a reverse proxy such as Nginx or Apache (installed via the system package manager from an internal mirror or pre-installed on the server image) to handle TLS termination, static files, and load balancing.

Checklist: Offline Flask Deployment

• ☑ Generate a fully pinned requirements.txt with pip freeze.
• ☑ Run pip download on a machine matching the target OS and Python version.
• ☑ Include the WSGI server (e.g., Gunicorn) in requirements.txt.
• ☑ Bundle application code and the wheelhouse together.
• ☑ Transfer the bundle via an approved secure channel.
• ☑ Create a virtual environment on the target server.
• ☑ Install with pip install --no-index --find-links=./wheelhouse -r requirements.txt.
• ☑ Start the application with your WSGI server and manage it with systemd or supervisor.
• ☑ Configure a reverse proxy for production traffic.

Following this process ensures repeatable, network-independent deployments for Flask microservices, regardless of how locked down the target environment is.

This article is an independent explanation of offline Flask deployment techniques. It was prompted by a post originally published on the Theodo blog (November 2016).