Containerization – Best DevOps https://www.bestdevops.com Lets Learn, Do it & Share! Thats a Best DevOps!!! Tue, 13 Oct 2026 23:56:07 +0000 en-US hourly 1 https://wordpress.org/?v=7.1.3 What’s Inside Your Containers? Using Open Source Tools to Gain Visibility and Security https://www.bestdevops.com/whats-in-your-containers-try-an-open-source-tool-to-find-out/ https://www.bestdevops.com/whats-in-your-containers-try-an-open-source-tool-to-find-out/#respond Thu, 16 Jan 2020 05:59:11 +0000 http://www.bestdevops.com/?p=10054 Container adoption has a well-known pattern: developers start using Docker or similar tools on their workstations, containers spread into CI pipelines, and before long the entire organization is running workloads in Kubernetes—sometimes before the security team has had a meaningful look. If your shop is already waist-deep in containers, you are not alone.

This reality places security practitioners in a difficult position. You must simultaneously secure the container runtime and orchestration layer, validate the applications and libraries packed inside each image, maintain rigor in every other area of your program, and rapidly build expertise across container engines, orchestration platforms, microservice patterns, and cloud-native tooling. Any force multiplier helps.

The Core Problem: Opacity Inside Container Images

A container image is not simply a binary. It is a layered filesystem that can contain an operating system base, language runtimes, third-party libraries, application code, configuration files, and—if developers are not careful—credentials and secrets. Without deliberate inspection, it is easy to deploy images whose full software bill of materials (SBOM) is unknown, including packages that carry known vulnerabilities.

Traditional vulnerability scanning typically requires a live, network-accessible host. Container images sitting in a registry or on a developer’s workstation are not live hosts. You need a different approach: static image analysis that unpacks each layer of the image and inventories its contents without executing the container.

Anchore Engine: Deep Inspection for Container Images

One open source tool that addresses this problem is Anchore Engine, released under the Apache License 2.0. Anchore Engine unpacks a container image layer by layer and produces a detailed inventory of everything inside, including:

  • Operating system packages (RPMs, DEBs, APKs, and others)
  • Language ecosystem packages (Python, Ruby gems, Node modules, Java JARs, and similar)
  • Runtime versions (JRE, Node.js, Python interpreter, etc.)
  • Configuration files that affect container behavior
  • Secrets and credentials inadvertently baked into the image

According to Anchore’s own documentation and public statements, the engine then maps the discovered packages against known vulnerability databases, surfacing CVE identifiers for any vulnerable software found. This gives you something functionally similar to a vulnerability scan report, but produced statically against the image—no running container required.

Why Visibility Into Image Contents Matters

Knowing exactly what is inside a container image is valuable in several contexts beyond simple vulnerability detection:

  • Software architecture reviews: Image analysis exposes accidental or undocumented dependencies that would otherwise be invisible in design reviews.
  • Threat modeling: A complete package inventory makes attack surface analysis much more accurate.
  • Secrets management: Automated inspection can flag credentials, tokens, or private keys that developers accidentally included in an image layer.
  • Compliance and audit: Regulators and auditors increasingly ask organizations to demonstrate they know what software is running in production. A generated SBOM satisfies that requirement.
  • Patch management: When a new CVE is published, you can immediately query which images contain the affected package rather than waiting for a traditional scan cycle.

Integrating Container Image Scanning Into Your Workflow

There is no single right place to run image analysis. The best answer depends on your organization’s maturity, risk tolerance, and pipeline structure. Consider three common integration points:

1. Developer Workstations and Inner Loop

Enabling developers to run image analysis locally gives them immediate feedback. They can identify and remove stale libraries, unnecessary OS packages, or vulnerable base image versions before a single line is pushed to a shared repository. This is the leftmost shift possible and generally the cheapest place to fix a problem.

2. CI/CD Pipeline Gates

Integrating image scanning as a pipeline stage—immediately after the image build step and before the image is pushed to a registry—creates an automated quality gate. A pipeline script can invoke the scanner, evaluate the resulting policy report, and fail the build if images do not meet defined standards (for example, no critical CVEs, no embedded secrets). This is the most impactful integration point for most teams because it prevents vulnerable images from reaching the registry at all.

Anchore Engine exposes a REST API and CLI that are straightforward to call from Jenkins, GitLab CI, GitHub Actions, CircleCI, and similar systems. Pre-built integrations and plugins exist for several of these platforms.

3. Registry and Production Monitoring

Images already in production or sitting in a registry can be analyzed retrospectively. This is especially useful when a new CVE is published and you need to determine your exposure across a large fleet of deployed images. Point Anchore Engine at your registry, analyze the image manifests, and query the results against the new CVE.

Getting Started With Anchore Engine

Anchore Engine is designed to run as a set of containerized services orchestrated with Docker Compose. If you already have Docker and Docker Compose installed, the initial setup requires only a handful of shell commands to pull the Docker Compose configuration file and start the services. No upfront licensing cost is involved for the open source edition.

At a high level, the workflow looks like this:

  1. Start the Anchore Engine services using Docker Compose.
  2. Add the image you want to analyze (by name or digest) to the engine’s queue.
  3. Wait for the analysis to complete (the engine pulls the image, unpacks layers, and builds the inventory).
  4. Query the results: package list, CVE matches, policy evaluation outcome.

The engine ships with a default policy that flags the most commonly encountered vulnerability classes. You can customize policies to match your organization’s risk thresholds—for example, blocking on critical or high-severity CVEs while warning on medium-severity findings.

Open Source vs. Commercial Alternatives

Anchore Engine is not the only tool in this space. Other open source options—such as Trivy (from Aqua Security) and Clair (from Quay/Red Hat)—address overlapping use cases. Commercial offerings from vendors including Aqua Security, Prisma Cloud (Palo Alto Networks), Sysdig, and Snyk provide broader platform coverage, managed infrastructure, and enterprise support.

The value of starting with an open source tool is not that it is necessarily better than a commercial product—it is that you can start today. You can validate that image scanning provides real value in your environment, experiment with where it fits best in your pipeline, and build the internal case for a commercial investment if you need one. An open source tool is an ideal on-ramp and a practical stopgap, not necessarily the final destination.

Key Takeaways

  • Container images often contain far more software than their owners realize, including vulnerable packages and accidentally embedded secrets.
  • Static image analysis does not require a running container, making it applicable to images at any stage of the software supply chain.
  • Anchore Engine is a free, Apache-licensed tool that unpacks container images layer by layer and maps their contents to CVE databases.
  • The most impactful integration point for most teams is the CI/CD pipeline, where scanning can act as an automated gate before images reach a registry.
  • Starting with an open source scanner costs nothing but time and provides immediate visibility, even if a commercial solution is the eventual goal.

Original article sourced from TechNewsWorld (technewsworld.com), January 2020.

]]> https://www.bestdevops.com/whats-in-your-containers-try-an-open-source-tool-to-find-out/feed/ 0 How to Successfully Use Containers: Benefits, Risks, and Best Practices https://www.bestdevops.com/how-to-successfully-use-containers/ https://www.bestdevops.com/how-to-successfully-use-containers/#respond Tue, 11 Jun 2019 06:30:08 +0000 http://www.bestdevops.com/?p=5272 In traditional software development, code that works perfectly in a developer’s local environment can behave unexpectedly once deployed to staging or production. Environment differences — OS versions, library versions, configuration drift — cause bugs that are difficult to reproduce and expensive to fix. Containers address this problem directly by packaging an application together with everything it needs to run.

Adoption of containerization technologies like Docker and Kubernetes has grown rapidly across the industry, and understanding both the benefits and the risks is essential for any DevOps team looking to move workloads into containers successfully.

Why Use Containers?

Containers change the fundamental model of software delivery. A container image bundles application code, runtime, libraries, and configuration into a single, portable artifact. Developers build the image once, test it, validate it, and then deploy it — knowing that what runs in production is byte-for-byte identical to what was tested.

This model provides several concrete advantages over traditional virtual machine (VM)-based deployments:

  • Lower resource overhead: Containers share the host operating system kernel rather than running a full guest OS, so they consume significantly less CPU, memory, and storage than VMs. This allows more workloads to run on the same underlying hardware.
  • Portability: A container image runs the same way on a developer’s laptop, a CI server, and a cloud provider’s managed Kubernetes cluster — eliminating the “works on my machine” problem.
  • Consistent delivery pipeline: The same container image can be used at every stage of the pipeline — build, test, staging, and production — removing environment-specific surprises.
  • Faster deployments and scaling: Containers start in seconds, not minutes, and container orchestrators like Kubernetes can spin up or tear down replicas automatically in response to load.
  • Support for modernization: Teams can incrementally containerize legacy monolithic applications while simultaneously building new cloud-native microservices alongside them.

These qualities make containers especially valuable in hybrid and multi-cloud environments, where workloads need to move across different infrastructure providers without requiring application changes.

Containers Add Value — But Also Introduce Risk

It is important to be clear about what containers do not solve. They eliminate environment-drift problems, but they do not make software immune to bugs, outages, or security vulnerabilities.

Security Vulnerabilities in Base Images

Container images are built on top of base images — typically minimal Linux distributions — and those base images frequently contain unpatched vulnerabilities. Security research from Snyk has found that widely used public Docker images commonly carry dozens of known vulnerabilities, particularly when they have not been recently updated. Any container built on an outdated base image inherits all of those vulnerabilities.

Practical mitigations include:

  • Regularly scanning images with tools such as Trivy, Snyk Container, or AWS ECR image scanning before pushing to a registry.
  • Rebuilding and republishing images whenever base image updates are available.
  • Using minimal base images (e.g., Alpine or distroless images) to reduce the attack surface.
  • Enforcing image provenance policies so that only images from trusted, scanned registries can be deployed.

The Uniformity Risk

Containers solve the mixed-environment problem by ensuring identical software runs everywhere. This is both their greatest strength and a significant risk amplifier. If a vulnerability or misconfiguration is baked into an image, every replica instantiated from that image is equally vulnerable. An exploit that works against one container works against all of them simultaneously.

This makes rigorous image validation before deployment — not after — essential. Vulnerability scanning, image signing, and admission controllers in Kubernetes (such as OPA/Gatekeeper or Kyverno) can help enforce policy gates that prevent vulnerable images from reaching production.

The Cost of Downtime

Application outages carry real economic impact regardless of whether the underlying infrastructure uses containers or VMs. Organizations should not assume that containerization alone reduces their operational risk; it shifts the risk profile rather than eliminating it. Proper testing, observability, and incident response processes remain essential.

A Better Approach to Testing Containerized Applications

Moving to containers requires a corresponding evolution in how QA and testing are done. Simply lifting an existing testing approach and applying it to containerized apps is insufficient.

  • Test under production-like conditions: Containers can behave differently depending on CPU limits, memory constraints, network policies, and storage configurations. QA environments should mirror production resource settings as closely as possible.
  • Integrate security scanning into CI/CD: Vulnerability scanning should happen automatically at image build time, not as a one-off manual check. Block pipeline progression if critical vulnerabilities are detected.
  • Test network behavior: Microservices running in containers communicate over the network. Test for latency, timeouts, partial failures, and retry logic — not just happy-path flows.
  • Use chaos engineering: Tools like Chaos Monkey or Litmus can deliberately introduce failures (container crashes, network partitions) to verify that the system degrades gracefully and recovers correctly.
  • Validate container orchestration configuration: Kubernetes manifests, Helm charts, resource quotas, and network policies are all part of the deployable artifact. Lint and test them as part of the pipeline.

Key Practices for Successfully Running Containers in Production

Beyond testing, several operational practices distinguish teams that run containers reliably in production from those that struggle:

  • Run containers as non-root: Avoid running processes inside containers as the root user. Use a dedicated, low-privilege user in your Dockerfile.
  • Set resource limits: Always define CPU and memory requests and limits in Kubernetes pod specs to prevent a single noisy container from starving others on the same node.
  • Use health checks: Define liveness and readiness probes so that the orchestrator knows when to restart a failed container or stop routing traffic to one that is not yet ready.
  • Store configuration externally: Use environment variables, ConfigMaps, or secrets management tools (such as HashiCorp Vault or Kubernetes Secrets) rather than baking configuration into images.
  • Implement centralized logging and monitoring: Container logs are ephemeral. Ship them to a centralized system (e.g., ELK stack, Datadog, or CloudWatch) and instrument applications with metrics and distributed tracing.
  • Maintain an image registry hygiene policy: Delete old, unused image tags regularly. Old images are security liabilities and inflate storage costs.

Containers Are Powerful — But Not a Silver Bullet

Containers genuinely improve developer productivity, deployment consistency, and infrastructure efficiency. They are one of the most impactful operational shifts a software team can make. However, they introduce their own failure modes and security considerations that teams must plan for explicitly.

The teams that get the most value from containers are those that treat image security, testing fidelity, and orchestration configuration with the same rigor they apply to application code. Containers used thoughtfully — with proper scanning, testing, resource management, and observability — enable faster, safer software delivery. Containers used carelessly can propagate vulnerabilities and misconfigurations at scale.

Invest the time to understand the risks, build the right tooling into your pipeline, and containers will become one of the most reliable parts of your infrastructure.

]]> https://www.bestdevops.com/how-to-successfully-use-containers/feed/ 0 Maritime Containerization Market Overview: Key Players, Segments, and Regional Outlook https://www.bestdevops.com/maritime-containerization-market-outlook-highlights-major-opportunities-likely-to-steer-demand-during-forecast-period-china-ocean-shipping-group-company-cosco-china-cma-cgm-sa-france-agili/ https://www.bestdevops.com/maritime-containerization-market-outlook-highlights-major-opportunities-likely-to-steer-demand-during-forecast-period-china-ocean-shipping-group-company-cosco-china-cma-cgm-sa-france-agili/#respond Sat, 08 Jun 2019 06:51:49 +0000 http://www.bestdevops.com/?p=5202 The global maritime containerization industry encompasses the shipping, handling, and logistics of standardized cargo containers across ocean routes. As global trade volumes have grown, so too has the strategic importance of containerization — touching sectors from oil and gas to consumer goods. This article summarizes the market landscape, major players, and segmentation framework commonly used to analyze this industry.

Source note: This article is based on a market report summary originally published on foxworldmedia.com and cited by researchreportcenter.com. Specific figures, forecasts, and growth rates from the underlying paid report have not been independently verified.

What Is Maritime Containerization?

Maritime containerization refers to the intermodal freight transport system that uses standardized ISO shipping containers to move goods by sea. The system dramatically reduces loading and unloading times at ports, lowers cargo damage rates, and enables seamless transfer between ships, rail, and trucks. Today, containerized shipping accounts for the majority of globally traded manufactured goods.

Market research in this space typically draws on primary data (industry interviews, surveys) and secondary data (trade association databases, government shipping statistics, company annual reports, and financial filings) to build forecasts across a multi-year horizon.

Key Industry Players

The maritime containerization market is dominated by a relatively small number of large carriers, port operators, and logistics companies. The following organizations are frequently cited as major participants in this market:

  • China Ocean Shipping (Group) Company — COSCO (China): One of the world’s largest shipping conglomerates, operating a significant global container fleet.
  • CMA CGM SA (France): A leading global container shipping company with a wide network of trade routes.
  • A.P. Moller-Maersk Group (Denmark): Consistently one of the largest container shipping operators by fleet capacity.
  • Mediterranean Shipping Company S.A. — MSC (Switzerland): A major privately held container shipping line.
  • APL Limited (USA): A container carrier operating across key Asia–Pacific and trans-Pacific trade lanes.
  • Evergreen Marine Corp Ltd (Taiwan): A large container shipping company with a global route network.
  • Dubai Ports World Limited — DP World (UAE): A leading port operator and logistics provider across multiple continents.
  • Agility Logistics (Kuwait): A global logistics and supply chain company with operations across emerging markets.
  • SSA Marine Inc. (USA): A major stevedoring and terminal operating company.
  • Transport Corporation of India Ltd (India): A logistics and supply chain company serving domestic and international freight needs.
  • Amerijet International Inc. (USA): An air and ocean cargo carrier focused on the Americas.
  • Metro Ports (USA) and American Stevedoring Incorporated (USA): Port services and stevedoring operators in North American ports.
  • China Shipping Container Lines Company Limited (China): A major Chinese state-affiliated container carrier.
  • Evergreen Shipping Agency Corp (USA): The North American agency arm of Evergreen’s global operations.
  • Exel PLC (USA) and Gati Ltd (India): Supply chain and express logistics operators serving regional and global markets.

Market Segmentation

By Product Type

Maritime containerization research typically segments the market along the following product dimensions:

  • Ocean Vessel: The type and size of vessel (e.g., ultra-large container ships, feeder vessels) used to transport containers.
  • Cargo Type: The nature of goods shipped — dry bulk, refrigerated (reefer), hazardous, or general cargo — each requiring different container specifications.
  • Port Management Model: How port terminals are operated, including public landlord models, private concession models, and fully privatized operations.

By Application

The industries relying on containerized maritime shipping span several sectors:

  • Oil & Gas: Includes equipment, chemicals, and related materials shipped in specialized containers.
  • Agriculture: Bulk grains, perishables, and processed food products frequently rely on reefer and dry containers.
  • Consumer Goods: Electronics, apparel, appliances, and retail goods make up a significant share of global container volumes.
  • Others: Automotive parts, construction materials, pharmaceuticals, and industrial machinery.

By Region

Regional analysis in maritime containerization typically covers:

  • North America
  • China
  • Rest of Asia-Pacific
  • United Kingdom
  • Europe (excluding UK)
  • Central & South America
  • Middle East & Africa

Asia-Pacific — and China in particular — accounts for a disproportionately large share of global containerized trade volumes, given its role as a manufacturing hub for exported goods.

What a Comprehensive Market Report Typically Covers

A full maritime containerization market research report generally includes the following analytical sections:

  • Market Overview: A summary of total market size, forecast growth trajectory, and key product and service categories within the industry.
  • Competitive Landscape: Analysis of market share, revenue comparisons, and competitive positioning among major carriers and port operators.
  • Regional Evaluation: Assessment of regional growth drivers, trade flows, port throughput metrics, and investment trends across major geographies.
  • Company Profiles: In-depth profiles of leading players, covering fleet capacity, revenue, routes served, strategic initiatives, and pricing.
  • Manufacturing and Operating Cost Analysis: Breakdown of cost structures including fuel, labor, port fees, and vessel maintenance.
  • Supply Chain and Sourcing Strategy: Examination of upstream and downstream relationships, including raw material sourcing for container manufacturing and buyer concentration.
  • Marketing and Distribution Channels: How carriers and logistics providers reach customers, including digital freight platforms, direct contracting, and freight forwarder networks.
  • SWOT and Macro-Environment Analysis: Evaluation of strengths, weaknesses, opportunities, and threats, alongside macroeconomic, regulatory, and geopolitical factors affecting global shipping demand.

Industry Context and Relevance for DevOps and Technology Teams

Maritime containerization is directly relevant to technology practitioners building logistics platforms, supply chain visibility tools, port automation systems, and IoT-based cargo tracking solutions. The shift toward digitalized port operations — including automated terminal management systems, real-time container tracking via AIS (Automatic Identification System), and predictive ETAs powered by machine learning — makes this market an active area of software development and systems integration investment.

DevOps teams working in logistics technology should be aware of the scale of this industry: the volume of transactions, the criticality of uptime for port operating systems, and the complexity of integrating data from carriers, customs authorities, and end customers across different regulatory environments.

Summary

The maritime containerization market is a foundational pillar of global trade, dominated by a handful of major carriers and port operators. Understanding its segmentation — by vessel type, cargo type, port model, application sector, and geography — provides useful context for logistics software developers, supply chain architects, and anyone building technology that interfaces with global shipping infrastructure.

For access to the full research report, the original citation directs readers to researchreportcenter.com, though readers should evaluate the source and data methodology independently before citing figures in professional or academic contexts.

]]> https://www.bestdevops.com/maritime-containerization-market-outlook-highlights-major-opportunities-likely-to-steer-demand-during-forecast-period-china-ocean-shipping-group-company-cosco-china-cma-cgm-sa-france-agili/feed/ 0 Docker Database Breach: 190,000 Users Affected — What Happened and What to Do https://www.bestdevops.com/docker-database-hacked-190000-users-affected/ https://www.bestdevops.com/docker-database-hacked-190000-users-affected/#respond Fri, 03 May 2019 12:56:45 +0000 http://www.bestdevops.com/?p=5069 Original reporting credit: Tom’s Hardware. Additional details sourced from Bleeping Computer.

Overview of the Breach

In late April 2019, Docker — the widely used container platform that lets engineering teams build, ship, and run applications in isolated environments — announced that an unauthorized party had gained access to one of its databases. Approximately 190,000 user accounts were affected. The exposed data included:

  • Usernames
  • Hashed passwords
  • GitHub and Bitbucket OAuth access tokens (for a subset of affected users)

Docker disclosed the incident through its Support Center and via direct notification to potentially affected users, rather than through its main blog or social media channels — a communications decision that drew criticism from the security community.

What Was Compromised and Why It Matters

Usernames and Hashed Passwords

For users whose only exposed data was a username and a hashed password, the immediate risk was relatively limited — provided Docker used a strong hashing algorithm (such as bcrypt) with appropriate salting. Hashed passwords are not directly usable by an attacker; however, weak or commonly used passwords can still be cracked through dictionary attacks or rainbow table lookups. Users who reused their Docker password on other services faced a much higher risk of credential stuffing attacks on those accounts.

GitHub and Bitbucket Access Tokens — The Higher-Risk Exposure

The more serious concern involved OAuth access tokens for GitHub and Bitbucket. Docker Hub supports automated builds — a feature that links a Docker Hub repository to a source code repository on GitHub or Bitbucket so that a new container image is built automatically whenever code is pushed. These integrations rely on OAuth tokens that grant Docker Hub permission to read (and sometimes write) to the linked repositories.

According to reporting by Bleeping Computer, depending on the permissions granted to each token, a malicious actor in possession of these tokens could potentially:

  • Read private source code repositories
  • Modify repository contents, including injecting malicious code into build pipelines
  • Trigger downstream supply-chain attacks affecting consumers of those Docker images

Docker stated that it revoked all known compromised access tokens promptly upon discovering the breach, which should have prevented active exploitation of those tokens after the revocation point. However, any access that occurred before revocation cannot be ruled out without a thorough audit of repository access logs.

Recommended Remediation Steps

Docker advised affected users to take the following actions. Even if you were not directly notified, these are good security hygiene practices for any Docker Hub account holder:

  1. Change your Docker Hub password immediately. Choose a strong, unique password not used on any other service.
  2. Check for password reuse. If you used the same password elsewhere, change it on those services as well. Consider using a password manager.
  3. Reconnect GitHub and Bitbucket integrations. If your account used automated builds, revoke the existing OAuth connection and re-authorize it to generate a fresh token. Do this from the settings pages of both Docker Hub and the respective source code platform.
  4. Audit your repository access logs. Review recent activity in your GitHub or Bitbucket repositories for any unexpected commits, branch modifications, or access from unfamiliar IPs.
  5. Enable multi-factor authentication (MFA). Where supported, enable MFA on Docker Hub, GitHub, and Bitbucket to add a layer of protection beyond passwords alone.
  6. Review OAuth token permissions. Apply the principle of least privilege when re-authorizing integrations — grant only the repository permissions that are actually required for the automated build workflow.

Broader Implications: Supply Chain Security

This incident is a clear example of why supply chain security matters in a containerized DevOps workflow. Automated build pipelines that connect source code repositories to image registries are a powerful productivity feature — but they also represent a trust boundary that attackers can exploit. If a token with write access to a repository is compromised, an attacker could inject malicious code that ends up packaged into a Docker image and pulled by downstream users, potentially at scale.

Teams that rely on Docker Hub automated builds or similar CI/CD integrations should treat OAuth tokens as high-value secrets, rotate them regularly, and monitor for anomalous repository activity as part of standard operations.

Context: Timing and Disclosure Approach

The breach was disclosed during a particularly active period for Docker. The company had just announced a partnership with Arm on April 24, 2019, aimed at simplifying deployment to cloud, edge, and IoT environments running Arm-based processors. DockerCon, the company’s annual container industry conference, also ran from April 29 to May 2. Docker’s choice to disclose the breach via its Support Center rather than its main communications channels was noted by observers as a way of limiting the incident’s visibility during a high-profile week for the company.

Regardless of the communications strategy, the incident highlights that even infrastructure vendors trusted to secure development pipelines are not immune to breaches — and that users of any platform bear responsibility for applying available security controls on their own accounts.

Further Learning: Docker Security

If this incident has prompted you to review your Docker security posture more broadly, consider exploring dedicated training on container security practices, including image scanning, secrets management, runtime security, and network policy configuration. DevOpsSchool offers a course titled Deep Dive into Docker Security that covers advanced Docker security topics for practitioners.

]]> https://www.bestdevops.com/docker-database-hacked-190000-users-affected/feed/ 0 Container-Based Database Platforms: Fueling DevOps https://www.bestdevops.com/container-based-database-platforms-fueling-devops/ https://www.bestdevops.com/container-based-database-platforms-fueling-devops/#respond Tue, 02 Oct 2018 04:58:53 +0000 http://www.bestdevops.com/?p=4447 Source- containerjournal.com

Database systems traditionally have relied on rather complex, OS-specific clustering technology to provide high availability. Depending on vendor and configuration, these technologies included shared storage devices, transaction-based replication, storage replication or some combination thereof. In many cases, these configurations required complex storage and network configuration supported by multiple teams at significant cost.

As the use of virtualization became more widespread, virtual machines offered a degree of high availability, and many hypervisors offered some form of disaster recovery provided by storage replication. The database files were contained within a virtual file format, which made it easy and cheaper to implement; however, since the hypervisor was not aware of the transactional state of the database, it meant failing over could return a database in an inconsistent state.

The vast majority of early container deployments were web and application servers. This was for a couple of reasons, the biggest being the challenges of data persistence in an ephemeral container environment. When Google introduced Kubernetes, databases got both consistent storage and namespaces. This provided a couple of major benefits: It introduced high availability into the solution and separated the storage tier from the compute plane. These factors enabled administrators to have a high degree of flexibility in changing the compute resources for the database tier.

Containers offer many benefits over virtual machine deployment, since they run closer to the system and share operating system binaries. The first of these benefits is performance. Compared to a virtual machine, a database container will have better I/O response time. Secondly, containers allow for simplified deployment using their infrastructure as code model and higher density over virtual machine solutions. Also, since all of the containers on a given host share the same base operating system, the amount of patching is reduced.

Another major benefit is the ability to containerize a gold image or use a vendor-supplied image of an RDBMS. While an RDBMS such as MySQL has a very simple installation process, Oracle and SQL Server both require a fairly complex install, patching and post-installation configuration.

While most of the initial support for database containers was in the open source community for relational databases such as Postgres and MySQL and NoSQL solutions such as MongoDB, Redis and others, commercial database vendors including Oracle, Microsoft and IBM have introduced support for both Docker and Kubernetes in recent years. Kubernetes, via its persistent volumes and service configuration, allows for these databases to be highly available and perform actions such as rolling upgrades for RDBMS patches. Separating data and compute also allows for easier DevOps deployment workflows because of fewer moving pieces.

While implementing a new platform such as Kubernetes can be challenging for many enterprises, the “container as a service” offerings in the major public clouds (Amazon, Google and Microsoft) have made it easier to get started. There are still some challenges around maturity, security and integration with existing third-party applications.

Microsoft has made a big investment into SQL Server and the Kubernetes platform, recently introducing SQL Server 2019 at the Ignite conference. This release includes a new group of services called SQL Server Big Data Clusters, which combine a SQL Server database engine with Spark for machine learning and flexible compute and data pools that take advantage of the underlying Kubernetes infrastructure. This kind of commitment to the platform from a major vendor shows both its early maturity and significant future.

It will be several years before the deployment platform of choice for database servers is containers, and the biggest remaining challenge is disaster recovery. While the Kubernetes platform itself is highly available within a given data center, disaster recovery still requires the database platform to supply a mechanism to perform replication. Some platforms already support this, and I would envision more support to come from all the major vendors in the next year. The advantages of containers are very clear both from DevOps deployment scenarios as well as simplified administration, and this will ultimately drive more database workloads in this direction.

]]> https://www.bestdevops.com/container-based-database-platforms-fueling-devops/feed/ 0 How to Make Docker Containers Faster and Improve Performance https://www.bestdevops.com/how-to-make-docker-containers-faster-and-improve-performance/ https://www.bestdevops.com/how-to-make-docker-containers-faster-and-improve-performance/#respond Sat, 05 Aug 2017 06:54:30 +0000 http://www.bestdevops.com/?p=2348 Docker containers already offer a significant performance advantage over traditional virtual machines. Because containers share the host kernel rather than emulating full hardware, they start faster, consume less memory, and impose lower CPU overhead. Even so, there are concrete steps you can take to squeeze more speed and efficiency out of your Docker workloads.

1. Keep Container Images Lean

Every binary, library, and daemon you package into a container image is a liability: it increases image size, lengthens pull times, and may consume CPU or memory at runtime. Build images that contain only what your application actually needs at run time.

  • Start from a minimal base image. Alpine Linux-based images (e.g., python:3.12-alpine) are often tens of megabytes smaller than their Debian or Ubuntu counterparts.
  • Use multi-stage builds. Compile or bundle assets in a build stage, then copy only the final artifacts into a slim runtime image. This keeps compilers, test tools, and build dependencies out of production images.
  • Remove unnecessary services. If your application does not need an SSH daemon, cron, or a mail server, do not include them. Unnecessary services waste resources and expand the attack surface.
  • Combine RUN instructions. Each RUN layer in a Dockerfile adds metadata and can leave behind cached package lists. Chain related commands (apt-get update && apt-get install -y ... && rm -rf /var/lib/apt/lists/*) to keep layers small.

2. Run Docker on Bare-Metal Hosts

Nesting containers inside a full virtual machine (VM) introduces a second layer of abstraction. The hypervisor consumes CPU cycles and memory for emulation, and I/O must traverse both the VM’s virtual device stack and the host’s physical stack. Running Docker directly on a bare-metal server eliminates that overhead.

If workload isolation is still required, consider system container runtimes such as LXD or OpenVZ. These provide a lightweight guest environment with process and namespace isolation while allowing near-native hardware access — a middle ground between full VMs and uncontained processes.

3. Use a Minimalist Host Operating System

The host OS competes with your containers for CPU, memory, and I/O bandwidth. A general-purpose Linux distribution running a full desktop environment, logging agents, and system services you do not need will reduce the resources available to containerized workloads.

Purpose-built container host operating systems are designed to run Docker (or another container runtime) and little else. Examples include:

  • Alpine Linux — extremely small footprint, suitable for both container images and host OS use.
  • Flatcar Container Linux (the community continuation of CoreOS Container Linux) — immutable, auto-updating, built specifically for running containers at scale.
  • Bottlerocket (AWS) — a security-focused, minimal host OS for container workloads.

Whichever host OS you choose, verify that it still meets your security requirements: kernel hardening, SELinux/AppArmor profiles, audit logging, and timely security patch delivery are non-negotiable in production.

4. Adopt a Microservices Architecture

A monolithic application bundled into a single container starts slowly, scales awkwardly, and redeploys entirely even when only one feature changes. Decomposing your application into microservices — small, independently deployable services each running in its own container — offers several performance benefits:

  • Faster startup. A container running a single focused service loads less code and initialises fewer dependencies.
  • Granular scaling. You can scale only the service under load, rather than duplicating the entire monolith.
  • Faster deployments. Updating one microservice redeploys only its small image rather than the entire application.

Microservices introduce their own complexity (network latency between services, distributed tracing, service discovery), so weigh the trade-offs for your specific workload.

5. Leverage Docker’s Build Cache

Docker caches the result of each Dockerfile instruction. If the instruction and its inputs have not changed since the last build, Docker reuses the cached layer instead of re-executing the step. Used correctly, this can dramatically shorten image build times in CI/CD pipelines.

Key guidelines for effective cache use:

  • Order instructions from least to most frequently changed. Place COPY instructions for dependency manifests (package.json, requirements.txt) before copying application source code. That way, dependency installation is only re-run when manifests change.
  • Understand cache invalidation. Any change to a layer — including a file modification or an updated base image digest — invalidates all subsequent layers. Structure your Dockerfile to minimise the blast radius of common changes.
  • Use --cache-from in CI. Pass a previously built image as a cache source in pipeline builds where the local layer cache is not persisted between runs.
  • Be cautious with apt-get update. If cached, apt-get update can produce stale package metadata. Always pair it with apt-get install in the same RUN instruction so both are invalidated together.

Summary

Optimising Docker performance is not a single configuration switch — it spans image design, infrastructure choices, application architecture, and build tooling. The five strategies above form a strong baseline:

  1. Build lean, minimal container images.
  2. Run Docker on bare metal (or lightweight system containers) where possible.
  3. Use a purpose-built, minimal host OS.
  4. Break monoliths into microservices for faster startup and targeted scaling.
  5. Structure Dockerfiles to exploit the build cache effectively.

Apply these techniques incrementally, measure the impact in your specific environment, and prioritise the changes that deliver the greatest gain for your workload profile.

Original source reference: containerjournal.com (2017)

]]> https://www.bestdevops.com/how-to-make-docker-containers-faster-and-improve-performance/feed/ 0 20 Docker Security Tools Compared: A Comprehensive Guide https://www.bestdevops.com/20-docker-security-tools-compared/ https://www.bestdevops.com/20-docker-security-tools-compared/#respond Sat, 05 Aug 2017 06:43:25 +0000 http://www.bestdevops.com/?p=2344 Source: Originally based on a comparison published by sysdig.com.

Docker is not inherently insecure. Features such as process isolation via user namespaces, resource encapsulation with cgroups, immutable images, and minimal dependency shipping all reduce the attack surface significantly. That said, there is much more you can do beyond basic image vulnerability scanning. This guide covers 20 container and Docker-specific security tools, organized by use case, to help you implement defence-in-depth for your containerized workloads.

1. Anchore Navigator

Homepage: https://anchore.io/
License: Commercial; some services are free to use.
Use Cases: Pre-production image analysis, CVE newsfeed.

Anchore Navigator provides deep inspection of public Docker images, including full visibility into image contents, build history, and discovered CVE threats — each linked to a complete issue description and known fixes. You can analyze your own private images and subscribe to images you frequently deploy, receiving security alerts when new vulnerabilities are disclosed. Upgrading to the commercial tier unlocks additional scanning depth and subscription features.

2. AppArmor

Homepage: http://wiki.apparmor.net
License: Open Source.
Use Cases: Runtime protection, Mandatory Access Control (MAC).

AppArmor is a Linux Mandatory Access Control (MAC) system that lets administrators assign per-program security profiles governing filesystem access, network capabilities, link and execute rules, and more. When a forbidden action is attempted, AppArmor blocks it and can log the violation for review.

AppArmor is often considered a more accessible alternative to SELinux — you only need to learn the profile language syntax to start writing your own rules. Docker automatically generates and loads a default AppArmor profile for containers named docker-default, and you can create specific security profiles for individual containers or the applications running inside them.

3. Aqua Security

Homepage: https://www.aquasec.com/
License: Commercial.
Use Cases: Pre-production analysis, runtime protection, compliance and audit.

Aqua Security is a commercial container security platform designed from the ground up for containerized environments. Key capabilities include security auditing, container image verification, runtime protection, automated policy learning, and intrusion prevention. The platform supports orchestration tools including Docker Swarm, Mesos, Kubernetes, and OpenShift, and can be deployed on-premises or in public cloud environments. A programmatic API is available for CI/CD pipeline integration.

4. Black Duck Docker Security

Homepage: https://www.blackducksoftware.com
License: Commercial.
Use Cases: Pre-production analysis, vulnerability newsfeed, license and legal risk assessment.

Black Duck Hub specializes in container inventory and reporting, mapping known CVEs to indexed image contents and generating cross-project risk reports. The tool can pinpoint specific libraries, software packages, or binaries that introduce security risk and automatically suggest known fixes.

A differentiating feature is Black Duck’s “License Risk” analysis, which evaluates the open-source license obligations introduced by every component bundled into your container images — an important but often overlooked compliance concern.

5. Cilium

Homepage: https://www.cilium.io/
License: Open Source.
Use Cases: HTTP-layer network security, network-layer security policies.

Cilium provides transparent network security for container applications using eBPF (extended Berkeley Packet Filter), a Linux kernel technology that enables high-performance, programmable data-path filtering without kernel module changes. Cilium allows you to define and enforce both network-layer (L3/L4) and HTTP-layer (L7) security policies based on container or pod identity rather than IP addresses.

Cilium leverages BPF for core data-path filtering, packet mangling, monitoring, and redirection. These capabilities require Linux kernel version 4.8.0 or newer.

6. CoreOS Clair

Homepage: https://coreos.com/clair/docs/latest/
License: Open Source.
Use Cases: Pre-production static vulnerability analysis, CVE newsfeed.

Clair is an open-source project for static vulnerability analysis of container images, supporting both AppC and Docker image formats. It periodically refreshes its vulnerability database from configured CVE data sources, indexes software packages found in container image layers, and alerts or blocks deployment when insecure software is detected.

Because analysis is static, containers are never executed during the scan — meaning you can detect threats before they reach a running system. Clair is the security engine powering the CoreOS Quay container registry.

7. Docker Capabilities and Resource Quotas (Built-in)

Homepage: https://www.docker.com
License: Open Source.
Use Cases: Runtime protection, resource-based denial-of-service prevention.

Docker’s built-in security primitives are frequently underutilized. Two in particular deserve attention:

  • Control Groups (cgroups): A Linux kernel feature that limits the access processes and containers have to system resources — CPU, RAM, IOPS, and network bandwidth. Without cgroup limits, a single runaway container can starve other workloads on the same host.
  • Linux Capabilities: Rather than granting containers full root access, capabilities allow you to remove specific privileges from the root account or grant narrowly scoped permissions to user accounts. For example, you can allow a container to bind to port 80 without granting it full network administration rights.

8. Docker Bench for Security

Homepage: https://github.com/docker/docker-bench-security
License: Open Source.
Use Cases: Compliance auditing, configuration baseline checking.

Docker Bench for Security is a shell script that checks for dozens of common best practices around deploying Docker containers in production. It is packaged as a Docker container itself, making it trivially easy to run — a single docker run command produces results for approximately 250 checks covering both running containers and the Docker host.

The tests are based on the CIS Docker Community Edition Benchmark and are suitable for assessing Docker CE and Docker Swarm deployments.

9. Dockscan

Homepage: https://github.com/kost/dockscan
License: Open Source.
Use Cases: Compliance auditing, lightweight configuration checks.

Dockscan is a lightweight Ruby script that analyzes Docker installations and running containers on both local and remote hosts. It can be run with a single command and generates HTML reports. Checks include whether resource limits are configured, whether containers are spawning excessive processes or have a high volume of modified files, and whether the Docker host is allowing containers to forward traffic directly to the host gateway.

10. Falco

Homepage: https://www.sysdig.org/falco/
License: Open Source.
Use Cases: Runtime anomaly detection, alerting, forensics.

Sysdig Falco is a behavioral monitoring tool that detects anomalous activity on Linux hosts using the Sysdig kernel instrumentation layer. Unlike enforcement tools such as Seccomp or AppArmor — which block syscalls at the kernel level — Falco runs in user space and uses a kernel module to intercept system calls, then evaluates them against a rich set of rules.

Because it operates in user space, Falco can integrate with external systems like Docker, Kubernetes, and Mesos to enrich alerts with container and orchestration metadata. Rules can target container-specific context, letting you monitor container behaviour without modifying or instrumenting the containers themselves. A default rules file ships with sensible out-of-the-box detections.

11. HashiCorp Vault

Homepage: https://www.vaultproject.io/
License: Free with commercial enterprise tier.
Use Cases: Secrets management, credential distribution, trust management.

HashiCorp Vault is a comprehensive secrets management platform covering passwords, TLS certificates, API keys, access tokens, and SSH credentials. Key features include time-based secret leases, fine-grained access control, dynamic on-the-fly secret generation, and key rolling (renewing encryption keys without losing access to data encrypted under old keys).

Vault maintains a detailed audit log of all secret access and manipulation, enabling operators to trace suspicious activity. In containerized environments — where software entities are constantly created and destroyed — secure, auditable secret distribution is a critical concern. Vault itself can be deployed as a Docker container.

12. NeuVector

Homepage: http://neuvector.com/
License: Commercial.
Use Cases: Real-time runtime protection, compliance and audit.

NeuVector focuses on real-time security at runtime, automatically profiling the behavior of applications, containers, and services to detect security escalations and anomalous activity. Privileged “enforcer” containers are deployed on each physical host with access to the local Docker daemon. NeuVector aims to be non-intrusive by performing automatic behavioral discovery to assist operators in designing appropriate security policies without requiring prior manual configuration.

13. Notary

Homepage: https://github.com/docker/notary
License: Open Source.
Use Cases: Trusted image publishing, supply chain integrity, tamper detection.

Image forgery and tampering is a significant supply-chain risk in Docker-based deployments. Notary is a tool for publishing and managing cryptographically signed, trusted collections of content. It allows you to define trusted publishers and create signed image collections — similar in concept to signed package repositories in traditional Linux distributions, but applied to Docker images.

Notary goals include guaranteeing image freshness (ensuring clients receive the most up-to-date, non-vulnerable version), trust delegation between users, and secure distribution even over untrusted transport channels or mirrors.

14. OpenSCAP

Homepage: https://www.open-scap.org/
License: Open Source.
Use Cases: Compliance auditing, formal security certification.

OpenSCAP provides a suite of automated audit tools that examine software configuration and known vulnerabilities using the NIST-certified Security Content Automation Protocol (SCAP). You can define custom rules and assertions, and routinely verify that deployed software meets your organization’s security standards.

The toolset is designed not only for security hardening but also for producing the formal documentation and reports needed to satisfy official compliance standards. For Docker environments, OpenSCAP provides the oscap-docker tool, which can audit both running containers and stopped (cold) images.

15. REMnux

Homepage: https://remnux.org/
License: Open Source.
Use Cases: Malware analysis, digital forensics.

REMnux is a security-focused Linux distribution based on Ubuntu, designed to assist malware analysts with reverse-engineering malicious software. It bundles a large collection of pre-installed analysis tools including Wireshark, ClamAV, tcpextract, Sysdig, and many others.

REMnux is intended as a self-contained forensics toolkit you can carry on a USB drive and deploy when you suspect a system has been compromised. Conveniently, the REMnux project also publishes many of its integrated tools as individual Docker containers, making difficult-to-install security applications immediately accessible.

16. SELinux

Homepage: https://selinuxproject.org
License: Open Source.
Use Cases: Runtime protection, Mandatory Access Control (MAC), role-based access control.

Security-Enhanced Linux (SELinux) is a Linux kernel security module providing mandatory access controls, mandatory integrity controls, role-based access control (RBAC), and type enforcement. It is frequently compared to AppArmor — both implement MAC — but SELinux is generally considered more powerful, flexible, and correspondingly more complex to configure.

In a Docker context, SELinux provides an additional layer of access policies to isolate the host from containerized applications, limiting the blast radius if a container is compromised.

17. Seccomp

Homepage: https://www.kernel.org
License: Open Source (Linux kernel feature).
Use Cases: Syscall filtering, runtime protection.

Seccomp (Secure Computing Mode) is a Linux kernel sandboxing facility rather than a standalone tool. It uses Berkeley Packet Filter (BPF) rules to filter system calls at the kernel level, functioning conceptually like an iptables firewall for syscalls. With Seccomp you can selectively allow or deny specific syscalls per container — for example, preventing any container from modifying file permissions.

Docker has integrated Seccomp support since Docker Engine version 1.10, using a JSON-based DSL to define profiles that are compiled to Seccomp filters. A detailed technical comparison of Seccomp, Falco, AppArmor, and SELinux is available on the Sysdig blog.

18. Sysdig

Homepage: https://www.sysdig.org/
License: Open Source, with commercial products built on the core technology.
Use Cases: System-level debugging, anomaly investigation, container forensics.

Sysdig is a full-system exploration and debugging tool for Linux. It captures all system calls made by all processes, enabling administrators to troubleshoot OS-level bugs or investigate unexpected process behaviour. The command-line interface uses syntax familiar to tcpdump users, and a ncurses-based UI (similar to htop) allows interactive navigation and filtering of events.

Critically for container security, Sysdig’s capture files allow you to perform post-mortem forensics on containers even after they have been deleted — a key capability in ephemeral containerized environments where evidence can disappear within minutes.

19. Tenable Flawcheck

Homepage: https://www.tenable.com/flawcheck
License: Commercial.
Use Cases: Pre-production image scanning, vulnerability detection in CI/CD pipelines.

Tenable — best known for the Nessus vulnerability scanner — acquired Flawcheck, a container-focused security solution. FlawCheck scans container images as they are built, before they reach production, leveraging Tenable’s existing vulnerability, malware, and intrusion detection knowledge base and adapting it to containerized, agile CI/CD workflows.

20. Twistlock

Homepage: https://www.twistlock.com/
License: Commercial.
Use Cases: Pre-production analysis, runtime protection, compliance and audit.

Twistlock is a commercial security suite purpose-built for containerized environments, covering vulnerability management, access control, analytics, forensics, and security standards compliance. It integrates directly into CI/CD pipelines through native plugins for popular tools such as Jenkins and TeamCity, as well as callable webhooks, so every build and test environment can trigger an automated image scan before reaching production.

Choosing the Right Tools for Your Environment

No single tool covers every threat vector. A layered approach is recommended:

  • Image scanning (pre-production): Clair, Anchore, Black Duck, Aqua Security, Twistlock, Tenable Flawcheck
  • Runtime protection and MAC: AppArmor, SELinux, Seccomp, NeuVector, Aqua Security, Twistlock
  • Network security: Cilium
  • Secrets management: HashiCorp Vault
  • Compliance and auditing: Docker Bench for Security, OpenSCAP, Dockscan
  • Behavioural monitoring and forensics: Falco, Sysdig, REMnux
  • Trust and supply chain: Notary

Start with the built-in Docker primitives (cgroups, capabilities, Seccomp, AppArmor/SELinux), add image scanning to your CI/CD pipeline, and layer runtime monitoring and secrets management on top. Commercial platforms like Aqua Security and Twistlock can consolidate many of these functions if operational simplicity is a priority.

]]> https://www.bestdevops.com/20-docker-security-tools-compared/feed/ 0