Today’s blend of third-party application dependencies and polyglot software development often makes assessing risk difficult. With many new cloud-native deployment models, it can be tricky to discover potential vulnerabilities. These threats take the form of insecure default settings in Kubernetes, over-permissive states, CVEs that threaten container integrity, and other vulnerable conditions.

Plugging gaps throughout the cloud-native strata is now crucial to avoid exposing data and breaking privacy regulations. Yet, gaining visibility into these holdings is challenging, and traditional application security practices may not cut it within a cloud-native context.

A new 2021 CISO Report released by Dynatrace, “Precise, automatic risk and impact assessment is key for DevSecOps,” demonstrates increased false positives and a general lack of real-time container runtime vulnerability scanning among enterprise IT departments worldwide. Of the 700 CISOs surveyed, most report that microservices, containers and Kubernetes have caused security blind spots across their organizations.

Below, we’ll cover the major takeaways from the study to see why new cloud-native development models are leaving security gaps. We’ll consider how CISOs can increase security throughout the cloud-native IT stack while still reaping the rewards of container-based delivery models.

Cloud-Native Security Blindspots
Adopting new container-based technologies may introduce multiple nuanced security implications. Of the tech leaders surveyed, 89% report that microservices, containers and Kubernetes have created new vulnerabilities within their organizations.

Engineers are shipping code to maintain release velocity and to meet the demand for digital innovation. In the process, 28% of CISOs say application teams sometimes bypass vulnerability scans to speed up software delivery. Due to these realities, it’s no surprise that 71% of CISOs admit they are not fully confident code is free of vulnerabilities before going live in production.

As some teams bypass security pre-production, it follows that more aren’t applying holistic security scanning during production. As a result, nearly all organizations (97%) lack real-time visibility into runtime vulnerabilities, found the report.

Adding to the number of cloud-native blindspots is information overload. The report calculated that, on average, an organization receives a staggering 2,169 new alerts of potential application security vulnerabilities each month. Out of this high volume of alerts, 77% of CISOs say they are mostly false positives and not actual exposures.

Due to minimal testing and a high number of false positives, organizations are likely to leave many vulnerabilities unpatched. To make matters worse, developers, pressed for time, may lack the bandwidth to resolve known issues.

Faster Delivery, Harder Detection
Much of the setbacks in vulnerability detection may lie in a general shift toward new ways of working and faster delivery cycles. For example, 63% of CISOs reported that DevOps and agile methodologies are making it harder to detect and manage vulnerabilities.

In addition, 74% of CISOs say traditional security controls such as vulnerability scanners no longer fit today’s cloud-native world, and 77% of CISOs agree that automation could help. This could involve replacing manual deployment, configuration and management with more automated approaches. Others agree that security-as-code is required to ensure high container security best practices.

Another result of more frequent delivery is increased over-reporting. Of the total security vulnerability alerts received per month, CISOs report that only 42% of these alerts require action. More rapid release cycles could thus exacerbate already crowded vulnerability detection systems.

Who Owns Vulnerability Management?
Only 3% of organizations have real-time visibility into runtime vulnerabilities in containerized production environments, the study found. In reality, vulnerability scanning in production is only happening once a month or less within 68% of organizations.

To compensate for this gap, most CISOs believe the onus is not on the security team but the application and DevOps teams to manage and respond to vulnerabilities—85% of CISOs believe these groups should hold responsibility for vulnerability management. These findings come to light as other stakeholders advocate for increased full-cycle developer control and encourage the DevOps mindset across roles.

Handing more control to developers sounds good, yet, 64% of respondents say developers don’t always have time to resolve vulnerabilities before code goes into production. As a result, over one-quarter of application teams bypass vulnerability scanning entirely. Often, engineering teams may not frequently sync with security groups, and DevSecOps principles are not yet pervasive throughout all organizations.

The report also highlighted the fact that existing security tools are often ineffective; 69% of CISOs say security tools slow down DevOps because they only focus on one state of the software delivery cycle.

Cloud Use Requires Security Repositioning
As cloud architectures rise in prominence, organizations are adopting cloud-native tools like microservices, containers and Kubernetes. There is also the chance of multi-tool conditions further complicating the vulnerability scanning process. With more systems to protect than ever before, teams may have to reposition their approach to vulnerability detection.

“The increased use of cloud-native architectures has fundamentally broken traditional approaches to application security,” said Bernd Greifeneder, founder and chief technology officer at Dynatrace. “This research confirms what we’ve long anticipated: manual vulnerability scans and impact assessments are no longer able to keep up with the pace of change in today’s dynamic cloud environments and rapid innovation cycles.”

The Global CISO report, “Precise, automatic risk and impact assessment is key for DevSecOps,” from Dynatrace, queried 700 chief information security officers (CISOs) on their monitoring practices concerning vulnerabilities. To review the report’s full findings,

The concept of shifting security left is not new, but historically this has meant little more than inserting security processes in the middle of development and slowing everything down. In this article, I’ll describe older shift-left methods that have not worked — and how a modern approach to shifting left can have a high impact on risk reduction and create a healthy balance of freedom and responsibility for cloud-native development teams.

I believe CISOs have no choice but to embrace this industry change. They must enable development teams to move faster while simultaneously reducing the catastrophic risk of deploying broken infrastructure.

The DevOps Shift-Left

Traditionally, for an application developer, the infrastructure you needed was a server. So you would file a ticket, and someone from the ops team would create a VM instance or order a physical server. Some number of days, weeks or months later, the server would be delivered through a combination of ClickOps, scripts and manual processes.

Today, modern application developers are directly provisioning applications to the cloud by using cloud platforms like AWS, Google and Azure without any additional assistance from IT or platform teams. Developers release and update software on demand in the cloud using continuous integration and continuous deployment (CI/CD) for rapid software releases and updates.

ow using declarative Infrastructure as Code (IaC) — things like Terraform, CloudFormation, Kubernetes manifests, Helm Charts, etc. — to automate the provisioning of their cloud infrastructure. The code manages the resources for their applications — servers, databases, networks, logs, application deployment and configurations — and reduces the burden for IT, Ops and site reliability engineering (SRE) teams.

This has shifted DevOps left and allowed developers to provision their own infrastructure using software development practices such as writing, testing and executing code. If they want to make changes or tear down the infrastructure, they code it, test it and then deploy the changes.

Volumetric Rise In Attack Surface

The power and scale of modern software development cycles adds a new risk dimension to security. Software provisioned with IaC can go from developer laptops to clouds to customers in seconds.

The problem is that there is often little to no room for traditional security intervention in this development process. Rapid deployment of IaC by multiple developers of varying skill and experience levels means there’s a high chance for mistakes. These problems manifest at runtime, where security teams have good visibility with solutions that detect configuration problems. The problem is that by this point, these problems can be difficult and costly to fix.

Last generation’s shift-left security has been trying to shift left for years. You can try to catch vulnerabilities and misconfigurations before they are exploited to reduce the risk of breaches in production. You can run a static analysis of application code and dynamic analysis of live applications, generate reports, pummel your ticketing systems for remediation work, and curse at false positives. It’s a frustrating process that was designed to work in slow multistep release cycles with defined approval gates. It requires pauses between deploys to complete security scans, and then it requires time for remediation activity to catch up.

Inserting security into development has also traditionally meant hiring and deploying software security experts into development teams. These experts may be available for a plethora of tasks: “threat modeling” new architectures, defining security features, consulting on risk priority, training up security champions and such. All of these tasks are costly and time-intensive.

On top of this, in most organizations I’ve worked with, security is vastly outgunned in headcount. Large development teams (DevOps, SREs, developers) that are armed with CI/CD and IaC are unstoppable. I’d estimate that the current industry ratio sits around 1.5 security experts per 100 software engineers. This model simply does not scale when companies are confronting modern decentralized development. To quote Bilbo Baggins in The Lord of the Rings, “I feel thin, sort of stretched, like butter scraped over too much bread.”

The Modern Shift-Left Security Strategy

A modern shift-left approach shifts security responsibilities to those creating software, the developers, and it shifts it to the beginning of the process when the developers are provisioning infrastructure. By testing Terraform and CloudFormation for security before execution, just as they would test their application code for quality and reliability, developers can make any fixes before committing code to production — where it could affect customers and result in data loss.

This shift weaves security into the existing CI/CD processes. Developers should leverage APIs to integrate security solutions into the tools they already use while giving security teams visibility into what has been tested and fixed. If there are issues, the security team should track them back to dev owners and establish policies and templates so that they don’t repeat the same mistakes. This approach could finally help security make a significant impact on reducing risk for cloud-native software development. Development teams don’t have to experience security as “yet another tool” or as “yet another process” that slows things down. Security should become a key aspect of the development process and enable developers to ship secure, reliable solutions without thinking too hard.

Getting the challenge of integrating a security mindset into DevOps practices right will become even more important as new markets of IT such as the Internet of Things spin up, writes Andrew Slavkovic of CyberArk

In some companies, security teams make it impossible for DevOps to deliver any new code until they fix the security issues in it

Chief information security officers (CISOs) are under growing pressures to find ways to overcome the consistent barrage of new cyberthreats.

Part of the strain locally is being driven by the introduction of new government policies, such as open banking.  When it’s phased in from July 2019, the way banks handle customer data and how they approach security and privacy will be scrutinised by the public.

The increased expectation from customers across all industries to access secure and convenient services at their fingertips has given rise to an application development boom within organisations. And managing the fast-paced and ideas-driven DevOps community is one area of business that CISOs collectively find challenging, particularly when it comes to setting and enforcing secure systems and safe practices.

A relatively new discipline propelled by speed and ingenuity, DevOps emerged without in-built security. Furthermore, those in DevOps tend to push back on security requests due to the belief that such controls will limit their ability to innovate quickly.

However, without securing the development pipeline, the risks are high. In fact, CISOs need to rethink their approach to securing DevOps.

As Khadir Fayaz, vice president of global security strategy, engineering and architecture, at Pearson puts it, “All it takes is one developer making an error to result in access keys being checked into code repositories.”

DevOps tools can be compromised to take over applications or an organisation’s entire cloud network.

An example of this was when the details of 50,000 drivers of ride-sharing service Uber were stolen after hackers accessed a private GitHub repository used by Uber software engineers, extracting valid credentials. These credentials provided access to an Amazon Web Service (AWS) account containing the data which was stolen.

A recent study from researchers at North Carolina State University revealed that over 100,000 GitHub repositories contain exposed authentication secrets, including 85,311 unique Google API keys, 37,781 unique RSA Private Keys and 47,814 unique Google OAuth IDs. The researchers also found that thousands more repositories are leaking new, unique secrets every day.

To combat the challenge, CISOs must tailor their management approach to the specific needs of the DevOps community. Reshaping how security policies and procedures are enforced, to limit and modify each user’s access to DevOps tools and applications, will demonstrate to developers how restricting access to privileged credentials can help protect an organisation from malicious attacks.

Showing – instead of telling – the DevOps community why managing secrets and privileged credentials is important, and how the practice can be successfully built into application development, helps demystify any misconceptions about security.

“Instead of having developers look at policies, standards and frameworks to figure out what they should do, build consumable security components that are easily adopted. Think in terms of, ‘How do I allow my developers to do the right thing without stifling this agility?’” suggests Fred Gibbins, senior vice president and chief information security oOfficer at American Express.

One example of this is breaking the build, a strategy that has been used by companies like Starbucks. With this approach, security teams make it impossible for DevOps to deliver any new code until they fix the security issues in it. This ensures risk scoring is built into automation tools, so that if the risk exceeds a certain threshold, the build automatically breaks, and developers need to fix it before they can deliver.

CISOs’ that tailor their approach to help DevOps teams understand the need for security by design illustrate their understanding of the DevOps space, how the community works, and what their needs are. CISOs who have made this step have expanded traditional security programs, such as monitoring privileged access across the tool chain, to the DevOps community with little protest.

At the same time, transitioning to an attacking mindset will ensure CISOs are more likely to be able to detect an incident or potential incident before it happens. This proactive approach to security, however, requires a shift from legacy approaches to security.

In the past, CISOs have held a reactive mentality towards security, relying heavily on people being compliant to certain policy documents and having a centralised security management system for all company secrets and credentials.

Introducing security by design and encouraging the cross pollination of security and DevOps teams, should help to remove any feeling that security is authoritarian and instead foster a collaborative partnership between the two.

There a few ways to do this. Setting up a centre of excellence or engineering council with developers and security members who are responsible for selecting and configuring tools, enhancing the developer experience, and promoting good engineering practices is one way.

Another is appointing security champions or community leaders to serve as resources to drive the implementation of practices across the company.

“If the overall culture doesn’t allow security to be partners with DevOps environment, security runs the risk of being a ‘check the box’ compliance exercise. Security awareness also has to be embedded in the company’s culture – especially with DevOps team members,” says Dawn Cappelli, vice president and chief information security officer at Rockwell Automation.

Getting the challenge of integrating a security mindset into DevOps practices right will become even more important as new markets of IT such as the Internet of Things spin up. Using their approach to DevOps security as a standard in the new era, CISOs will not only have to understand new technologies but also modify their method to managing security protocols and safe practices.

CISOs should never be complacent. Continual reviews of policies and governance will keep organisations on high alert about new and potential attacks.