DevSecOps – the what, why and how

I have worked in the security caper since the early 2000s in various capacities inside enterprises and vendors, and in other IT fields for 10 years or so prior to that.  During that time, I have observed a shift inside IT organisations as they initially struggled to cope with the advent of new security threats, and then spun up security as what we might term a dedicated “Centre of Excellence” capability.

The initial impetus for this was absolutely necessary: large vendors such as Microsoft and Oracle struggled to manage security vulnerabilities in their products and move to a responsible disclosure model; organisations for the first time were attempting to engage with their customers and partners via the internet and could no longer get away with sloppy infrastructure deployments or in-house developed apps; browser vulnerabilities and user naivety meant that phishing scams were rife.  Organisations responded by prioritising security, creating dedicated CSIO and / or CRO positions and funding the buildout of teams.

In subsequent years, organisations moved progressively on to Message Oriented Middleware, standardised J2EE / .NET frameworks, Enterprise Severer Bus approaches, and on-prem, virtualised Private Cloud.  To varying degrees, well-resourced and skilled in-house Security teams have been able to keep pace with the changes and assisted organisations navigating through these transformations.  With the most recent round of changes to adopt public IaaS, PaaS, and SaaS though, things seem to have become unstuck.

Why security has to change to scale

Many enterprises are now at risk of being disrupted by new tech-oriented alternatives and the pace at which change is being driven means that their security team cannot scale to retain the necessary levels of visibility and keep pace.

For a while, organisations have been attempting to throw money at continuing to build out quality security teams, but things have now reached a stage where there aren’t enough good people out there to fill the vacancies.  A plethora of specialised tools have emerged to help make the job of operational security more manageable, but this is at best a reactive approach and doesn’t address the underlying problems of increasing code quality or infrastructure build posture.

While a certain amount of the business change is down to API based consumption of Public Cloud services, I am not convinced this is the root of the problem.  Good Security teams have been able to get their heads around the industry trends as they emerge (whether by re-skilling / cross-skilling infrastructure or devs / training up a new generation of security specialists).  Public Cloud has some security concerns, but these are not insurmountable – cloud vendors and compliance approaches are adapting to transform the way in which consumption of cloud services are able to be governed.  In fact, it was quite common for organisations to leverage niche SaaS based solutions (e.g. CRM from Salesforce, Oracle) in the years preceding rollout of more general-purpose IaaS and PaaS.

Ironically, the problem at the heart of the breakdown in security capability is its very strength – by centralising security expertise, we have (by corollary) deskilled a generation of admins and devs.

“Let security fix this once and fix it well”, “let’s make sure that we’ve got clean separation between security infrastructure and other enabling technology”, “don’t worry, Security has got this”.  With these sorts of phrases, deliberately or as a by-product, we have been training our non-security brethren not to apply to themselves to designing or implementing security.

For Security, DevOps is the game-changer, not cloud

The key transformative change that has occurred is the adoption of DevOps.

This has happened in parallel with emerging cloud adoption and so in the minds of some have been conflated, but is fundamentally a change in the operational model, and not directly related to specific technology.  DevOps approaches typically become easy to adopt in IaaS and PaaS based environments as they have been built with this approach in mind from the ground up.  They are still possible with SDN and private cloud deployments though, admittedly with varying degrees of success dependent on the vendor in question.

I will go through some further real-world examples related to management of security infrastructure, but first, let’s look first at how Security is incorporated into DevOps.

For adoption at an organisational level to succeed, we need an approach to deal with the inevitable sense that Security has of losing visibility of solutions, and dev teams splintering and rolling their own solutions that don’t leverage repeatable patterns.  Lack of discipline has always been a concern for IT organisations, and DevOps done badly or sloppily is no exception.  To resolve this, there will need to be a reorientation within organisation to put developers back on the hook for the security properties of their solutions.

Measurement of security metrics is essential for this to operate effectively.  Metrics need to be consistently embedded in DevOps teams and consistently governed by Security for this to be successful.  Done well, this opens up much greater levels of visibility over the security properties of solutions.  Security has always been time constrained, so ensuring that developers write tickets that define testable and measurable outcomes allows security to get greater traction in the development process.  Devs are able to define and execute targeted tests against aspects of the solution and these can take place independently of each other.

Tests can operate at a variety of points in a DevOps Continuous Integration pipeline:

1. Check-in – static code scan

2. Build – dependency checks

3. Unit test – either positive security test cases, e.g. does the anti-malware agent on my candidate operating system catch the sample; or negative security test cases, e.g. does my micro-services API cope with unexpected input that is thrown at it

4. Integration test – do my checks get through to SIEM solutions and trigger the appropriate alerting and alarming solutions or (better) automated response solutions.

‍

‍

Ensuring that devs are considering security in these phases will allow confidence in the security properties of the solution to be increased as code progresses through the pipeline.  It will feed back security to devs early in the cycle.  Traditionally, dedicated security teams would at best get visibility of the code and access for testing after the solution passed integration phase.  Dedicated pen testing is still a valid quality check, but in a DevSecOps model this is the equivalent of an internal audit and not the primary mechanism to evaluate security.

We’ve adopted these approaches in recent customer engagements and seen vastly improved results from external penetration tests.  A continuous deployment approach allows fixes and improvements (including security control implementation) to be rapidly deployed.  Under CI/CD with measurable security, independent security testing moves from being tied to change and instead moves to a governance role.

So, what’s next?

The harsh reality is that it can take years to transform an organisation, to embrace a new approach, and to realise the benefits of SecDevOps.

Here are some DevSecOps tips to get you started:

1. Train your SecOps team in DevOps techniques and see if they can improve the agility of their siloed solution that they maintain.

2. Embed Security Architects / in house pentesters into DevOps teams and get them working shoulder to shoulder writing and working on tickets.

3. Engage application owners and negotiate some metrics that are able to be tracked through the Integration pipelines, see how this is bubbled up and what correlation there is with subsequent pentest results.

I hope a little of this inspires the Security teams within IT outfits out there to think about what opportunities there are to improve the security posture of your organisation.

If you are working in DevOps, I hope you can see how you can start to take on the responsibility for security that is necessary for your organisation to succeed.  If your organisation isn’t empowering you to improve security, then my advice would be to vote with your feet – find somewhere that does.

In my day job, I work for Versent as our Practice Director for Security and Identity in Sydney.  We do a bunch of stuff related to Public Cloud migration, Identity and Access Management solutioning, and transformation to API based Serverless and Containerised solutions.  It’s my job to help our teams adopt the approaches described here and help our customers move and adapt to take best advantage of them.