DevSecOps-Playbook: step-by-step implementation guide
👩🏿‍💻

DevSecOps-Playbook: step-by-step implementation guide

📅 [ Archival Date ]
Nov 5, 2022 10:22 PM
🏷️ [ Tags ]
DevSecOps
✍️ [ Author ]
Paul McCarty

DevSecOps Playbook - Version 1.2 - August 2022

This playbook will help you introduce effective DevSecOps practices in your company, regardless of size. We provide explicit guidance and actionable steps to introduce security controls, measure their effectiveness, and demonstrate value for money to your business leaders. Following this playbook will help teams build materially more secure applications, and that in the end, is the intent.

Some background

This playbook was inspired by several documents and I want to call them out here:

The "Minimum Viable Secure Product", or MVSP is a project that I have profound respect for. The MVSP is a great way to determine how mature an organization's security practices are.

Timo Pagel's amazing "DevSecOps Maturity Model" or DSOMM is a project that I only found recently. There is some overlap between the DSOMM and this document and you should definitely browse through the DSOMM and explore the different maturity levels.

Shift Left

All companies and applications are unique. Blanket statements like "Shift Left" aren't helpful without context. Enterprises and startups have different tech stacks, funding, workforces, regulation, and more. Context matters, and this document provides a roadmap to use your context to determine your next DevSecOps destination.

We want to secure our applications, from cradle to grave. To do this, we have created five "domains" that cover the lifecycle of a software application. There are a total of 58 "controls" or "functions" in the DevSecOps Playbook that are spread evenly across those five domains. We've also added a Compliance Addendum for anyone that is interested in aligning with a specific compliance framework.

Priority and Difficulty explained

We use two rating systems: Priority indicates the order you should implement controls and Difficulty indicates how hard implementation is for this control.

image

The Playbook

Development Environment

The developer's laptop is where most of the magic happens, but also where most of the problems are introduced. If you want to shift as far left as you can this is where you want to land much of your embedded security.

Control
Name
Priority
Description
Difficulty
Maps to security frameworks
1.1
Secure Code Training
2
Developers who receive Secure Code Training are less likely to introduce security bugs, be aware of tooling that can support them, and design systems with security in mind.
Medium
CIS8APRA234NIST 800-53BSSDF1.1
1.2
Source Code Versioning
1
Version Control Systems introduce peer review processes, an auditable history, and consistent work patterns between software engineers.
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.3
.gitignore
1
.gitignore files help prevent accidental commits of sensitive, debug, or workstation specific data
Easy
APRA234CIS8NIST 800-53BSSDF1.1
1.4
Pre-Commit Hook Scans
2
A Pre-Commit Hook for security scans provides timely feedback to engineers and helps to prevent vulnerable code being introduced to a repository
Easy
APRA234CIS8NIST 800-53BSSDF1.1
1.5
Commit Signing
2
Sign all commits to verify that the author is genuine
Easy
APRA234CIS8NIST 800-53BSSDF1.1
1.6
IDE plugins
2
Most IDE's support the use of third-party plugins, and devs should implement these tools to highlight security issues as they happen in realtime while they are programming.
Easy
APRA234CIS8NIST 800-53BSSDF1.1
1.7
Local Software Composition Analysis
1
Helps you find and fix libraries with known security issues
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.8
Local Static Code Analysis
2
Helps you find and fix security vulnerabilities in your source code
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.9
Local Sensitive Data Analysis
1
Audits your repository for secrets, credentials, API keys and similar in dev environment. Secrets stored in source code are visible to other people
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.10
Application Baseline
3
Create a "recipe" for building the application from ground up that takes into consideration its risk and compliance requirements, data sensitivity, stakeholders and relationships with other systems, as well as its technical components.
Medium
APRA234CIS8ISM GSDNIST 800-53BSSDF1.1

Source code management (SCM)

Most companies now store their source code in cloud based repositories like GitHub, Bitbucket or Gitlab. Even if you don't, you will use a centralized place for your software engineers to store their code. Centralization and versioning means that these developers can work together without (mostly) stepping on each others toes. Joe and Molly can both be working on the same component, file or function but their changes won't necessarily break the other one's changes. SCM is also a GREAT place to deploy security functions like server side git hooks and multi-factor authentication for your developers!

Control
Name
Priority
Description
Difficulty
Maps to security frameworks
2.1
Source Code Management
1
Use a centralized source code management (SCM) system like Bitbucket, GitHub or Gitlab
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
2.2
User Roles
1
Create unique user and team roles so that access to source code can be tailored
Easy
APRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.3
SSH
2
Use the SSH protocol to access your repositories instead of HTTPS
Easy
APRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.4
GPG Key
2
Add a GPG key to your central SCM provider. This will guarantee that the person commiting changes to source really are who they say they are
Easy
APRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.5
Multi-Factor Authentication
1
Make sure all developers use multi-factor authentication (MFA) when pulling, fetching or pushing code to remote. This is especially important if you use company email as your login for SCM
Easy
APRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.6
Server Side Git Hook
3
Utilize a server side git hook like update or post-receive hook to run automatic scans
Medium
APRA234CIS8NIST 800-53BSSDF1.1
2.7
Developer Collaboration
2
Use collaboration tools to document the changes to a software application
Difficult
APRA234CIS8NIST 800-53BSSDF1.1
2.8
Pull Requests
1
Enforce pull or merge requests so all code is verified by team lead or senior engineer
Easy
APRA234CIS8ISO27001NIST 800-53BSSDF1.1
2.9
Peer Reviews
1
Enforce peer reviews by software engineers colleagues to increase code quality and security
Easy
APRA234CIS8ISO27001NIST 800-53BSSDF1.1
2.10
CODEOWNERS
1
Create a CODEOWNERS file in the repository that identifies people and teams that own specific parts of the repository and should be consulted via PR when those parts of the repo are modified.
Easy
APRA234CIS8ISO27001NIST 800-53BSSDF1.1
2.11
SECURITY.md
1
Create a SECURITY.md file in your repository that explains who to contact if you find a security issue in the application
Easy
APRA234CIS8ISO27001NIST 800-53BSSDF1.1

CI/CD Pipelines and Automation

Modern web applications are built using modern continuous integration and deployment processes. This means that you run tests specific to whatever environment you are pushing to whether that's DEV, STAGING or PROD.

Control
Name
Priority
Description
Difficulty
Maps to security frameworks
3.1
CI/CD Pipeline
1
Implement a CI/CD pipeline
Medium
APRA234CIS8ISM GSDISO27001SSDF1.1
3.2
Application Environments
2
Create separate environments for dev, staging and prod, and treat each as independent with its own data, testing and requirements
Medium
CIS8ISM GSDISO27001SSDF1.1
3.3
Application Data Separation
3
Make sure that dev and test environments are not using the same data as production. If the use of live data is required then make sure that data is anonymized.
Difficult
CIS8ISM GSDISO27001SSDF1.1
3.4
CI/CD Administration
3
Create and enforce user or team roles so that only the appropriate people can change or disable tests and deployment requirements
Medium
CIS8ISM GSDISO27001SSDF1.1
3.5
Credential Store
1
Create a secure encrypted place to store senstive credentials like passwords, API keys, etc.
Medium
APRA234CIS8ISM GSDNIST 800-53.2bSSDF1.1
3.6
Centralized Software Composition Analysis
1
Scan source code for vulnerable libraries and open source software from within a CD stage
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53.2aSSDF1.1
3.7
Centralized Static Code Analysis
2
Scan source code for vulnerabilities in the source code itself from within a CD stage
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53.2bSSDF1.1
3.8
Centralized Sensitive Data Analysis
2
Scan source code for secrets, credentials, API keys and similar from within a CD stage
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
3.9
DAST
3
Scan running application for vulnerabilities
Medium
CIS8ISM GSDISO27001NIST 800-53BNIST 800-53BSSDF1.1
3.10
Transient Test Compute
2
Verify that the compute you use in CI/CD pipelines are up to date and using most recent applications and operating systems
Medium
CIS8ISM GSDISO27001SSDF1.1
3.11
Harden Transient Compute
3
Harden the transient compute you are using in your pipelines. Follow CIS guidelines for container hardening.
Difficult
CIS8ISM GSMISM GOSHSSDF1.1

Deployment

Applications are deployed somewhere whether that's an AWS Lambda, S3 bucket or some old crusty server in the corner of the server room. In any case, DevSecOps best practices mean that you need to include that deployment location in your processes.

Control
Name
Priority
Description
Difficulty
Maps to security frameworks
4.1
Valid SSL Certificate
1
Create and use a valid SSL certificate for each application URL, or implement a wildcard cert
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.2
Encrypt Traffic
1
Encrypt all traffic that's public facing
Medium
APRA234CIS8CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.3
Redirect to HTTPS
1
Configure web service to redirect all inbound requests to port 80 to the secure HTTPS endpoint
Easy
CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.4
HSTS
1
Enable HSTS in your webserver, load balancer or CDN
Easy
CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.5
CSP
1
Enable content security policy (CSP) in the webserver, load balancer or CDN
Easy
CIS8ISM GSDISO27001NIST 800-53B
4.6
Use Current Software
1
Use the most recent versions of application components, languages, frameworks and operating systems
Difficult
CIS8ISM GSDISO27001SSDF1.1
4.7
Alternative Deployment
3
Have tested and working alternative way to deploy changes to your application other than using your standard process with GitHub or Bitbucket in case they go down. This must include the ability to push to PROD from local in emergencies.
Difficult
CIS8NIST 800-53BSSDF1.1
4.8
security.txt
1
Create a security.txt file in the root of your application so people know how to contact you about security issues
Easy
CIS8ISM GSDSSDF1.1
4.9
X-Forwarded-By
2
Configure your webservers, load balancers & web proxies to include the X-Forwarded-By: header
Easy
APRA234 ATM D-2-d-iCIS8NIST 800
4.10
Logging
1
Collect application logs in realtime and send to centralized storage or SIEM
Medium
CIS8 16.11APRA234ISM GSMNIST 800SSDF1.1
4.11
WAF
2
Implement a web application firewall (WAF) to protect your application from known attacks
Medium
APRA234CIS8NIST 800-53.2a
4.12
CDN
3
Use a content delivery network (CDN) whenever possible to add availability and security to you applications
Medium
APRA234CIS8ISM GNNIST 800-53.2a
4.13
Harden Operating System
2
Harden operating system using industry best practices from CIS, ISM, etc
Difficult
CIS8ISM GSMISM GOSHSSDF1.1
4.14
Encrypt Storage
3
Encrypt all filesystems, disks and cloud storage
Medium
CIS8NIST 800-50bSSDF1.1
4.15
SBOM
3
Generate a real-time software bill-of-materials (SBOM)
Medium
CIS8ISM GSDNIST 800-53BSSDF1.1
4.16
Monitor Application
1
Monitor your application in real-time so you know when its state changes for the worse (or better). This includes uptime, performance and security monitoring
Medium
CIS8NIST 800-53BSSDF1.1
4.17
Cloud Security Posture
2
If your application is deployed in the cloud or uses cloud native services then a solution should be employed to verify that those cloud resources are secure and follow best practices
Medium
CIS8NIST 800-53BSSDF1.1
4.18
Centralized Container Analysis
2
Scan any containers built for deployment for vulnerabilities
Easy
APRA234CIS8ISM GSDISO27001NIST 800-53.2aSSDF1.1
4.19
IaC
2
Use infrastructure as code to build all application environments
Medium
CIS8ISM GSMISM GOSHSSDF1.1

Organizational Techniques

People don't deploy applications, organizations do. Some steps in the DevSecOps playbook need to be owned by the Organization itself.

Control
Name
Priority
Description
Difficulty
Maps to security frameworks
5.1
Penetration Testing
2
Have your application pentested regularly
Medium
CIS8ISM GSDNIST 800-53BSSDF1.1
5.2
Threat Modeling
3
Build a collaborative way for developers and security staff to understand the threat landscape for an individual application
Difficult
CIS8ISM GSDNIST 800-53BSSDF1.1
5.3
SIEM
3
Implement a SIEM and send all application, system and cloud logs to it
Difficult
CIS8NIST 800-53BSSDF1.1
5.4
Attack Surface Management
1
Identify public facing resources via automation
Easy
CIS8CIS8NIST 800-53BSSDF1.1
5.5
Sovereignty
1
Require that all code is written in, stored in, or otherwise served from a location and/or sovereignty that aligns with your org's requirements
Medium
ISM GCSRISO27001
5.6
Vulnerability Disclosure
1
Create and publish a set of procedures to let people contact you when they find security issues in your app
Easy
CIS8ISM GSDSSDF1.1
5.7
Bug Bounty
3
Setup a bug bounty program to incentivize security researchers to tell you about vulnerabilities they find
Medium
CIS8ISM GSDNIST 800-53BSSDF1.1
image

Compliance - Security Framework Reference Material

Because this is meant to be a manifesto about how to do DevSecOps, we have to be cognizant that there are three groups of people that this affects: Developers, Operations and InfoSec. Historically, there are many compliance frameworks that address the InfoSec community and to a lessor extent Operations teams. But software development was never mentioned until only recently. Understanding this, I wanted to note that most commonly there is only one section of a particular compliance framework that relates to software development. This is a list of the specific sections of several larger compliance frameworks that deal with software development specifically.

  • NIST 800-53b SA 10, 11, 15, 16 and 17
  • NIST 800-218 "Secure Software Development Framework"
  • ISO 27001 Annex A Section 14
  • CIS Section 16
  • Australian ISM "Guidelines for Secure Development"

Below you will find links to several security frameworks that align with this document. I have personally spent many years implementing CIS controls into my application environments. CIS is a wonderful framework as its very prescriptive and easy for an engineer to understand. This is not to say that CIS controls are easy to implement. They are not! Regardless, you can't deny the ubiquity of ISO27001 and SOC2 and I want this document to help orgs looking to meet those requirements as well. In fact, SecureStack has started a SOC2 program and in parallel to writing this document I am busily mapping SOC2 requirements and will eventually add them to this document.

I had a number of Australian friends suggest that I tackle the Australian ISM and APRA CPS 234, so I've added both of these as well. This is a work in progress and I encourage anyone that is interested to jump in and suggest mappings. You can add an issue in GitHub or simply create a PR.

NIST 800

NIST 800-218 (2022) "Secure Software Development Framework" (SSDF) version 1.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

  1. SA-11: Developer Testing and Evaluation
  2. SA-15: Development Process, Standards, and Tools
  3. SA-16: Developer-Provided Training
  4. SA-17: Developer Security and Privacy Architecture and Design

CIS Critical Security Control 16: Application Software Security

The Center for Internet Security is an organization that has been providing very prescriptive security controls since 2000. There are a total of 18 security control groups with section 16 being the group we will be referencing for this document.https://www.cisecurity.org/controls/application-software-security

It's not the focus of this document, but CIS maintains an amazing set of benchmarks and build playbooks for most operating systems. I have been using these templates for years and they are a great resource: https://www.cisecurity.org/cis-benchmarks/

Australian ISM

The Australian Cyber Security Centre has authored a document called the "Information Security Manual" or ISM as its commonly called. The landing page for this document is https://www.cyber.gov.au/sites/default/files/2021-12/Information%20Security%20Manual%20%28December%202021%29.pdf.

This document is large and has a very broad scope. You can download the complete ISM at https://www.cyber.gov.au/sites/default/files/2021-12/Information%20Security%20Manual%20%28December%202021%29.pdf

In late 2021 the ACSC released the "Guidelines for Software Development" (GSD). These are a general set of guidelines for embedding secure coding practices into an organization. These guidelines are far from authoritative and are not very prescriptive with my favorite snippet from the GSD being this little gem: "Platform-specific secure programming practices are used when developing software, including using the lowest privilege needed to achieve a task, checking return values of all system calls, validating all inputs and encrypting all communications." Is that a catch all or what?! Wow! Regardless, I am respectful of the energy that went into this set of guidelines and will continue to try and bring visibility to it as much as I personally can.

APRA CPG 234

The Australian Prudential Regulation Authority (APRA) is part of the Australian government and is charged with regulating the financial industry. It published the "Prudential Standard CPS 234" document in 2019 which outlines high level information security requirements.

This document is organized in an unusual way with 8 "attachments" at the end of the doc. It is in these attachments that the security controls and expectations are laid out. You can find the APRA 234 document at https://www.apra.gov.au/sites/default/files/cpg_234_information_security_june_2019_0.pdf.

ISO27001 Annex A.14: System Acquisition, Development & Maintenance

ISO27001 is an international standard on how to manage information security. It measures the maturity of an organization with a total of 114 controls spread across 14 groupings. ISO27001 is built on the principle of gathering the documentation around these security controls in an information security management system (ISMS).

Group A.14 revolves around the acquisition and development of IT systems. It is the only part of the ISO27001 specification that mentions specific

What's been mapped so far?

  • mapped all of ISM GSD
  • mapped all of SSDF
  • mapped CIS section 16

What's left to do?

  • map CIS section other than 16 to all items
  • map remaining ISO 27001 Annex 14
  • create compliance section for ISO 27002
  • map brand new ISO 27002 controls
  • map remaining NIST 800-53
  • map remaining ISM infrastructure

About the author

My name is Paul McCarty and I'm the founder of SecureStack. I created this document as a way to capture in one place the steps I took to implement DevSecOps functions into my team. If you have any questions you can contact me at [email protected] or on twitter @eastside-mccarty