Compliance as Code

March 9, 2017 Phil Dorczuk

Codifying Your Configuration Standards

If you have already gone through a PCI DSS, SOC, HIPAA/HITECH, or ISO assessment, you already know that detailed configuration standards are a must. If you haven’t been through one of these assessments …get ready for some serious typing!

Regardless of compliance requirements, configuration standards are the foundation of a secure environment. Without configuration standards, engineers have nothing to guide them and you could end up with configuration drift or mis-configured systems. Configuration standards are also critical in ensuring that all systems of a specific type are hardening in a consistent and industry-accepted manner.

Now configuration standards are great but the real challenge is how to apply and test them. The simplest and probably least effective way is to have some kind of build checklist that administrators work through when building a new system and test it during your annual assessment when the auditor requests your server configurations.

As an example, here is what a checklist might look like for an engineer to ensure that Telnet is not enabled:


The problem with this is that it is extremely error-prone and if it is only performed when the server is provisioned, someone may install Telnet after the fact and you definitely do not want to find out that some server configuration is out of compliance during your assessment when it is too late to fix it.  

At the other end of the spectrum is Infrastructure as Code and/or Compliance as Code. The key to both of these is automation.  With Infrastructure as Code, you translate your configuration standards into machine-readable code that will be automatically applied when servers are provisioned. This is most commonly done using Configuration Management (CM) tools such as Ansible, Chef, Salt, and Puppet. For example, maybe you have a requirement that Telnet is not installed (which is a standard requirement across any audit framework). When using configuration management tools, this becomes trivially easy to enforce as all the major CM tools allow you to ensure that packages are not installed on a system.

*Note: all the examples here will be done using Ansible however similar functionality and results are achievable using any of the CM tools mentioned above.

In this example I have an Ansible playbook which will make sure that the Telnet and Xinetd (daemon that listens for incoming requests over a network and launches the appropriate service for that request when called) programs are not installed and that the Xinetd service is not running. If you are not familiar with Ansible that is okay, the playbook is pretty self-explanatory. A nice feature in Ansible playbooks is the “tags” element. You can map any number of tags to a play and this lets you run certain subsets of plays based on those tags.


So here I run the playbook against a server that does have Telnet and Xinetd installed and you can see in the “changed” sections that Ansible removed those packages.


And here I run the same play again and you can see that since the Telnet and Xinetd packages were already uninstalled, nothing changes and the tool reports back that no changes were needed. This is one of the major advantages over using a CM tool as opposed to say Bash scripts. CM tools are idempotent meaning that if the server is already configured correctly, then no change is made. This is especially useful for something like appending an entry in the file system table (e.g., /etc/fstab). A bash script will append the entry every time it is run which would leave multiple entries for the same mount point. A CM tool will detect that the entry already exists and make no change to the server.


Now this is not going to be a full tutorial on how to provision servers with CM tools. There are plenty of resources online which can assist you in this much better than I ever could. This is just to show you an example of how you can codify your configuration standards. In fact, with enough detail and tags in your Ansible playbooks, Chef recipes, Puppet manifests, or Salt formulas, they could actually be your configuration standards. And if you keep those configurations in a version control repository such as Git or Bitbucket, you can track any changes made and provide evidence of annual reviews based on commits.

Compliance as Code can be best thought of as a subset of Infrastructure as Code. The idea behind Compliance as Code is that once a server is provisioned using any method, you can run a series of tests against it to confirm that it meets your configuration standards. There are a couple of major players in this space, most notably ServerSpec and InSpec (Chef implementation of ServerSpec). Now you may be wondering what the use case is for Compliance as Code. In the example above, we coded in our configuration requirements when provisioning the server. As we saw in the code example, if you run a CM tool and tell it to install or remove a package, it will provide an indication whether or not it made a change. The main benefit of Compliance as Code is that it lets you write tests rather than configurations. Writing tests let you be more granular than writing configuration code.  ServerSpec tests also fit better into a Plan, Do, Check, Act (PDCA) lifecycle where the CM tool is the Do phase and then your ServerSpec tests form the Check phase to ensure that the CM tool functioned as expected and did not bring any of your systems out of compliance. Or maybe you are an independent auditor and want to provide a way to streamline the audit process for your customers. Rather than have the customer manually pull configurations from a sample of servers, instead you just provide them a set of ServerSpec tests which they can run against the sampled devices. ServerSpec is ideal for this because it does not make any changes to systems and operates over SSH so does not require any additional software outside of SSH which a lot of customers will appreciate.

Here is an example set of Serverspec tests that validate whether Telnet is installed and active.


When run against an insecure server, the following results are displayed:


And when the appropriate corrections are implemented:


In conclusion, I would highly recommend using both Infrastructure as Code and Compliance as Code. Infrastructure as Code is fantastic for provisioning your servers in a consistent manner and baking security configurations into the build process. Compliance as Code is great for continuously monitoring the compliance of systems and providing evidence of that monitoring to external or internal auditors.

About the Author

Phil Dorczuk

Phil Dorczuk is a Senior Associate with Schellman & Company, LLC. Prior to joining Schellman & Company, LLC in 2013, Phil worked as a PCI DSS auditor with Coalfire Systems and a consultant at GTRI. At Coalfire, Phil specialized in PCI DSS audits and gap assessments and at GTRI specialized in Cisco network equipment installation and configuration.

More Content by Phil Dorczuk
Previous Article
HITRUST Webinar with Advize Health
HITRUST Webinar with Advize Health

Thursday, March 30th | 2:00PM - 3:30PM EST

Next Article
The Wacky World of GRC
The Wacky World of GRC

Few areas of technology are as contradictory as governance, risk and compliance. A company might...


First Name
Error - something went wrong!