Building a generic sysadmin policy scanner

After writing the same scripts several times I decided it is time for a generic solution to check Debian servers for configuration consistency. As incidents and mistakes happen each organization collects a set of learnings (let's call it policies) that should be followed in the future. And one important truth is that the free automation and CM tools we use (Chef, Puppet, Ansible, cfengine, Saltstack...) allow to implement policies, but do not seem to care much about proofing correct automation.

How to ensure following policies?

But how to really ensure following these policies? The only way is by checking them and revisiting the check results frequently. One could build a script and send a daily/weekly mail report. This is always a custom solution and that's what I did several times already. So I do it one final time, but this times in a generic way.

Generic Policy Scanning

For me a generic configuration consistency / policy scanner has at least the following requirements:

1. Optional generic pre-defined policies
2. Optional custom user-defined policies
3. Policies checked locally on the host
4. Policies checked from CM systems
5. Per host/hostgroup policy enabling
6. Generic discovery of your hosts
7. Dynamic per policy/group/host result filtering
8. Customizable mail reports
9. Result archival for audits
10. Some simple trending
11. Daily diffs, New findings, Resolved Isses
12. Acknowledging Findings

I started implementing a simple solution (entirely bash and SSH based, realizing requirements 1,2,3,4,6,7,9,10) with https://github.com/lwindolf/polscan. It is quite easy to setup by configuring the type of

• Host list provider (e.g. Chef, Puppet, mcollective)
• SSH access available
• Sudo yes/no

and you can run it instantly with the default set of policy scanners (which of course not necessarily all make sense for all type of systems).

Implemented Scanners

• Network - empty-hosts
• Network - hostname-resolve
• Network - ignore-broadcast-requests
• Network - ignore-icmp-requests
• Network - no-ip-src-routing
• Network - rp-filter
• Network - syn-cookies-on
• Network - tcp-wrapper
• Performance - swappiness
• Puppet - apt-repos-managed
• Puppet - not-disabled
• Puppet - ssh-keys-managed
• Puppet - sudoers-managed
• Puppet - users-managed
• Security - apache-server-tokens
• Security - apparmor-no-complain
• Security - aslr-enabled
• Security - dmesg-restrict
• Security - enforce-history
• Security - ipv4-forwarding
• Security - ipv6-forwarding
• Security - kptr_restrict
• Security - nginx-server-tokens
• Security - nginx-size-limits
• Security - no-at
• Security - no-autofs
• Security - no-avahi
• Security - no-compiler
• Security - no-core-dumps
• Security - no-ctrlaltdel
• Security - no-portmap
• Security - no-root-aliases
• Security - no-telnetd
• Security - ntpd-active
• Security - nx-enabled
• Security - pam-cracklib
• Security - pam-no-nullok
• Security - pending-restarts
• Security - pending-updates
• Security - pending-updates2
• Security - pending-updates3
• Security - remote-fs-mounts
• Security - repo-enabled
• Security - securetty
• Security - selinux-enabled
• Security - sysrq-disabled
• Ssh - hashknownhosts
• Ssh - legacy-disabled
• Ssh - no-keyboard
• Ssh - no-root
• Ssh - no-tcp-forward
• Ssh - no-x11-forward
• Ssh - privilege-separation
• Ssh - sftp-disabled
• Ssh - strict-mode
• System - apt
• System - home-partition
• System - mounts
• System - ntpd-slew
• System - tmp-partition
• System - unattended-upgrades
• System - var-partition
• Updates - release

By setting up the results and the static HTML (instructions in README.md) in some webserver document root you can browse through the results.

Screenshots

Result overview:

Filter details: