Securing Jenkins

I’ve setup a Jenkins server to manage CI/CD for my various projects, and I want to verify if my setup is overly insecure and what things I could do improve my setups security posture. This post will serve mainly as a guide and checklist for myself.

Securing the Server

I used the linode marketplace to setup my instance. Securing a server is a big topic, and for now, I just want to make sure the basics are covered:

• Separate non-root user for Jenkins app
  • This was handled by the marketplace installation
• Audit who’s on the sudoer list. Should only be Jenkins user.

grep -Po '^sudo.+:\K.*$' /etc/group

• Only HTTPS and SSH are accessible
  • I’ve added Cloud Firewall rules for this, can also setup ufw rules on the machine.
• Secure SSH login
  • I’ve disabled password login, X11 forwarding , and root login
• Audit open ports, ports open on 0.0.0.0 or `` should only be ssh (port 22) and jenkins (port 443)

sudo ss -lntup

• Update firewall rules to allow connections from specific Jenkins agents for distributed builds.
  • I can’t don’t currently have any agents and the controller port is disabled by default.
  • If I were to add any agents, I’d pick a specific non-random port for the controller and update the firewall to only allow connections from the specific agent IPs to the controller port. https://www.jenkins.io/doc/book/security/services/

References:

• Securing a Linux Server: A guide on how to secure a home server.
• Audit web server security regularly
  • https://tuxcare.com/blog/linux-system-hardening-top-10-security-tips/
• SSH Security https://www.ssh.com/academy
• https://linuxhandbook.com/check-if-user-has-sudo-rights/
• Things to double check, murky and suspicious advice:
  • https://www.hostinger.com/tutorials/vps-security?utm_campaign=Generic-Tutorials-DSA|NT:SE|LO:USA-t4&utm_medium=ppc&gad_source=1&gclid=EAIaIQobChMIuPSn0I2xigMVwTUIBR0v_CEmEAAYASAAEgJqAvD_BwE

Securing the Web UI

Configure the server so that only logged in users can read data from the instance. As the number of users grows, It’s best to employ the principles of least privilege when giving access to users and use a Project-based Matrix authorization strategy. The goal is to only give access to the pipelines that a user needs access to.

Another good strategy, is to limit the builds done on the Jenkins controller instance and leverage distributed builds. This helps with:

• Limiting access to the Jenkins machine and the credentials stored in it
• Build side-effects that could compromise the controller machine or alter its configuration

Managing Pipeline Secrets

Use Credentials to store secrets and use them in the pipelines. When adding secrets try to restrict their scope to just the pipelines that need them if possible and restrict user access to system level secrets.

There’s a common gotcha that you should be wary of when using secrets in your Jenkins pipeline: Groovy string interpolation can leak the secrets. Commands using double quotes " will get interpolated, so be sure to write your commands using single quotes ' unless you have a good reason and need the interpolation feature. String interpolation can also cause command injection if you’re taking user-provided strings.

The secrets are only stored on the controller in the _$JENKINS_HOME/secrets/_ directory. The secrets are communicated to the agents through SSH. It’s important to make sure the secrets directory is only readable by root and the jenkins instance, and to avoid including it in the backup. You should probably back it up in a place separate from the main backup (maybe vault? or another secrets manager) , it can be easily restored later.

References:

• https://blog.gitguardian.com/how-to-handle-secrets-with-jenkins/
• https://www.jenkins.io/doc/book/security/credentials/

Speaking of backups, the next post will discuss how to setup regular backups of Jenkins and how to test the backups.