Automating my operations - aka DevOps

As part of my day job, I regularly interact with systems such as jenkins, terraform, ansible/salt, etc.. without really understand the full power of these systems and what they enable. Time to change and dive in deeper and actually use them for my personal projects.

So, what are common tasks that I need to automate:

• Creating / Destroying the necessary infrastructure, such as setting up VM’s with a sane default firewall with all the common dependencies pre-built into the image or installed as part of user-data.
• Deploying the apps to the setup machines with any necessary configuration
• Shutting down and restarting services
• Updating configuration
• Deploying new changes to the machines
• Creating a staging/dev environment to test changes in before they are deployed to production.

These tasks can usually be automated and managed via:

• Infrastructure As Code (IaC): Where you use code to automate the provisioning and configuration of infrastructure. It comes in two main flavors:
  • Infrastructure Provisioning: Manage infrastructure. Think creation of VM instance and other resources and the ability to tear them down easily. A popular tool for IaC is Terraform but Pulumi and OpenTofu (Terraform fork) are popular alternatives. Terraform has a declarative approach, where you describe the end state of your infrastructure, and it executes the procedures necessary to get there.
  • Configuration Management (CM): Managing the applications, configurations, and environment in the deployed infrastructure. While Infrastructure provisioning tools also enable you to manage configuration, it’s better to use a different tool that’s focused on configuration management such as Ansible, Salt, Puppet.
• Continuous Integration & Continuous Deployment (CD**)**
  • Used to setup automation pipelines to achieve a variety of tasks. Jenkins is one of the most popular open source CI/CD tools, though a variety of other exists that are easier to configure.
  • Common tasks to test changes on a specified environment before they can be integrated to the main branch or a release.
  • Another pipeline could take cut release and run them through a testing phase on a staging/QA environment with some bake time before deploying them to the prod environment.
  • Another pipeline could be used to deploy config changes via CM (ansible) to the target machines.

I will be using Jenkins to setup a CI/CD pipeline to test and deploy my projects, and Terraform to provision the necessary infrastructure, and Ansible to configure the machines for my projects. This is mainly to save myself time and minimize issues by having easily reproducible environments.

Topics I’ll be exploring further:

• How to manage secrets in these different tools
• How to securely share and use resulting IaC state
• The practicalities of operationalizing these services and getting them to work together seamlessly.