Should I stop work on CfnMason or any project?

Off and on for years, and at various companies, I have developed various tools to manage complex AWS CloudFormation templates and stacks. This came out of the lack of tooling that was associated with CloudFormation itself. It was not that CFN was bad, (it is notoriously picky) it is just that it was designed with the intention of being a way to treat your infrastructure in code based manner. That is really not true. CFN was created as a templating language with defined spec.

Because of these limitations, I have built closed and open source solutions to manage the complexities that involve working with CloudFormation. Recently I even started revamping a tool that I wrote years ago to manage complex CloudFormation Stacks. This update was done on the behest of a few people that actually utilize the tool and wanted to be taken from the messy state it is currently in, into something that could be tied into their current applications. It was from this that I began working on CfnMason as a python module.

However, recently I started working with CDK. CDK is Am,azon’s CloudFormation Development Kit. My first thought was that it was going to be horrible, and why would anyone ever use it. Now, this was before it was a fully supported implementation and was only really viable when used with JavaScript. And, don’t get me wrong, I don’t hate JS, but I do most of my coding in Python these days. So, when I finally had a chance to use it for work, I found that I really like it, and that it was actually an excellent tool.

So, that brings me back to the original question. How do I know when it is time to stop working on a project? The main answer in my mind is you have to figure out that for yourself. When I started writing this, I was pretty sure that I was going to say that I am no longer going to be working on updating CfnMason. But, as I wrote this, I realized that not everyone is going to be able to move over to CDK. There are probably thousands of CFN stacks that have been created over the years, that require updates and tweaks, that are not a good fit to move over to CDK. As of yet, I don’t know of a way to take a template that is in AWS and convert it into a working CDK script so that you can develop on it from there.

This is why writing is sometimes the best way to find an answer to a problem, even technical ones. At face value, there are a number of projects that seem like they should just be discarded and never used again. But, once you analyze the situation properly, you might realize that there is a reason to move forward with development of a seemingly dead solution. It might even be to use it as a growth platform. Or, it could be that while there are new tools available, that for some, older and simpler tools are also still needed.

So, at the end of the day, what started as a note to say that I am no longer going to be working on CfnMason, has been turned around to me stating that I am going to try and get it done. Ha, yeah, even I laughed at that. Although now that the nation is in lock down, there is more of a chance that I might get it finished.

Terraform vs Cloudformation — First Thoughts

Building out AWS infrastructures by hand is not something that should be taken on lightly. Just as in building out a Data Center or service in any cloud environment (Google, VMWare, etc), building out a bunch of systems by hand is a good way to inherit a bunch of technical debt and a headache or three. On top of that, it is slow and cumbersome to build out servers in a manual fashion, and since it is done manually, it is easy to make mistakes and introduce snowflakes into your environment. Instead, we need to address the build out with automation tooling.

There are a number of options to build out an infrastructure in AWS. As such, this is not to say that Terraform and Cloudformation are the only options. However, they are two options that have decent support, are mature, and are used by more than one person. Another option is to roll your own solution going straight against the AWS api in any number of languages. That however, is potentially more trouble than it is worth. Maybe later we can discuss ways to build out and probe an environment with your own tools, but for now let us stick to talking about Terraform and CloudFormation.

What are CloudFormation and Terraform? 

CloudFormation is a tool written by Amazon Web Services as a way to create and control a collection of resources within AWS. It is under continuous development and improvement by the AWS team, for use on AWS.

Terraform is a product produced by Hashicorp. The goal is a tool that is designed to treat your infrastructure as code. It is designed to work with multiple cloud providers, and be versioned under source control like any of your code projects.

CloudFormation

I have used cloud formation to build out entire environments, single servers, and for testing. It is a very useful tool for what it does, with a few limitations.

 

CloudFormation is AWS specific. Very shortly after a new service comes out, it is supported via CloudFormation. This means that almost any service you are going to want to use with CloudFormation will be available.

There are a plethora of examples on how to use CloudFormation. AWS is great about providing examples, and they do not fall short when it comes to Cloudformation. They have examples on how to use various services, and even how to integrate with Chef and Puppet.

A downside of CloudFormation is that if you don’t manage all your services via CloudFormation you can end up in a state where you cause CloudFormation to get into a hung state. This can happen if you delete a resource that was created in CloudFormation.

Terraform

Terraform has been designed to be cloud agnostic with different providers. This means that if you are in a mixed environment, you can use the same tool to build out your infrastructure on AWS, GCC, or Azure. This is definitely a plus if you are not dedicated to AWS, but this can be a disadvantage if you want to use Terraform to manage a new service.

Because Terraform is not designed specifically for AWS, you may end up in a situation where you will have to write your own plugin to manage a service/resource inside of AWS. It is great that it is open source, and anyone can add features, but a bit of a pain in that you may be stuck waiting on new AWS features to be supported.

Support is mixed. There are not as many examples on how to use Terraform as there are for Cloudformation. You may end up using a search engine to try and find examples for the features that you want to use.

But, it has been designed to treat your infrastructure as code, and so it does offer ways to track your environment, and inputs.

Conclusion

At this point it is to early to say. I am going to have to think about it more, and do a deep dive into both Terraform and CloudFormation.