Easily enable SSO (Single Sign-On) to protect your AWS Accounts

Enabling SSO on your AWS Accounts is strongly encouraged, adds additional security, access to multiple accounts from a single entry point (with different permission sets), and above all, it is free.

The following is a walk-through of how to enable Single Sign On in the AWS console. These instructions are for people with individual accounts to a few accounts. Actually, these instructions are just a good general starting point for smaller size orgs.

Enable Organizations

  • Log into the AWS Console as the Root account
    • Note: This should be in the primary account that will hold the Organization if you have more than one account. These instructions only go through getting SSO enabled, not adding accounts to an Organization.
  • Go to Organizations (once again another service free of charge)
  • Click “Create organization” and then click “Create organization”
    • Note, this can only be done for one account in the org, and it would be best to do this in a new account if you are going to be adding a large number of other accounts or people.
    • If this is your only account it is not a big deal. Just add it to your base account.
  • Verify your email address to finish the setup process.
    • This is a default. I think it is for security reasons to ensure that not just anyone that has admin access to the account creates an Organization.
    • The email will be sent to the root account email address.

Enable Single Sign On

AWS SSO Landing Page
  • There are two options here. Click the link to “Enable AWS Single Sign On” or search for SSO in the top level search bar.
  • Click “Enable AWS SSO”.
    • Assumptions here:
      • You are going to use AWS as the Identity Source.
      • You have already created the Organization
      • You are happy to be getting this done, and looking forward to being able to access all your accounts from a single location. (Without being charged extra for it). (This sounded much funnier when I was writing it the first time)
Default Landing Page for SSO After Account Creation

Configure Single Sign On

  • Go to Settings.
    • Identity Source — Leave this set to AWS SSO.
      • For me this makes sense. I don’t have a third party solution for this, and I don’t want to have to manage it.
    • User Portal — you can create a custom url. This is nice so that you do not have to attempt to remember the default one that is provided.
    • Multi-factor Authentication — these are my preferred settings.
      • Prompt for MFA – Every time they sign in (always-on)
      • MFA types – for me, either works, but I like to use Authy, because then I can get into it from anywhere. And, if my device breaks, I still have access to my keys.
      • If a user does not yet have a registered MFA device
        • Require them to provide a one-time password sent by email to sign in
        • If you are going to protect your accounts, go all the way with it.
      • Who can manage MFA devices — Users can add their own.

Create Users and Groups

The Create Group Pop-up
  • Next step is to create Groups and Users
    • I like to add the Group first because then I can just add the Users to all the groups when I create them.
    • Users are the people that will have access to the accounts.
    • The groups are just groups of users. The permissions are not tied to there.

Configuring Account Access

Setting up and configuring AWS Single Sign On is almost complete. The last remaining steps are to create Permission Sets and then to assign them to users or groups that are bound to accounts. As a default, I like to create an Admin and ReadOnly permission set. You can attach the users/groups to the accounts first, but I prefer to create them the other way and to create the permission sets, and then attach users/groups to the accounts.

  • Create Permission Set
    • Standard that you can select from a list. These are limited to an hour.
    • Custom. Can still be almost the same, but, one nice thing is that you can change the timeout to up to 12 hours.
Dialog for adding permission set
  • Assign users and groups from the AWS Organization Tab.
    • I only have the 1 account. Click the check box and then click “Assign Users” (it should really read assign users and groups)
    • I like to assign groups and not users. So I have added both to my mix here.
    • Note, you can only add 1 group at a time or they get all the permission sets.
  • On the AWS accounts page. Click on the Name of the account to get an overview of it.

You can now login via SSO

All you have to do now is go to the link that is provided in the configuration. Once you log in, you will get a list of the accounts that you have available to you and the roles that you can assume in each one.

That is really all there is too it. It is quick and easy to setup, and after you start using it, you will wonder why you did not do it sooner. This is a great solution for solo developers, small companies, and even larger companies if you want to get into integration with your own federation servers.

Should I stop work on CfnMason or any project?

Off and on for years, and at various companies, I have developed various tools to manage complex AWS CloudFormation templates and stacks. This came out of the lack of tooling that was associated with CloudFormation itself. It was not that CFN was bad, (it is notoriously picky) it is just that it was designed with the intention of being a way to treat your infrastructure in code based manner. That is really not true. CFN was created as a templating language with defined spec.

Because of these limitations, I have built closed and open source solutions to manage the complexities that involve working with CloudFormation. Recently I even started revamping a tool that I wrote years ago to manage complex CloudFormation Stacks. This update was done on the behest of a few people that actually utilize the tool and wanted to be taken from the messy state it is currently in, into something that could be tied into their current applications. It was from this that I began working on CfnMason as a python module.

However, recently I started working with CDK. CDK is Am,azon’s CloudFormation Development Kit. My first thought was that it was going to be horrible, and why would anyone ever use it. Now, this was before it was a fully supported implementation and was only really viable when used with JavaScript. And, don’t get me wrong, I don’t hate JS, but I do most of my coding in Python these days. So, when I finally had a chance to use it for work, I found that I really like it, and that it was actually an excellent tool.

So, that brings me back to the original question. How do I know when it is time to stop working on a project? The main answer in my mind is you have to figure out that for yourself. When I started writing this, I was pretty sure that I was going to say that I am no longer going to be working on updating CfnMason. But, as I wrote this, I realized that not everyone is going to be able to move over to CDK. There are probably thousands of CFN stacks that have been created over the years, that require updates and tweaks, that are not a good fit to move over to CDK. As of yet, I don’t know of a way to take a template that is in AWS and convert it into a working CDK script so that you can develop on it from there.

This is why writing is sometimes the best way to find an answer to a problem, even technical ones. At face value, there are a number of projects that seem like they should just be discarded and never used again. But, once you analyze the situation properly, you might realize that there is a reason to move forward with development of a seemingly dead solution. It might even be to use it as a growth platform. Or, it could be that while there are new tools available, that for some, older and simpler tools are also still needed.

So, at the end of the day, what started as a note to say that I am no longer going to be working on CfnMason, has been turned around to me stating that I am going to try and get it done. Ha, yeah, even I laughed at that. Although now that the nation is in lock down, there is more of a chance that I might get it finished.

Setting up the cfnmason project for Python

Cfnmason is yet another tool that can be used to manipulate cloudformation stacks. It is not designed to be a replacement for CloudFormation like Terraform, but as a means of making building and managing them easier. I had written a version of this ages ago in Ruby, but with most of my work now being in Python, I am creating a new version in Python. As I have never created an exportable Python package, this will track the process of building and releasing a new PyPi package.

Oh, and to keep things interesting, I am doing this on a mix of Windows 10 and Linux.

Creating the default layout using Poetry

The first thing that we need to do is to create the base package. I could do it by hand, but I want to try out the Poetry package and see if I will hate the decision later.

c:\dev> poetry new cfnmason
...
c:\dev\cfnmason> tree \f \a

|   pyproject.toml
|   README.rst
|
+---cfnmason
|       __init__.py
|
\---tests
        test_cfnmason.py
        __init__.py

Init a new Git repo and add a .gitignore file

These are more civilized times. As such, I almost always create a git repository when I am working on a project, even small ones. They may or may not be public, and it can fluctuate on which platform I use to host my code. This time I am opting for Github, and have uploaded the code.

I have been using gitignore.io to generate base .gitignore files for ages. Type in a few operating systems, the language you are coding for, any IDEs, etc, and you can have a useful .gitignore file right out of the gate. Sure you can do it by hand, but this is quick and easy, and it can always be edited later.

Modifying the Readme file

Poetry starts you with a README.rst file. I don’t know about you, but I have been working with MarkDown for ages. It is common on a number of platforms and there is support for it in a number of editors. I understand the RST files are really designed for technical documentation, but I can burn that bridge later. For now, I need a decent starting point.

We could have started by writing out the template by hand. This would have been long and tedious. Instead, I am using a template. By using a template, I am up and running quickly. I can add and remove parts that I do or do not need, and hopefully, I will not forget a major part. As this is the first pass, I am not going to update the entire thing, but at least get the project name in and the fact that I am working on it.

The license file

The last thing needed before I start working on the code is the license file. Which license you choose is up to you. Personally, I like to use the MIT license. It lets people do what they want. As this is an opensource project, I feel that people should be able to use it as much or as little as they want.

The easiest way to do this is using the Github console. Just add a new file via the web interface and name it LICENSE or LICENSE.md in all caps. Then you will have the option to choose from a list of known licenses.

That is it and next steps

Hmmm. This took longer to write than the work to get the base package up and running. And, that is going to be expected. But, it should also show some of the thoughts and considerations that are needed when creating a new project. Especially if it is going to open for the world.

With that I will leave you to it. In the next couple of days, I am going to start adding in the libraries that are needed to begin working on the project. As I have written this in Ruby before, I have a rough layout in my head of how I want the application to work. And, I know what libraries are needed to meet the core functionality.

The question that I have for myself is if I should take the time to ensure that I update the design docs. Having a design doc is a great way to ensure that you are not missing any features. I am going to have to sleep on it.

An Introduction to AWS Step Functions – Part 2

In Part 1, I went through a whirlwind tour and left example code on how to create a Lambda function and a Step Function. The thing is, I did not explain how the code works. That is what this article is about. I am going to be using the same code from the previous post, but will be going through the code and explaining it. We shall see how well the formatting works.

Jumping right into it. This is the entire code for the StepFunction from the previous post. As of right now, this code has to be json. I did not find any information about supporting yaml. Here is the documentation provided by AWS. It is pretty thorough, but it could be a bit clearer. My explanation could be better or worse.

{
  "Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function",
  "StartAt": "LambdaFunction",
  "States": {
    "LambdaFunction": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:aws-serverless-repository-hello-w-helloworldpython-1JQ8TEEDUAHCE",
      "ResultPath": "$.taskresult",
      "Retry": [
        {
          "ErrorEquals": ["CustomError"],
          "IntervalSeconds": 1,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.TaskFailed"],
          "IntervalSeconds": 30,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.ALL"],
          "IntervalSeconds": 5,
          "MaxAttempts": 5,
          "BackoffRate": 2.0
        }
      ],
      "Next": "ChoiceState"
    },
    "ChoiceState": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.taskresult.value1",
          "StringEquals": "value1",
          "Next": "SuccessState"
        },
        {
          "Variable": "$.taskresult.count",
          "NumericLessThan": 5,
          "Next": "LambdaFunction"
        }
      ],
      "Default": "FailState"
    },
    "SuccessState": {
      "Type": "Succeed"
    },
    "FailState": {
      "Type": "Fail",
      "Cause": "Invalid response.",
      "Error": "ErrorA"
    }
  }
}

Looking at this for the first time, it all seems a bit overwhelming. However, after breaking it down, you will see that while you have to be precise with it, Step Functions are fairly easy to read. (Writing them is still a PITA, but that is just what it is) When you get down to it, the Step Function configuration file only contains 3 sections: ‘Comment‘, ‘StartsAt‘, and ‘States‘. That is it.

Let us take a quick look at the the first two mentioned above, Comment and StartsAt. The former of these is just a comment so you know what the purpose of the Step Function is. On the other hand, you could use this for whatever comment you like. Then, there is StartsAt. StartsAt, actually links to a State. And the States section is what contains all the real meat of the Step Function. The value that is used for StartsAt is the name of one of the States.

States

States are where everything really happens when it comes to Step Functions. They are the individual units that when strung together, make the magic happen. When you look at it from this perspective, it sounds ridiculously easy. But, just like everything else, the devil is in the details. The key to understanding States is to know what the various components that make up a State, and the type of states that are available.

Since this is an introduction, and based off of my code here, I am not going to go through all the options. However, there are docs from Amazon on it. The think about it is, it is not simple. Based on the docs, the only required field is Type. Great, but what are the various types? How do I link them together to make a cohesive whole? What are these other options and how do I pass variables from one section to the next and use them over and over again in a loop? This information is available, but it is scatter all over the documentation and you have to piecemeal it all together to figure out what is available.

What are the State types?

  • Pass – used to take input and pass to output. Basically, used to debug step functions, so you know what the heck is happening between States.
  • Task – this is where the work gets done. It can be a Lambda function or any other supported service. This will take your input and from there provide output to another State to be consumed
  • Choice – handles branching in your Step Function by looking at the output from the previous State and sending the Step Function to perform the next decided upon State in the workflow.
  • Wait – allows you to put a pause in your Step Function to wait for an action to finish, like deploying CloudFormation or some other long running task.
  • Succeed – the state machine has finished with success. This one is actually simple and does just what it says.
  • Fail – the end state where all has not gone according to plan. This is the opposite of Succeed, and allows for some reporting on what went wrong.
  • Parallel – run States at the same time. I have not yet worked with this, so I do not have as much information as I would like on this type of State

In the example above, I use only 4 of the States from above, Task, Choice, Succeed, and Fail. Mainly because I wanted to touch on a good bit of the basics without going to far down the rabbit hole. For my example, my Task was a Lambda function. To me this is the most logic way to use step functions, but there are probably many more that I have not considered. I also wanted to employ a loop. This being that I wanted to test flow control, and passing variable to and from Lambda functions. I could have strung the Lambda functions together in a straight line, but I wanted to test out branching as well.

The big factor to remember is that the order of the States does not matter. What matters is linking which State comes next. You could use a single Choice State to manage the flow through the entire Step Function, and based upon the input move onto any number of other States or back to a previous one.

[code]
],
“Next”: “ChoiceState”
},
[/code]

This little block of code is a great example. The Next keyword is used, and the following word is the name of the next State block that is going to be executed. The name of each State is identified by the start of the json template block. In the example above, I am not going to a State of type Choice, but have created a State that is called ChoiceState. In retrospect, much of this could have been done cleaner, but it was an example that I put together rather quickly. This is the declaration:

[code]
},
“ChoiceState”: {
“Type”: “Choice”,
“Choices”: [
[/code]

I am going to end here for today. The next section is going to be about passing data from the Step Function to Lambda, and how to reuse it. But, I want to work on a smaller block of code and focus on just the one thing.

Getting Started with AWS Step Functions Part I

AWS came out with Step Functions a few years ago, and up until recently, I have not had the opportunity to dive in and give them a try. Yes, I could build my own pipeline or state machine, but the idea behind Step Functions is that it does most of the heavy lifting for you. That, and it ties into other AWS services. As such, I decided to dive into getting started, and looked at the demo options and walkthroughs that were available. None of them met my needs, so I rolled my own.

The idea is to see how I can create a Step Function that will run multiple loops, and call a Lambda function multiple times. What I wanted to test was the following:

  • Pass Variables into the Step Function and see how they are handled
  • Call a Lambda function multiple times
  • Create a loop using the Step Function DSL
  • Test output from Lambda and make a decision based upon it
  • Figure out any gotchas and how to trigger Step Functions

Let’s dive in. Now, this is the final result. It took me a few iterations to actually get to this point. Smarter people than I might be able to get it done on one go, but not I.

Lambda Code

I came up with a simple Lambda function written in Python 3.6. All that I wanted to do was to perform a loop with Step Functions, and then get output the values. Simple. And as you can see, this code is pretty simple. It could be streamlined, but it was quick and easy to write.

def lambda_handler(event, context):
    print('value1 = ' + event['key1'])
    print('value2 = ' + event['key2'])
    print('value3 = ' + event['key3'])
    taskresult = event.get('taskresult', None)
    if taskresult is None:
        count = 0
    else:
        count = taskresult.get('count', None)
    
    if count is None:
        count = 0
    else:
        count = count +1
        
    if count < 5:
        output = {
            'count' : count,
            'value1' : 'ThereIsNoSpoon';
        }
    else:
        output = { 'value1' : event['key1'],
                    'value2': event['key2'],
                    'count' : count }
    return output

Now we need to move onto the body of what we are working on, and that would be the Step Function. Step Functions have their own language or domain specific language (DSL) that is used to define the state machine. I wanted more than just a “Hello World” example. The idea was to loop through a step functions. Make sure that I could call it multiple times, and then either go to a success or failed state

AWS Step Function Code

{
  "Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function",
  "StartAt": "LambdaFunction",
  "States": {
    "LambdaFunction": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:aws-serverless-repository-hello-w-helloworldpython-1JQ8TEEDUAHCE",
      "ResultPath": "$.taskresult",
      "Retry": [
        {
          "ErrorEquals": ["CustomError"],
          "IntervalSeconds": 1,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.TaskFailed"],
          "IntervalSeconds": 30,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.ALL"],
          "IntervalSeconds": 5,
          "MaxAttempts": 5,
          "BackoffRate": 2.0
        }
      ],
      "Next": "ChoiceState"
    },
    "ChoiceState": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.taskresult.value1",
          "StringEquals": "value1",
          "Next": "SuccessState"
        },
        {
          "Variable": "$.taskresult.count",
          "NumericLessThan": 5,
          "Next": "LambdaFunction"
        }
      ],
      "Default": "FailState"
    },
    "SuccessState": {
      "Type": "Succeed"
    },
    "FailState": {
      "Type": "Fail",
      "Cause": "Invalid response.",
      "Error": "ErrorA"
    }
  }
}

The way that this code works is as follows. Everything works around States. So, you have to move from State to State. This is a key concept when it comes to Step Functions. Now, there are multiple State types, but I am not going to go into that now. The key factor is that you will go through and loops if a proper return value is not returned. Looking at it now, it looks like a bunch of gobbledygook. I am going to have to come back and write up how this works later.

This is what the visual representation looks like when viewed in the AWS Step Function page. There is a defined ‘Start’ and ‘Stop’. The other stages match what was named in the previous section. The code works to present a model that you can follow.

The cool think about AWS Step Functions is that they guarantee a run. And in a situation where you need to ensure that the code is run, and you need a guarantee. This is mostly due to the cost that is associated with it. Running Lambda that Triggers on SQS would be cheaper, but not as easy to ensure.

Back on with our stuff now. We are looking at how we execute the AWS Step Function. Now we need to execute it. Right now, I am not going to go into the logic around passing variables around. Needless, you will need to understand that when writing your own, and I am going to have to revisit it.

Execution. A couple of items to note.

  • Each execution has to have a unique name.
    • Note, this will bite you when you are testing, and think about this when executing it via automation.
  • It takes in an action just like Lambda, via json
  • Making a small change in the inputs can cause madness
{
  "key1": "value1",
  "key2": "value2",
  "key3": "value3"
}

The output will also be in json, and you can see the results in the visual display.

{
  "key1": "value1",
  "key2": "value2",
  "key3": "value3",
  "taskresult": {
    "value1": "value1",
    "value2": "value2",
    "count": 5
  }
}

This is what the output looks like.

I will go into more detail on the breakdown of the Step Function in the next post. There is a lot to be covered, and this just scratches at the surface.