An Introduction to AWS Step Functions – Part 2

In Part 1, I went through a whirlwind tour and left example code on how to create a Lambda function and a Step Function. The thing is, I did not explain how the code works. That is what this article is about. I am going to be using the same code from the previous post, but will be going through the code and explaining it. We shall see how well the formatting works.

Jumping right into it. This is the entire code for the StepFunction from the previous post. As of right now, this code has to be json. I did not find any information about supporting yaml. Here is the documentation provided by AWS. It is pretty thorough, but it could be a bit clearer. My explanation could be better or worse.

{
  "Comment": "A Retry example of the Amazon States Language using an AWS Lambda Function",
  "StartAt": "LambdaFunction",
  "States": {
    "LambdaFunction": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:aws-serverless-repository-hello-w-helloworldpython-1JQ8TEEDUAHCE",
      "ResultPath": "$.taskresult",
      "Retry": [
        {
          "ErrorEquals": ["CustomError"],
          "IntervalSeconds": 1,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.TaskFailed"],
          "IntervalSeconds": 30,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        },
        {
          "ErrorEquals": ["States.ALL"],
          "IntervalSeconds": 5,
          "MaxAttempts": 5,
          "BackoffRate": 2.0
        }
      ],
      "Next": "ChoiceState"
    },
    "ChoiceState": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.taskresult.value1",
          "StringEquals": "value1",
          "Next": "SuccessState"
        },
        {
          "Variable": "$.taskresult.count",
          "NumericLessThan": 5,
          "Next": "LambdaFunction"
        }
      ],
      "Default": "FailState"
    },
    "SuccessState": {
      "Type": "Succeed"
    },
    "FailState": {
      "Type": "Fail",
      "Cause": "Invalid response.",
      "Error": "ErrorA"
    }
  }
}

Looking at this for the first time, it all seems a bit overwhelming. However, after breaking it down, you will see that while you have to be precise with it, Step Functions are fairly easy to read. (Writing them is still a PITA, but that is just what it is) When you get down to it, the Step Function configuration file only contains 3 sections: ‘Comment‘, ‘StartsAt‘, and ‘States‘. That is it.

Let us take a quick look at the the first two mentioned above, Comment and StartsAt. The former of these is just a comment so you know what the purpose of the Step Function is. On the other hand, you could use this for whatever comment you like. Then, there is StartsAt. StartsAt, actually links to a State. And the States section is what contains all the real meat of the Step Function. The value that is used for StartsAt is the name of one of the States.

States

States are where everything really happens when it comes to Step Functions. They are the individual units that when strung together, make the magic happen. When you look at it from this perspective, it sounds ridiculously easy. But, just like everything else, the devil is in the details. The key to understanding States is to know what the various components that make up a State, and the type of states that are available.

Since this is an introduction, and based off of my code here, I am not going to go through all the options. However, there are docs from Amazon on it. The think about it is, it is not simple. Based on the docs, the only required field is Type. Great, but what are the various types? How do I link them together to make a cohesive whole? What are these other options and how do I pass variables from one section to the next and use them over and over again in a loop? This information is available, but it is scatter all over the documentation and you have to piecemeal it all together to figure out what is available.

What are the State types?

• Pass – used to take input and pass to output. Basically, used to debug step functions, so you know what the heck is happening between States.
• Task – this is where the work gets done. It can be a Lambda function or any other supported service. This will take your input and from there provide output to another State to be consumed
• Choice – handles branching in your Step Function by looking at the output from the previous State and sending the Step Function to perform the next decided upon State in the workflow.
• Wait – allows you to put a pause in your Step Function to wait for an action to finish, like deploying CloudFormation or some other long running task.
• Succeed – the state machine has finished with success. This one is actually simple and does just what it says.
• Fail – the end state where all has not gone according to plan. This is the opposite of Succeed, and allows for some reporting on what went wrong.
• Parallel – run States at the same time. I have not yet worked with this, so I do not have as much information as I would like on this type of State

In the example above, I use only 4 of the States from above, Task, Choice, Succeed, and Fail. Mainly because I wanted to touch on a good bit of the basics without going to far down the rabbit hole. For my example, my Task was a Lambda function. To me this is the most logic way to use step functions, but there are probably many more that I have not considered. I also wanted to employ a loop. This being that I wanted to test flow control, and passing variable to and from Lambda functions. I could have strung the Lambda functions together in a straight line, but I wanted to test out branching as well.

The big factor to remember is that the order of the States does not matter. What matters is linking which State comes next. You could use a single Choice State to manage the flow through the entire Step Function, and based upon the input move onto any number of other States or back to a previous one.

[code]  
 ],  
  “Next”: “ChoiceState”  
 },  
[/code]

This little block of code is a great example. The Next keyword is used, and the following word is the name of the next State block that is going to be executed. The name of each State is identified by the start of the json template block. In the example above, I am not going to a State of type Choice, but have created a State that is called ChoiceState. In retrospect, much of this could have been done cleaner, but it was an example that I put together rather quickly. This is the declaration:

[code]  
 },  
 “ChoiceState”: {  
 “Type”: “Choice”,  
 “Choices”: [  
[/code]

I am going to end here for today. The next section is going to be about passing data from the Step Function to Lambda, and how to reuse it. But, I want to work on a smaller block of code and focus on just the one thing.