Glen Knight

CloudFormation Update – CLI + Third-Party Resource Support + Registry

November 18, 2019 by knightglen_sruobz

CloudFormation was launched in 2011 (AWS CloudFormation – Create Your AWS Stack From a Recipe) and has become an indispensable tool for many AWS customers. They love the fact that they can define a template once and then use it to reliably provision their AWS resources. They also make frequent use of Change Sets, and count on them to provide insights into the actions (additions, changes, and deletions) that will take place when the change set is executed.

As I have written about in the past, CloudFormation takes special care to implement a model that is consistent, stable, and uniform. This is an important, yet often overlooked, element of CloudFormation, but one that our customers tell us that they highly value!

Let’s take a look at a couple of the most frequent requests from our customers:

Performance – Over the past year, the number of operations performed on CloudFormation stacks has grown by 30% per quarter! The development team has worked non-stop to make CloudFormation faster and more efficient, even as usage grows like a weed, using a combination of architectural improvements and low-level optimizations. Over the past couple of months, this work has allowed us to raise a number of soft and hard limits associated with CloudFormation, and drove a significant reduction in average and maximum latency for Create and Update operations.

Coverage – We release new services and new features very rapidly, and sometimes without CloudFormation support. Our goal is to support new services and new features as quickly as possible, and I believe that we are making progress. We are also using the new CloudFormation Coverage Roadmap as a primary source of input to our development process, and have already addressed 43 of the issues.

Extensibility – Customers who make extensive use of CloudFormation tell us that they want to automate the creation of non-AWS resources. This includes resources created by their own development teams and by third-party suppliers of SaaS applications, monitoring tools, and so forth. They are already making good use of Custom Resources, but as always want even more control and power, and a simple way to manage them.

CloudFormation Registry and CloudFormation CLI
Today we are addressing your requests for more coverage and better extensibility with the launch of the CloudFormation CLI as an open source project.

You can use this kit to define and create resource providers that automate the creation of resources in a safe & systematic way. You create a schema, define a handler for five core operations, test it locally, and then publish your provider to a new provider registry that is associated with your AWS account.

We have also been working with a select set of third-party vendors, helping them to create resource providers for their SaaS applications, monitoring tools, and so forth. You will be able to obtain the providers from the vendors of interest and add them to your provider registry.

Finally, we are making a set of AWS resource providers available in open source form. You can use them to learn how to write a robust provider, and you can also extend them (in your own namespace), as desired.

Let’s dive in!

CloudFormation CLI
This set of tools gives you everything you need to build your own resource providers, including detailed documentation and sample code. The cfn (CloudFormation Command Line Interface) command helps you to initialize your project, generate skeleton code, test your provider, and register it with CloudFormation.

Here are the principal steps:

Model – Create and validate a schema that serves as the canonical description of your resource.

Develop – Write a handler (Java and Go now, with other languages to follow) that defines five core operations (Create, Read, Update, Delete, and List) on your resource, and test it locally.

Register – Register the provider with CloudFormation so that it can be used in your CloudFormation templates.

Modeling a Resource
The schema for a resource must conform to the Resource Provider Definition Schema. It defines the resource, its properties, and its attributes. The properties can be defined as read-only, write-only, and create-only; this provides CloudFormation with the information it needs to have in order to be able to modify existing resources when it is executing an operation on a stack. Here is a simple definition:

{
  "additionalProperties": false,
  "createOnlyProperties": [
    "/properties/Name"
  ],
  "primaryIdentifier": [
    "/properties/Name"
  ],
  "properties": {
    "Name": {
      "description": "The name of the configuration set.",
      "maxLength": 64,
      "pattern": "^[a-zA-Z0-9_-]{0,64}$",
      "type": "string"
    }
  },
  "typeName": "AWS::SES::ConfigurationSet",
  "description": "A sample resource"
}

Develop
The handlers make use of a framework that takes care of error handling, throttling of calls to downstream APIs, credential management, and so forth. The CloudFormation CLI contains complete sample code; you can also study the Amazon SES Resource Provider (or any of the others) to learn more.

To learn more, read Walkthrough: Develop a Resource Provider in the CloudFormation CLI documentation.

Register
After you have developed and locally tested your resource provider, you need to tell CloudFormation about it. Using the CloudFormation CLI, you submit the package (schema and compiled handlers) to the desired AWS region(s). The acceptance process is asynchronous; once it completes you can use the new resource type in your CloudFormation templates.

Cloudformation Registry
The CloudFormation registry provides per-account, per-region storage for your resource providers. You can access it from the CloudFormation Console:

Select Public to view the native AWS resources (AWS::*::*); select Private to view resources that you create, and those that you obtain from third parties.

You can also access the registry programmatically using the RegisterType, DeregisterType, ListTypes, ListTypeRegistrations, ListTypeVersions, and DescribeType functions.

Third-Party Support
As I mentioned earlier, a select set of third-party vendors have been working to create resource providers ahead of today’s launch. Here’s the initial list:

• Atlassian – DevOps Just Got a Whole Lot Easier with Opsgenie and AWS CloudFormation Registry and CLI.
• Datadog – Implement monitoring as code with Datadog and CloudFormation Registry.
• Densify – How to Adopt Continuous Optimization in AWS Using CloudFormation.
• Dynatrace – Simplify OneAgent installation with AWS CloudFormation Registry.
• Fortinet – –Fortinet Now Integrates with AWS CloudFormation Registry and CLI to Enhance Cloud Security.
• New Relic – Create New Relic Alerts in AWS CloudFormation Templates.
• Spotinst – AWS Cloudformation Custom Resource No Longer Required for Spotinst.

After registering the provider from a vendor, you will be able to reference the corresponding resource types in your CloudFormation templates. For example, you can use Datadog::Monitors::Monitor to create a Datadog monitor.

If you are a third-party vendor and are interested in creating a resource provider for your product, send an email to cfn-registry-onboarding@amazon.com.

Available Now
You can use the CloudFormation CLI to build resource providers for use in all public AWS regions.

— Jeff;

Source: AWS News