New – Amazon EventBridge Schema Registry is Now Generally Available
Amazon EventBridge is a serverless event bus that makes it easy to connect applications together. It can use data from AWS services, your own applications, and integrations with Software-as-a-Service (SaaS) partners. Last year at re:Invent, we introduced in preview EventBridge schema registry and discovery, a way to store the structure of the events (the schema) in a central location, and simplify using events in your code by generating the code to process them for Java, Python, and Typescript.
Today, I am happy to announce that the EventBridge schema registry is generally available, and that we added support for resource policies. Resource policies allow to share a schema repository across different AWS accounts and organizations. In this way, developers on different teams can search for and use any schema that another team has added to the shared registry.
Using EventBridge Schema Registry Resource Policies
It’s common for companies to have different development teams working on different services. To make a more concrete example, let’s take two teams working on services that have to communicate with each other:
- The CreateAccount development team, working on a frontend API that receives requests from a web/mobile client to create a new customer account for the company.
- the FraudCheck development team, working on a backend service checking the data for newly created accounts to estimate the risk that those are fake.
Each team is using their own AWS account to develop their application. Using EventBridge, we can implement the following architecture:
- The frontend CreateAccount applications is using the Amazon API Gateway to process the request using a AWS Lambda function written in Python. When a new account is created, the Lambda function publishes the
ACCOUNT_CREATEDevent on a custom event bus. - The backend FraudCheck Lambda function is built in Java, and is expecting to receive the
ACCOUNT_CREATEDevent to call Amazon Fraud Detector (a fully managed service we introduced in preview at re:Invent) to estimate the risk of that being a fake account. If the risk is above a certain threshold, the Lambda function takes preemptive actions. For example, it can flag the account as fake on a database, or post aFAKE_ACCOUNTevent on the event bus.
How can the two teams coordinate their work so that they both know the syntax of the events, and use EventBridge to generate the code to process those events?
First, a custom event bus is created with permissions to access within the company organization.
Then, the CreateAccount team uses EventBridge schema discovery to automatically populate the schema for the ACCOUNT_CREATED event that their service is publishing. This event contains all the information of the account that has just been created.
In an event-driven architecture, services can subscribe to specific types of events that they’re interested in. To receive ACCOUNT_CREATED events, a rule is created on the event bus to send those events to the FraudCheck function.
Using resource policies, the CreateAccount team gives read-only access to the FraudCheck team AWS account to the discovered schemas. The Principal in this policy is the AWS account getting the permissions. The Resource is the schema registry that is being shared.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GiveSchemaAccess",
"Effect": "Allow",
"Action": [
"schemas:ListSchemas",
"schemas:SearchSchemas",
"schemas:DescribeSchema",
"schemas:DescribeCodeBinding",
"schemas:GetCodeBindingSource",
"schemas:PutCodeBinding"
],
"Principal": {
"AWS": "123412341234"
},
"Resource": [
"arn:aws:schemas:us-east-1:432143214321:schema/discovered-schemas",
"arn:aws:schemas:us-east-1:432143214321:schema/discovered-schemas*"
]
}
]
}
Now, the FraudCheck team can search the content of the discovered schema for the ACCOUNT_CREATED event. Resource policies allow you to make a registry available across accounts and organizations, but they will not automatically show up in the console. To access the shared registry, the FraudCheck team needs to use the AWS Command Line Interface (CLI) and specify the full ARN of the registry:
aws schemas search-schemas
--registry-name arn:aws:schemas:us-east-1:432143214321:registry/discovered-schemas
--keywords ACCOUNT_CREATEDIn this way, the FraudCheck team gets the exact name of the schema created by the CreateAccount team.
{
"Schemas": [
{
"RegistryName": "discovered-schemas",
"SchemaArn": "arn:aws:schemas:us-east-1:432143214321:schema/discovered-schemas/CreateAccount@ACCOUNT_CREATED",
"SchemaName": “CreateAccount@ACCOUNT_CREATED",
"SchemaVersions": [
{
"CreatedDate": "2020-04-28T11:10:15+00:00",
"SchemaVersion": 1
}
]
}
]
}With the schema name, the FraudCheck team can describe the content of the schema:
aws schemas describe-schema
--registry-name arn:aws:schemas:us-east-1:432143214321:registry/discovered-schemas
--schema-name CreateAccount@ACCOUNT_CREATEDThe result describes the schema using the OpenAPI specification:
{
"Content": "{"openapi":"3.0.0","info":{"version":"1.0.0","title":"CREATE_ACCOUNT"},"paths":{},"components":{"schemas":{"AWSEvent":{"type":"object","required":["detail-type","resources","detail","id","source","time","region","version","account"],"x-amazon-events-detail-type":"CREATE_ACCOUNT","x-amazon-events-source":”CreateAccount","properties":{"detail":{"$ref":"#/components/schemas/CREATE_ACCOUNT"},"account":{"type":"string"},"detail-type":{"type":"string"},"id":{"type":"string"},"region":{"type":"string"},"resources":{"type":"array","items":{"type":"object"}},"source":{"type":"string"},"time":{"type":"string","format":"date-time"},"version":{"type":"string"}}},"CREATE_ACCOUNT":{"type":"object","required":["firstName","surname","id","email"],"properties":{"email":{"type":"string"},"firstName":{"type":"string"},"id":{"type":"string"},"surname":{"type":"string"}}}}}}",
"LastModified": "2020-04-28T11:10:15+00:00",
"SchemaArn": "arn:aws:schemas:us-east-1:432143214321:schema/discovered-schemas/CreateAccount@CREATE_ACCOUNT",
"SchemaName": “CreateAccount@ACCOUNT_CREATED",
"SchemaVersion": "1",
"Tags": {},
"Type": "OpenApi3",
"VersionCreatedDate": "2020-04-28T11:10:15+00:00"
}Using the AWS Command Line Interface (CLI), the FraudCheck team can create a code binding if it isn’t already created, using the put-code-binding command, and then download the code binding to process that event:
aws schemas get-code-binding-source
--registry-name arn:aws:schemas:us-east-1:432143214321:registry/discovered-schemas
--schema-name CreateAccount@ACCOUNT_CREATED
--language Java8 CreateAccount.zipAnother option for the FraudCheck team is to copy and paste (after unescaping the JSON string) the Content of the discovered schema to create a new custom schema in their AWS account.
Once the schema is copied to their own account, the FraudCheck team can use the AWS Toolkit IDE plugins to view the schema, download code bindings, and generate serverless applications directly from their IDEs. The EventBridge team is working to add the capability to the AWS Toolkit to use a schema registry in a different account, making this step simpler. Stay tuned!
Often customers have a specific team, with a different AWS account, managing the event bus. For the sake of simplicity, in this post I assumed that the CreateAccount team was the one configuring the EventBridge event bus. With more accounts, you can simplify permissions using IAM to share resources with groups of AWS accounts in AWS Organizations.
Available Now
The EventBridge Schema Registry is available now in all commercial regions except Bahrain, Cape Town, Milan, Osaka, Beijing, and Ningxia. For more information on how to use resource policies for schema registries, please see the documentation.
Using Schema Registry resource policies, it is much easier to coordinate the work of different teams sharing information in an event-driven architecture.
Let me know what are you going to build with this!
— Danilo
Source: AWS News
