目前客户使用越来越多云上的服务来搭建事件型驱动应用,但需要了解它们之间的通讯技术,给开发人员带来额外的不便
EventBridge Pipes是点对点集成的解决方案,让生产者和消息者进行通讯,不再需要写额外的胶水代码
这个pipe包括源端和目标端,中间有可选的过滤和enrich功能,它不仅简单、省钱、而且可靠,有失败重试功能
使用EventBridge Pipes
有以下步骤:
EventBridge Pipes
减少了学习服务的成本、避免写服务之间的集成代码,并保证了可靠性和一致性
接下来我们看一个例子,使用EventBridge Pipes
将SQS与Step Functions来集成
下面图是创建好的SQS和Step Functions,我们需要将SQS队列中的事件传入到Step Functions来处理:
进入 Amazon EventBridge Pipes页面,选择 Create pipe:
在这个页面,可以配置一个pipe并设置IAM权限,进入 Pipe settings 页面:
在Permissions
部分,为这个pipe创建一个新的IAM role或使用已有的role。为了改善开发者体验,EventBridge Pipes
会计算出所需的最小权限,不需要手工添加。这里选择创建新的IAM Role:
接下来进入Build pipe
部分,在这里可以看到EventBridge Pipes
支持的事件源:
Then, I go back to the Build pipe section. On this page, I can see the available event sources supported by EventBridge Pipes.
选择SQS,如果需要做批处理,可以在Additional settings
里定义批的大小和窗口。
进入下一步,进行事件过滤,这一步是可选的,如果事件被过滤了则不会收费:
例如只处理RETURN_PURCHASE
大于100的事件:
{
"event-type":"RETURN_PURCHASE",
"value":100
}
在event pattern部分定义以下内容:
{
"event-type": ["RETURN_PURCHASE"],
"value": [{
"numeric": [">=", 100]
}]
}
可以点击test pattern
进行测试,进行验证:
在 Enrichment 部分,可以使用lambda或三方API对事件进行加工,也可以使用自带的 Enrichment Input Transformer进行加工:
最后一步是定义target
来处理接收到的事件:
这里选择AWS支持的目标端:
I select my existing AWS Step Functions state machine, named pipes-statemachine
.
In addition, I can also use Target Input Transformer by referring to the Transforming Amazon EventBridge target input
documentation. For my case, I need to define a high priority for events going into this target. To do that, I define a sample custom event in Sample events/Event Payload and add the priority
: HIGH
in the Transformer section. Then in the Output section, I can see the final event to be passed to the target destination service. Then, I select Create pipe.
In less than a minute, my pipe was successfully created.
To test this pipe, I need to put an event into the Amaon SQS queue.
To check if my event is successfully processed by Step Functions, I can look into my state machine in Step Functions. On this page, I see my event is successfully processed.
I can also go to Amazon CloudWatch Logs to get more detailed logs.
Things to Know Event Sources – At launch, Amazon EventBridge Pipes supports the following services as event sources: Amazon DynamoDB, Amazon Kinesis, Amazon Managed Streaming for Apache Kafka (Amazon MSK) alongside self-managed Apache Kafka, Amazon SQS (standard and FIFO), and Amazon MQ (both for ActiveMQ and RabbitMQ).
Event Targets – Amazon EventBridge Pipes supports 15 Amazon EventBridge targets, including AWS Lambda, Amazon API Gateway, Amazon SNS, Amazon SQS, and AWS Step Functions. To deliver events to any HTTPS endpoint, developers can use API destinations as the target.
Event Ordering – EventBridge Pipes maintains the ordering of events received from an event sources that support ordering when sending those events to a destination service.
Programmatic Access – You can also interact with Amazon EventBridge Pipes and create a pipe using AWS Command Line Interface (CLI) , AWS CloudFormation , and AWS Cloud Development Kit (AWS CDK) .
Independent Usage – EventBridge Pipes can be used separately from Amazon EventBridge bus and Amazon EventBridge Scheduler. This flexibility helps developers to define source events from supported AWS and self-managed services as event sources without Amazon EventBridge Event Bus.
Availability – Amazon EventBridge Pipes is now generally available in all AWS commercial Regions, with the exception of Asia Pacific (Hyderabad) and Europe (Zurich).
Visit the Amazon EventBridge Pipes page to learn more about this feature and understand the pricing. You can also visit the documentation page to learn more about how to get started.
Happy building!