azure多功能成像好用吗_如何使用Azure功能处理高吞吐量消息

azure多功能成像好用吗

Authored with Steef-Jan Wiggers, Azure MVP.

由Azure MVP Steef-Jan Wiggers撰写。

With Microsoft Azure, customers will push all types of workloads to its services. Workloads are ranging from datasets for Machine Learning purposes to a large number of messages for the Service Bus. In any case, Azure like any cloud provider is elastic enough to deal with any size of workload. Scalability and availability are common phrases for cloud-computing. Moreover, you leverage the cloud for it and pay for using it.

借助Microsoft Azure，客户会将所有类型的工作负载推向其服务。 工作负载的范围从用于机器学习的数据集到用于服务总线的大量消息。 无论如何，Azure像任何云提供商一样具有足够的弹性来处理任何规模的工作负载。 可伸缩性和可用性是云计算的常用短语。 此外，您可以利用云并为使用云付费。

消息传递场景 (Messaging scenario)

In this blog post, we will look at a specific messaging scenario. Suppose we have a large number of messages pushed to a service bus topic from a LOB application. Furthermore, multiple listeners are attached to subscriptions on a topic. The number of messages per subscription is 50 to a hundred per second — for the service bus, it is easy to handle. The challenge in this scenario is how to scale an Azure service consuming these messages at the same rate. Would you use functions, a web job or perhaps Service Fabric?

在此博客文章中，我们将研究特定的消息传递场景。 假设我们有大量消息从LOB应用程序推送到服务总线主题。 此外，多个侦听器附加到某个主题的订阅。 每个订阅的消息数量是每秒50到一百个-对于服务总线，它很容易处理。 这种情况下的挑战是如何扩展以相同速率使用这些消息的Azure服务。 您会使用功能，网络作业还是Service Fabric？

For this blog post, we choose a client application generating ~100,000 messages per minute or around 1,666 messages per second. Each message is sent to a topic in Azure with five subscriptions. Each subscription has a corresponding Azure Function consuming the message (service bus topic trigger) and inserting it into a Cosmos DB document collection.

对于此博客文章，我们选择一个客户端应用程序，该应用程序每分钟生成100,000条消息，或每秒大约1,666条消息。 每条消息都发送到具有五个订阅的Azure中的主题。 每个预订都有一个对应的Azure功能，该功能使用消息(服务总线主题触发器)并将其插入Cosmos DB文档集合中。

Azure功能 (Azure Functions)

Azure Functions are part of the Azure Web + Mobile suite of App Services. They are designed to enable the creation of small pieces of meaningful, reusable methods. These methods are easily shared across services. These serverless, event-driven methods are often referred to as “nano-services” due to their small size. Although an Azure Function can contain quite a bit of code, they are typically designed to serve a single purpose and respond to events in connected services.

Azure Functions是App Services的Azure Web +移动套件的一部分。 它们的目的是创建有意义的，可重用的小方法。 这些方法很容易在服务之间共享。 这些无服务器的，事件驱动的方法由于体积小，通常被称为“纳米服务” 。 尽管Azure功能可以包含很多代码，但是它们通常被设计用于单一目的并响应连接服务中的事件。

In our scenario, the functions can respond to messages in a service bus queue or topic (subscription). The challenge for throughput lies with hosting the functions. Functions can run on a consumption plan or app service plan. The latter allows for upscale dimensioning. When running on consumption, you pay for the underlying infrastructure supporting your function in a Function App.

在我们的场景中，这些功能可以响应服务总线队列或主题(订阅)中的消息。 吞吐量面临的挑战在于托管功能。 功能可以在消费计划或应用服务计划上运行。 后者允许进行高档标注。 依靠消耗运行时，您需要为支持功能应用程序中功能的基础架构付费。

We choose to develop a simple function using the topic trigger binding and Cosmos DB binding for output. The function code looks as follows:

我们选择使用主题触发器绑定和Cosmos DB绑定进行输出来开发一个简单的功能。 功能代码如下：

using System;
using System.Threading.Tasks;
public static void Run(string mySbMsg, ILogger log, out object outputDocument)
{
log.LogInformation($"C# ServiceBus topic trigger function processed message: {mySbMsg}");
outputDocument = new
{
mySbMsg
};
}

The incoming message is sent as output (document) to a collection in CosmosDB. The potential limiting factor on the Cosmos DB side is the specified number of Request Units per second (RU/s). When this setting is set too low, throttling will occur, and you would see HTTP 429 messages appear.

传入消息作为输出(文档)发送到CosmosDB中的集合。 Cosmos DB端的潜在限制因素是指定的每秒请求单位数(RU / s) 。 如果此设置设置得太低，将发生限制，并且您会看到HTTP 429消息出现。

Furthermore, the reserved performance is specified regarding Request Units (RU) per second. Hence, each operation in Azure Cosmos DB, including writes, updates, reads, and queries, and updating a document, consumes CPU, memory, and IOPs. By specifying Request Units, you are provided with guaranteed performance and elasticity at any scale. For our setup, we choose 10000 RU/s.

此外，针对每秒的请求单位(RU)指定保留的性能。 因此，Azure Cosmos DB中的每个操作(包括写入，更新，读取和查询以及更新文档)都消耗CPU，内存和IOP。 通过指定请求单位，可以在任何规模上为您提供有保证的性能和弹性。 对于我们的设置，我们选择10000 RU / s。

使用Azure Functions测试设置 (Testing the setup with Azure Functions)

Once we run the test, we notice it takes up to 90 seconds for our message generator to send 100000 messages to a service bus topic in Azure (West-Europe). Hence, we have an outbound stream of over 1000 messages per second. Subsequently, it takes about an additional 180 seconds for five functions to read the corresponding subscriptions and write to the Cosmos DB collection. The documents are about ~ 1Kb in size each.

运行测试后，我们注意到消息生成器最多需要90秒才能将100000条消息发送到Azure(西欧)的服务总线主题。 因此，我们每秒有超过1000条消息的出站流。 随后，五个功能需要花费额外的180秒时间才能读取相应的订阅并写入Cosmos DB集合。 每个文档的大小约为1Kb。

Once we start the test, we see the number of in- and outgoing requests increase to 100 per second and growing during the trial to 500 per second.

一旦开始测试，我们就会看到传入和传出请求的数量增加到每秒100个，而在试用期间增加到每秒500个。

During the trial, we see the subscriptions fill up with messages and subsequently decrease over time to zero. In the end, about 180 seconds after the application sending the messages finishes. This behavior can be observed when running the test for the first time.

在试用期间，我们看到订阅中充满了消息，随后随着时间的流逝减少到零。 最后，在发送消息的应用程序完成后约180秒。 首次运行测试时，可以观察到此行为。

After all the subscriptions are read, and the functions finish processing the in- and outgoing request in the live metrics of the Application Insights drop to zero.

在读取所有订阅之后，这些功能完成了对Application Insights实时指标中的入站和出站请求的处理，将其降至零。

经过几次测试，结果如下： (After several tests the outcome is as follows:)

Message sender sends within 90 seconds 100000 messages to a Service Bus Topic: > 1100 messages/second.

消息发送方在90秒内向服务总线发送100000条消息主题：> 1100条消息/秒。

The five functions consume and process the 100000 messages in 270 seconds: > 350 messages/second. After one test the functions are warmed up. Any test following the first results in a throughput of over 1000 messages/second. This corresponds with the latency Cosmos DB promises when writing messages of around 1 Kb.

这五个功能在270秒内消耗和处理100000条消息：> 350条消息/秒。 经过一项测试后，功能将被预热。 第一次测试之后的任何测试都将导致每秒超过1000条消息的吞吐量。 这与Cosmos DB在编写大约1 Kb的消息时所承诺的等待时间相对应。

Note that when running functions on a consumption plan your app can go to sleep leading to cold start issues. According to a post from fellow MVP Chris ‘O Brien:

请注意，在消费计划上运行功能时，您的应用可能会进入睡眠状态，从而导致冷启动问题。 根据MVP同事Chris'O Brien的帖子：

Currently, the timeout period for Azure Functions is 20 minutes — so if there are periods where your function won’t run, you’ll suffer from this issue.

当前，Azure Functions的超时时间为20分钟-因此，如果在某些时间段内您的函数无法运行，您将遭受此问题的困扰。

This behavior shows when testing the scenario, the first time. After the functions are warm, the throughput triples from ~ 350 to ~ 1150 msg/sec.

第一次测试方案时，将显示此行为。 功能预热后，吞吐量从〜350增至〜1150 msg / sec，增加了两倍。

We re-run the tests with a function in a function app using an app service plan B3 (Standard) and the number of instances set to three. Also, the number of functions and subscriptions were limited to three.

我们使用应用程序服务计划B3(标准)在实例应用程序中使用某个函数重新运行测试，并将实例数设置为三个。 此外，功能和订阅的数量限制为三个。

The test with an App Service Plan using a B3 led to a throughput of ~ 1150 messages/sec each time without any warm-up issues.

使用B3的应用服务计划进行的测试导致每次〜1150消息/秒的吞吐量，而没有任何预热问题。

The graphs above show the number of in- and outgoing requests are around the measured throughput.

上图显示了传入和传出请求的数量在测得的吞吐量附近。

You could further experiment using a premium or isolated App Service Plan with more resources.

您可以使用具有更多资源的高级或隔离应用服务计划进一步进行试验。

Azure Web作业 (Azure WebJobs)

A useful means to automate tasks in the cloud is by leveraging Azure WebJobs hosted on Azure App Service. With the App Service, you can continuously read messages from a service bus topic (subscription) or queue without running into warm-up issues. For our scenario, Azure Function on a consumption plan is enough, regardless of the warm-up problems.

通过利用Azure应用服务上托管的Azure WebJob，在云中自动化任务的一种有用方法。 使用App Service，您可以连续地从服务总线主题(订阅)读取消息或在队列中排队而不会遇到热身问题。 对于我们的方案，无论预热问题如何，使用消耗计划上的Azure功能就足够了。

服务面料 (Service Fabric)

With Service Fabric you build distributed applications at scale leveraging the Azure infrastructure. The service is an open source project and powers core Azure services such as Azure Event Hubs, Cosmos DB, and Dynamics 365. You can use Service Fabric to develop services that auto-scale based on needs and thus any required throughput. Therefore, when you expect a performance unable to be achieved with Web Jobs or Azure Functions you can opt to go for a Service Fabric implementation. For our scenario, Azure Functions is sufficient to meet the 50 to 100 messages per second throughput.

使用Service Fabric，您可以利用Azure基础结构大规模构建分布式应用程序。 该服务是一个开源项目，并为诸如Azure Event Hub，Cosmos DB和Dynamics 365之类的核心Azure服务提供支持。您可以使用Service Fabric开发可根据需要(从而根据所需的吞吐量)自动扩展的服务。 因此，当您期望使用Web Jobs或Azure Functions无法实现性能时，可以选择采用Service Fabric实现。 对于我们的方案，Azure Functions足以满足每秒50到100条消息的吞吐量。

结语 (Wrap up)

In this blog post, we looked at a scenario of achieving a given throughput of messages from a topic to Cosmos DB. With a function, leveraging the Service bus and Cosmos DB binding, we can easily consume over 300 messages per second and insert the messages into a Cosmos DB collection. In case we exclude the warm-up issues, the functions can efficiently process over 1000 messages per second.

在此博客文章中，我们研究了实现从主题到Cosmos DB的给定消息吞吐量的方案。 通过使用服务总线和Cosmos DB绑定的功能，我们可以轻松地每秒消耗300条以上的消息，并将消息插入Cosmos DB集合中。 如果我们排除了预热问题，这些功能每秒可以有效处理1000条以上的消息。

Hence, we can conclude Azure Functions are a good option for handling around 1000 messages per second with the appropriate service plan in place.

因此，我们可以得出结论，使用适当的服务计划，Azure Functions是每秒处理大约1000条消息的一个不错的选择。

Note that this blog describes a small experiment and more options are available to process a high volume of messages in Azure.

请注意，此博客描述了一个小型实验，并且提供了更多选项来处理Azure中的大量消息。

管理与监控 (Management and Monitoring)

One may need a deeper insight into the Azure Serverless entities to leverage it. With the third party tool Serverless360, you can manage your composite cloud-native solution at one place. The tool monitors your Azure integration services like Logic Apps, Functions, Event Hubs, Service Bus, and API endpoints. Furthermore, you can:

可能需要更深入地了解Azure无服务器实体才能利用它。 使用第三方工具Serverless360 ，您可以在一个地方管理您的复合云原生解决方案。 该工具监视您的Azure集成服务，例如逻辑应用，功能，事件中心，服务总线和API终结点。 此外，您可以：

• In your service bus queues or topics, access active messages to know more details, process the dead letter messages to repair, resubmit or merely purge them.
  在服务总线队列或主题中，访问活动消息以了解更多详细信息，处理死信以进行修复，重新提交或仅清除它们。
• Detect and be alerted about violations occurring in your composite integration solutions.
  检测并警告您的复合集成解决方案中发生的违规情况。
• Integrate your Azure serverless monitoring with essential notification tools like PagerDuty, Microsoft Teams, ServiceNow, Slack, SMTP, and OMS.
  将您的Azure无服务器监视与基本通知工具(如PagerDuty，Microsoft Teams，ServiceNow，Slack，SMTP和OMS)集成在一起。
• Have full control over what your colleagues or consultants can see and do with the Azure resources in your environment.
  完全控制您的同事或顾问可以在您的环境中看到和使用Azure资源的内容。
• Governance and audit report provide detailed information on the four W’s — WHO accessed WHAT, WHEN, and WHY. Serverless360 collects, consolidates, and enables search filters on your account logs.
  治理和审计报告提供了有关四个W的详细信息-世卫组织访问了WHAT，WHEN和WHY。 Serverless360收集，合并并启用对帐户日志的搜索过滤器。

Originally published at www.serverless360.com on November 27, 2018.

最初于2018年11月27日发布在www.serverless360.com上。

翻译自: https://www.freecodecamp.org/news/how-to-use-azure-functions-to-process-high-throughput-messages-996d05d4ab23/

azure多功能成像好用吗