By using AWS re:Post, you agree to the Terms of Use

Welcome to AWS re:Post

re:Post gives you access to a vibrant community that helps you become even more successful on AWS

Learn AWS faster by following popular topics

see all
1/18

Recent questions

see all
1/18

Scaling AWS step-functions and comprehend jobs with Concurrent active asynchronous jobs quota

1. I am trying to implement a solution that integrates aws comprehend targeted sentiment along with step functions. And then make it public for people to use it as an api. 2. I need to wait until the job is complete before being able to move forward with the workflow. Since the comprehend job is asynchronous, **I created a wait time poller to periodically check the jobs status** using describe_targeted_sentiment_detection_job. Following a similar integration pattern as this https://docs.aws.amazon.com/step-functions/latest/dg/sample-project-job-poller.html. 3. However, there is seems to be a **Concurrent active asynchronous jobs quota of 10 jobs** according to https://docs.aws.amazon.com/comprehend/latest/dg/guidelines-and-limits.html#limits-active-jobs. If this is the case, I was thinking of **creating another poll to check if comprehend is free to do targeted sentiment before starting another comprehend job** 4. Given that the step functions charge for each polling cycle. And that there is a concurrent job limit of 10. I am worried about the backlog and respective costs that may be created if many step-function executions were to be started. For example, if 1000 workflows are started. Workflow number 1000 will have to be polling for an available comprehend job for a long time. Does anyone know if a solution is available to get around the concurrent active asynchronous jobs quota or to reduce the cost of step functions continually polling for a long time?
0
answers
0
votes
8
views
asked 5 hours ago

Simple IoT Core thing rule and SiteWise property ingestion config

Hi, I am seeking some basic plain english assistance please. Any assistance will be great appreciated. I have a temperature sensor Thing receiving MQTT data. * The Things name is: TestThing * The data includes 4 fields: "TS", "datetime", "class", "data" * The published Topic is: TestTopic * The IoT Core MQTT Test Client payload displays as: {"TS": 1656941515, "datetime": 04/07/2022 23:31:55, "class": Temperature, "data": 11} I do not know how to fill out the IoT Core rule correctly, and I am also not sure if the SiteWise Asset propertyAlias is correct. Based on what i have read (see below), the latest failure is: **IoT Core Rule config UI** * SQL Statement: SELECT * FROM 'TestTopic' * Property Alias: TestTopic * Time in seconds: ${TS} * Data type: DOUBLE * Value: ${data} * IAM role: created via the UI - create new role I have used Data type: DOUBLE although i note the incoming data is seen within IoT Analytics as integer. The SiteWise model Measurement definitions is also set to DOUBLE **SiteWise config** * Model Measurement definitions: Temperature * Asset Measurement: Temperature * Asset Measurement - Temperature field (enter a property alias): TestTopic I believe i have created the model and asset correctly. Although, I am not sure if i understand the what exactly is the Property, and thus the propertyAlias. based on the above, I believe i created the propertyAlias for the Temperature field as: TestTopic **Setup and read history** The setup is super simple for testing. A single IoT thing, pulling in MQTT topic data. The Thing was created as a single Thing with certificates. A MODBUS sensor connected to a gateway with MQTT. No OPC server, no Greengrass, no Lora. IoT Analytics and AWS QuickSight services can access the data. I have read: 1) all the AWS suggested questions, 2) the manual - docs pages on SiteWise IoT Core data ingestion, including the tutorial section, 3) the AWS workshop pages, 4) watch every wonderful Youtube on the topic (not many). Key articles read: https://docs.aws.amazon.com/iot-sitewise/latest/userguide/ingest-data-from-iot-things.html https://docs.aws.amazon.com/iot/latest/developerguide/iotsitewise-rule-action.html https://docs.aws.amazon.com/iot-sitewise/latest/userguide/connect-data-streams.html https://docs.aws.amazon.com/iot-sitewise/latest/userguide/iot-rules.html https://iot-sitewise.workshop.aws/en/40_aws-iot-sitewise-data-ingestion.html https://repost.aws/tags/TAGaSyCvg-SI2w6FYqm1H2RQ/aws-io-t-site-wise
0
answers
0
votes
4
views
asked 9 hours ago