Send message to AWS SQS Queue and Poll for Messages using WSO2 MI 4.2.0

In this article I’m going to discuss how we can use the WSO2 MI to,

• Create an AWS SQS Queue
• Send the message to the Amazon SQS queue
• Poll for messages received at the Amazon SQS queue using AmazonSQS Inbound Endpoint

This is a combination of the examples discussed in AmazonSQS Connector Example and AmazonSQS Inbound Endpoint Example.

When developing the project, I’m going to use the Visual Studio Code with the Micro Integrator extension (Not the integration studio).

Prerequisites

Follow the steps mentioned in the Setting up Amazon SQS document to create an Amazon account and obtain access key ID and secret access key. Keep them saved to use in the next steps.

Creating the Project

Since I have already installed the VS code extension, I will create a new project for this use case. Follow the steps in create integration project guide to set up the integration project.

Create a Queue in the Amazon SQS

Navigate to MI Project Explorer > Local Entries > Connections and click on the + sign next to Connections to open the Add New Connection view. Search for sqs and select Amazonsqs option.

Then provide the connection details. Here I have provided the Connection Name as AMAZON_SQS_CONNECTION . Also provide the relevant Region and the Authentication Properties.

Sequence to buildMessage

Navigate to MI Project Explorer > Sequences and click the + sign next to Sequences to open the Create New Sequence form.

Provide the Sequence name as buildMessage and click Create.

<?xml version="1.0" encoding="UTF-8"?>
<sequence name="buildMessage" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
    <property expression="json-eval($.companyName)" name="companyName" scope="default" type="STRING"/>
    <payloadFactory media-type="xml">
        <format>
            <m0:getQuote xmlns:m0="http://services.samples">
                <m0:request>
                    <m0:symbol>$1</m0:symbol>
                </m0:request>
            </m0:getQuote>
        </format>
        <args>
            <arg evaluator="xml" expression="$ctx:companyName"/>
        </args>
    </payloadFactory>
    <header name="Action" scope="default" value="urn:getQuote"/>
    <enrich>
        <source clone="true" type="body"/>
        <target property="target_property" type="property"/>
    </enrich>
</sequence>

Sequence to createQueue

Create the createQueue sequence as shown below. In this sequence, we create a queue in the Amazon SQS instance.

<?xml version="1.0" encoding="UTF-8"?>
<sequence name="createQueue" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
    <log category="INFO" level="custom">
        <property name="queueName" expression="$ctx:queueName" />
    </log>
    <amazonsqs.createQueue configKey="AMAZON_SQS_CONNECTION">
        <queueName>{$ctx:queueName}</queueName>
    </amazonsqs.createQueue>
    <log level="custom">
        <property expression="json-eval($)" name="queueURL"/>
    </log>
    <property expression="json-eval($.CreateQueueResponse.CreateQueueResult.QueueUrl)" name="queueURL" scope="default" type="STRING"/>
    <log level="custom">
        <property expression="$ctx:queueURL" name="queueURL"/>
    </log>
</sequence>

Sequence to sendMessage

Create sendMessage sequence as shown below. In this sequence, we send the message to the Amazon SQS Queue.

API to Invoke the Resource

Navigate to MI Project Explorer > APIs and click on the + sign next to APIs to open the Synapse API Artifact creation form.

Provide the API name as SQSAPI and the API context as /sqs and click Create.

You can go to the source view of the XML configuration file of the API and copy the following configuration.

<?xml version="1.0" encoding="UTF-8"?>
<api context="/sqs" name="SQSAPI" xmlns="http://ws.apache.org/ns/synapse">
    <resource methods="POST" uri-template="/sendToQueue">
        <inSequence>
            <property expression="json-eval($.queueName)" name="queueName" scope="default" type="STRING" />
            <sequence key="buildMessage" />
            <sequence key="createQueue" />
            <sequence key="sendMessage" />
            <respond />
        </inSequence>
        <faultSequence/>
    </resource>
</api>

Add Dependencies to MI

Add the dependencies listed in here to the <MI-HOME>/lib directory.

Build and deploy the carbon app

Refer to the documentation on how to build and export the Carbon Application.

Test the Scenario to Create and Send Messages to Queue

Start the MI server and invoke the below request.

curl — location ‘http://localhost:8290/sqs/ sendToQueue’ \ — header ‘Content-Type: application/json’ \ — data ‘{ “companyName”:”WSO2", “queueName”:”SQSTest” }’

You will be able to observe the below response in postman.

In the logs, you can observe below.

[2024-10-21 23:01:39,869]  INFO {LogMediator} - {api:SQSAPI} queueName = SQSTest
[2024-10-21 23:01:41,092]  INFO {LogMediator} - {api:SQSAPI} queueURL = {"CreateQueueResponse":{"CreateQueueResult":{"QueueUrl":"https://sqs.us-east-1.amazonaws.com/324037293231/SQSTest"},"ResponseMetadata":{"RequestId":"75882047-a77d-5d78-8819-7fd7354059ba"}}}
[2024-10-21 23:01:41,095]  INFO {LogMediator} - {api:SQSAPI} queueURL = https://sqs.us-east-1.amazonaws.com/324037293231/SQSTest

And in SQS console, you can see the queue is created and event is received.

Since the first part of creating and sending the messages to the queue is successful, let’s move on to the next part of the use case.

Creating Inbound Endpoint

Let’s first create a new inbound endpoint. Click on + in Inbound Endpoints to create a new inbound endpoint. Select the creation Type as AmazonSQS

Then it will open the below window. Fill the Inbound EP details. Provide the Access Key and the Secret Key as well. After providing the details, you can click on create option.

Then it will create the relevant resources, and you can use the Show Source option to view the configurations provided.

Sequence to process the message

In this example, for simplicity we will just log the message. So let’s modify the AmazonSQS-inboundSequence to log the received message.

<?xml version="1.0" encoding="UTF-8"?>
<sequence name="AmazonSQS-inboundSequence" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
    <log level="full"/>
</sequence>

Add Dependencies to MI

For the Amazon SQS Inbound Endpoint

Navigate to the connector store and search for AmazonSQS Connector. Click on AmazonSQS Inbound Endpoint and download the .jar file by clicking on Download Inbound Endpoint. Add the org.apache.synapse.amazonsqs.poll.class-1.1.1.jar to the <MI-HOME>/dropins directory.

Build and deploy the carbon app

Refer to the documentation on how to build and export the Carbon Application.

Test the Scenario

After deploying the modified Carbon Application, invoke the API the send messages to the queue. For example,

curl — location ‘http://localhost:8290/sqs/ sendToQueue’ \ — header ‘Content-Type: application/json’ \ — data ‘{ “companyName”:”IBM", “queueName”:”SQSTest” }’

And in SQS console, you can see the event is received for the queue.

And in the logs, we can observe the below, where the AmazonSQSPollingConsumer has polled for the events are received events from the queue. And the relevant message information is logged using the AmazonSQS-inboundSequence

[2024-10-21 23:12:44,320]  INFO {LogMediator} - To: , MessageID: urn:uuid:B4C4358C4BFD4A47771729532564299, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><m0:getQuote xmlns:m0="http://services.samples"><m0:request><m0:symbol>WSO2</m0:symbol></m0:request></m0:getQuote></soapenv:Body></soapenv:Envelope>

So thats all we are going to discuss in this article. Hope you can develop your use cases based on the above discussion.