Event pipeline simulation

1Description of the system

1.1Purpose

The app shows how one event moves through an event pipeline in two AWS regions. At the end of the pipeline, an API stores the event in an Aurora PostgreSQL database. The app also shows what occurs when a part of the pipeline or the database fails. The default view also shows a read path: two UIs that query the stored data. The Show control hides the read path.

A live simulation runs in the app. You can change the traffic, the topology, the failures, the API and the database in the Simulate panel. Two pages show the same simulation: a 3D page and a 2D page.

1.2Message path in short

1. The producer publishes an event to an SNS topic.
2. SNS sends a copy of the event to each subscribed SQS queue.
3. A pod asks SQS for a message with an HTTP request, and the answer carries the message. This is a pull. KEDA sets the number of pods.
4. The pod puts the SQS body into an Avro record.
5. The pod sends the Avro message over a Core connection, through PrivateLink and a Network Load Balancer (NLB), to one Artemis broker.
6. The pod deletes the SQS message only after the broker confirms the send.
7. The Artemis cluster puts the message on a broker that has consumers.
8. The broker sends a copy of the message to its paired broker in the other region. This copy is the mirror copy.
9. A broker sends the message over a Core connection to one of 10 consumers. This is a push. The consumer does its own work. Then it sends an HTTP request to an API pod in its region.
10. The API pod writes the event to the one Aurora writer: INSERT … ON CONFLICT (event_id) DO NOTHING. A request from us-east-2 crosses the region for this write.
11. The writer commits the row. Aurora replicates the row to the readers.
12. The API sends a 2xx response. The consumer then sends an acknowledgement (ack). The broker also mirrors the ack.
13. If the API sends a 4xx or a 5xx response, the consumer rolls back. The broker delivers the message again, or moves it to the dead-letter address.

The read path is separate from this message path. It only reads:

1. Center or Cube sends a query to the Query API of its region. Center and Cube are two separate deployments.
2. A Query API pod sends the query through the ALB to an API pod. The consumers use the same ALB and the same API pods.
3. The API pod reads the reader of its region.
4. The answer goes back on the same path.

1.3Block diagram

Figure 1 does not show the read path. Section 3.22 shows it. Figure 1 also does not show the DLQ processor, the S3 bucket or the ALB between the consumers and the API. Section 3.21 shows them. In the app, they are on the outer side of each region.

1.4Regions, queues and database

Each region has its own address and queue. The mirror copies messages under the names of their source region. Thus each region also holds the queue of the other region as a copy.

The consumers in each region read only the queue of their region. The mirrored ack deletes a standby copy. A consumer reads a standby copy only after a failover. In the two “Same queue name” modes of the app, both regions use the queue name events.queue.

Each region also has an API and an Aurora cluster. There is only one writer for the two regions. The table shows the start state of the app.

After a failover, the writer can be e1-2 or e2-1. The two APIs then write to that instance. The counter writer in the Simulate panel shows the current writer.

1.5Colour code

The app and the diagrams in this manual use the same colours. In print, the diagrams use darker shades of the same colours.

Other display rules in the app:

• In the SQS queue, a bright cube is a visible message. A dim cube is a message in flight.
• A pod or a consumer shows red for a short time after a failure.
• A broker that is down shows grey. Its stored messages also show grey.
• Cubes on top of a broker are its stored messages. Violet cubes are messages of this region. Cyan cubes are mirror copies.
• A consumer that waits for the API shows teal (2D view).
• An API pod that starts shows hatched. A pod with requests in flight shows teal. Small marks on the pod show its in-flight requests (2D view).
• An Aurora instance shows its role (WRITER, READER or PROMOTING…), an ACU gauge, its busy slots and its queued operations. Light blue cubes are its rows. A hollow cube is a row that has not replicated to this instance yet (2D view). The writer also shows its writes per second.
• Center and Cube show the number of rows in their last answer. A short red outline shows a failed query.
• A Query API pod uses the same display rules as an API pod, in periwinkle.

1.6Simulation values and assumptions

The rules in the app come from AWS, KEDA, Kubernetes, PostgreSQL and Artemis documentation. The times, rates and counts are illustrative, and simulation time is compressed. The app changes these values so that you can see the effects quickly:

The model also uses these assumptions. The research notes give them because the original request does not specify them:

• Each region has one SNS topic, and the producer publishes in its region. The panel has a switch for one topic to both regions.
• Each region has 10 Artemis consumers.
• The mirror pairs are broker N in us-east-1 and broker N in us-east-2.
• The consumers get to the brokers through the same PrivateLink endpoint service as the pods.
• Each region uses its own address and queue names (use1.* and use2.*). A failover makes the standby copy available to the other region.
• The API is after the Artemis consumers. The consumers call it over HTTP. Its HPA metric is in-flight + queued requests for each pod, with target 1.
• One Aurora writer serves the two regions. The us-east-2 API writes directly across the region. Write forwarding is the alternative.
• For an event, the API by default only writes. The read before write is an option (Read before write, off by default).
• Each region has one Center, one Cube and one Query API Deployment. The Query API calls the existing API through the existing ALB.
• By default, each region sends 1 query each second, and Center and Cube take turns. Each query goes to the next ready Query API pod. The app draws no load balancer in front of the Query API.
• A Cube query takes 4 times the time of a Center query on the reader, and costs 4 times the ACUs. A UI holds a maximum of 20 open queries. Above this limit, it skips the refresh.
• After a failover, the old us-east-1 instances come back as readers that already have all rows. A real failover builds the old primary volume again.

Serverless v2 scaling in the app is a simple demand rule. Each write costs 0.45 ACU-seconds. Each read and each Center query costs 0.15 ACU-seconds. Each Cube query costs 0.6 ACU-seconds. Each second, the app divides the demand by the current ACUs. Above 70%, the instance grows by 50% (at least 0.5 ACU). Below 30% for 3 polls, it shrinks by 0.5 ACU for each poll. Above 100%, each statement takes longer, up to 4 times. Real Serverless v2 scales on CPU, memory and network load. Refer to 3.19.

1.7Pull and push

The two queues give messages to their clients in different ways. SQS is pull: the client asks. Artemis is push: the broker sends.

Simulation The rings are only a display. The app gives a pod its message with no time for the ask, and it gives a consumer one message at a time, as with a small consumerWindowSize. Thus the pull and the push move at the same speed in the app. The app does not show the cost of empty polls or of a full client buffer.

2Components

This section has one subsection for each part that you can click in the app. A part that exists in each region has one subsection for the two regions. The part id is the id that the app uses, for example in the URL parameter pin.

The prefix e1- identifies a part in us-east-1. The prefix e2- identifies a part in us-east-2. Use the “Show in app” links to open the part in the app with its info card.

Parts of the “Both regions (live)” view

Parts of the “Both regions (live)” view: API and Aurora

These parts are after the Artemis consumers. The 2D view shows live data for them when you click them.

Parts of the “Inside Artemis” view

This view is a close-up of the us-east-1 cluster. It shows a fixed example: six pod connections pinned 3 / 2 / 1 and ten consumer connections pinned 5 / 5 / 0. The live simulation does not change this view.

Parts of the DLQ lane

These parts are on the outer edge of each region in the “Both regions (live)” view.

Load balancer in front of the API

This part is between the API and its callers (the consumers and the Query API) in each region, in the “Both regions (live)” view.

Parts of the read path

These parts are on the outer side of each region, in the “Both regions (live)” view. They only read. Set Show to Ingestion only to hide them. Simulation The diagram puts one of each part in each region. This is a model choice.

3Operation

This section tells how the system operates. Each subsection agrees with one step of the guided tour in the app. Each subsection also tells what the tour step does to the simulation.

4Procedures

4.1Open the app pages

1. Open the 3D page at /.
2. To open the 2D page, click Open 2D version ↗ in the title card.
3. To go back to the 3D page, click Open 3D version ↗ in the title card of the 2D page.
4. To open this manual, click Manual in the title card.

4.2Select a view

1. Find the view buttons at the top of the page.
2. To see the live simulation, click Both regions (live).
3. To see a close-up of the us-east-1 cluster, click Inside Artemis.

4.3Examine a part

1. Make sure that Explore is selected.
2. Move the pointer onto a part. A tooltip shows the name and the type of the part.
3. Click the part. The info card shows the description, the data and the related parts.
4. To go to a related part, click its button in the info card.
5. To close the info card, click × or push Esc.
6. In the 2D view, click one pod, one consumer or one broker to see its live data.

4.4Use the guided tour

The tour has 23 steps. Section 3 gives the text of each step. Steps 15 to 20 are about the API and Aurora. Step 21 is about the DLQ processor. Step 22 is about the read path.

1. Click Guided tour. The tour starts at step 1.
2. Read the text of the step. Look at the callouts in the scene.
3. To go to the next step, click Next → or push the → key.
4. To go to the previous step, click ← Back or push the ← key.
5. To play the animation of the step again, click ↻ Replay.
6. To go to a different step, click its mark in the progress bar.
7. To stop the tour, click Explore or push Esc.

4.5Move the 3D camera

1. To turn the model, drag with the left mouse button.
2. To zoom, turn the scroll wheel.
3. To pan, drag with the right mouse button.
4. On a touch screen, drag with one finger to turn. Pinch with two fingers to zoom.
5. To move the parts apart, open Display options. Then move the Exploded view slider.

4.6Pan and zoom the 2D diagram

1. To pan, drag the diagram.
2. To zoom, turn the scroll wheel or pinch with two fingers.
3. To fit the diagram to the window, double-click the diagram.

4.7Use URL parameters

You can give each state of the app as a link. Add the parameters after ?, and join them with &.

Examples: /?view=artemis&pin=nlb, /2d?mode=guided&step=10, /?panel=0&explode=0.6, /2d?pin=aurora-e1-1, /?pin=e2-api.

4.8Use the Simulate panel

The Simulate panel is on the right side of the page. On a narrow screen, the panel starts collapsed. Tap its header to open it.

Control reference

Change the traffic

1. Set the Publish rate slider to the rate that you want.
2. To add one event, click Publish 1.
3. To add a burst of events, click Burst 25.
4. Look at the counters pods us-east-1 and pods us-east-2. KEDA adds pods when the queue grows.

Change the capacity

1. Under Capacity (per region), find the four controls KEDA max pods (SQS), Artemis consumer count, API max pods (HPA) and Query API max pods (HPA).
2. For each control, click 10, 20, 30, 40 or 50. Query API max pods (HPA) also has 5, its default. The value applies to each region.
3. To make a load, click Burst 1000.
4. Look at the pod grid, the consumer grid, the API grid and the Query API grid. Each grid shows more, smaller cells for a higher value.
5. Look at the counters pods us-east-1, API pods us-east-1 and in SQS.
6. To go back to the default capacity, click Reset.

Change the topology

1. Under Fan-out, click 1 topic → both regions or Topic per region.
2. Under Artemis consumers, click one of the three options.
3. Look at the counters deliveries / event and duplicates.

Cause failures

1. To follow one poison message, click Send 1 poison → SQS DLQ or Send 1 poison → Artemis DLQ.
2. To keep messages in the Artemis DLQ, set DLQ processor on to off. To drain them to S3, set it to on.
3. Look at the event log. It shows each failed attempt.
4. To make a percentage of events poison, set one of the two Poison sliders.
5. To add random failures, set the Transient failures slider.
6. To make a backlog in Artemis, set Consumer work time (before the API call) to a high value, for example 4 s.
7. To change the delay of the mirror, set the Mirror lag slider.
8. To move stranded messages to brokers with consumers, set Redistribution on.
9. To delay each redelivery, set the Artemis redelivery-delay slider.

Show or hide the read path

1. Open the Simulate panel. Show is the first section.
2. To see only the ingestion pipeline, click Ingestion only.
3. To see the UIs and the Query API, click Ingestion + UIs. This is the default.
4. To change the load, set Center queries and Cube queries (analytics).
5. Examine the counters queries sent, Center answered, Cube answered, queries failed and queries skipped.

The counters API 2xx, API 4xx and API 5xx count consumer calls only. At the default rates, each region sends 1 query each second, and Center and Cube take turns. If you set different rates, they do not take turns.

Scale the API

1. Set API time per request to a high value, for example 0.8 s.
2. Click Burst 25.
3. Look at the counters API pods us-east-1, API pods us-east-2 and API queued.
4. Look at the API pods. The ALB uses round robin, so each ready pod gets the same number of requests, also when it is busy.
5. Look at the event log. It shows each HPA decision, for example “HPA us-east-1: 6 in flight ÷ 1 → 1 → 6 pods”.
6. Wait approximately 15 s after the burst. The HPA removes the pods that it does not need.

Change the Aurora settings

1. To increase the replica lag, set Replication lag to us-east-2 to a high value, for example 3 s.
2. To make the API read before it writes, set Read before write (dedupe on the local reader) to on.
3. To see duplicates removed, set Artemis consumers to Same queue name, both regions consume.
4. Look at the counters dedupe at read, dedupe at write and rows stored.
5. To limit the capacity, set Serverless v2 max capacity to a low value, for example 1 ACU.
6. Set Publish rate to 8 events/s.
7. Look at the counter ACU e1-1 / e1-2 / e2-1. The writer stops at the maximum, and the API pods increase.

Stop the writer and fail back

1. Click Kill writer (e1-1). The log shows the in-region failover to e1-2.
2. Look at the counter writer. It shows promoting… for 3 s, then us-east-1 (e1-2).
3. To start e1-1 again, click Restart e1-1. It comes back as a reader.
4. To move the writer back to e1-1, click Switchover to e1-1.
5. Look at the counter rows lost (RPO). It stays at 0: an in-region failover and a switchover lose nothing.

Compare NLB balancing with cluster balancing

1. Click Reset.
2. Wait approximately 30 s.
3. Write down the counters us-east-1 arrivals via NLB B1/B2/B3 and stored by cluster B1/B2/B3.
4. Click Reset again.
5. Set Pods open a new connection per message to on.
6. Wait approximately 30 s.
7. Compare the two counters with the values from step 3.

Result: with long-lived connections, the arrivals are not equal. With a new connection for each message, the arrivals are almost equal. The stored messages are equal in the two cases.

Stop a broker or a region

1. Click Kill B1. The button changes to Restart B1.
2. Look at the connection lines. They move to the other brokers.
3. To start the broker again, click Restart B1.
4. To send connections to the restarted broker again, click Reconnect clients.

5. To stop all of us-east-1, set Region outage (us-east-1) to on.
6. Look at the counters writer, API 5xx and rows lost (RPO). After 5 s, the writer is us-east-2 (e2-1).
7. To start us-east-1 again, set Region outage (us-east-1) to off. e1-1 and e1-2 come back as readers.
8. To move the writer back to us-east-1, click Switchover to e1-1. The app does not do this automatically.

Control the clock

1. Set the Speed slider to make the simulation faster or slower.
2. To stop the clock, click Pause. The button changes to Resume.
3. To start the clock again, click Resume.
4. To start again from zero with the default settings, click Reset.

Live counters

Event log

The event log shows the last 50 entries. Each entry starts with the simulation time. Yellow entries are warnings, light red entries are failures and green entries are recoveries.

Entries for the API and Aurora:

• HPA decisions, for example “HPA us-east-2: 4 in flight ÷ 1 → 1 → 4 pods (new pods ready in 2s)”.
• ACU changes, only when the integer part changes, for example “Aurora e1-1 (writer): 0.5 → 1.0 ACU (utilisation 120%)”.
• 503 bursts, one entry for each second, for example “API us-east-2: 6× 503 in the last second (writer promotion in progress); consumers roll back”.
• Read path: “Read path ON” and “Read path OFF”, “HPA Query API us-east-1: …” for its pods, and failed queries, one entry for each second, for example “Read path us-east-2: 2 queries failed in the last second (no Aurora instance up in us-east-2 to read from)”.
• Dedupe, failover, promotion and lost rows, for example “Aurora: 3 committed rows had not replicated to e2-1 → lost (RPO)”.

4.9Run a scenario by hand

1. Open the 3D page or the 2D page in Explore mode.
2. Click Reset.
3. Set the controls that the table gives for the scenario.
4. Wait approximately 60 s.
5. Read the counters and the event log. Compare them with the column “What to look for”.

4.10Run all scenarios with npm

1. Open a terminal in the project folder event-pipeline-v2-3d.
2. If this is the first run, type npm install.
3. Type npm run scenarios.
4. To change the traffic time of each scenario, type npm run scenarios -- --seconds 120.
5. To run the simulation in Node.js without a browser, type npm run scenarios -- --node. The numbers are the same.
6. Compare the table with 6.3.

The script opens the 2D page in a headless browser. It runs each scenario for 60 simulated seconds of traffic (default). Then it runs until nothing moves, for a maximum of 10 simulated minutes. The random generator has a fixed seed, so the numbers are the same each time.

The base settings of the script are: publish rate 2 events/s, Topic per region, Per-region queues, mirror lag 0.7 s, consumer work time 0.5 s, API time 0.25 s, replication lag 0.8 s, maximum 8 ACU, global failover on, Read before write off, redelivery delay 0, and capacity 10 / 10 / 10 in each region (KEDA max pods, Artemis consumers, API max pods). The column drain time (s) is the simulated time from the first publish until nothing moves or waits. The column stuck in Artemis counts messages that stay in Artemis after all other work stops. A stranded event counts two times: the original message and its mirror copy.

The script also examines these conditions after each scenario. The column check shows ok if they are all true:

• Each row belongs to a processed event. (Exception: a row of an event that went to the dead-letter address after its 200 was lost.)
• Each processed event has a row, unless a global failover lost it.
• The rows are not more than the unique events.
• At the end, nothing moves or waits.
• Each query that a UI sent got an answer or failed, and no query stays open.
• If a region stays down at the end (scenarios 50 and 51), the check does not need that region to be idle, because its SQS queue keeps its messages.

Scenarios 1 to 31, 39, 41 and 44 to 51 run with the read path off. The other scenarios turn it on.

The column acked, no row counts processed events that have no row in Aurora. Only a global failover (RPO) causes this.

4.11Control the simulation from the console

1. Open the developer tools of the browser.
2. Select the Console tab.
3. To change settings, type __sim.set({ rate: 5, consume: 'sharedBoth' }).
4. To stop broker 2 in us-east-1, type __sim.kill('e1', 1).
5. To read the counters, type __sim.counts().
6. To make the HPA add API pods, type __sim.set({ apiTime: 0.8 }).
7. To turn on the read before write, type __sim.set({ readCheck: true }).
8. To see more dedupe at write, type __sim.set({ replLag: 3, consume: 'sharedBoth' }).
9. To stop the writer, type __sim.killDb('e1-1'). To start it again, type __sim.restartDb('e1-1').
10. To move the writer back to e1-1, type __sim.switchover().
11. To turn off the mirrored queues, type __sim.set({ mirroring: false }).
12. To hide the read path, type __sim.set({ readPath: false }). To change its load, type __sim.set({ centerRate: 1, cubeRate: 1, qapiMax: 10 }).
13. To read the API and Aurora values, type __sim.counts().acu, __sim.counts().rowsLost or __sim.db.writer.

__sim.counts() also gives these API and Aurora values: apiPodsE1, apiPodsE2, apiQueued, apiOk, api4xx, api5xx, rows, writesPerSec, dedupRead, dedupWrite, crossWrites, rowsLost, acu (text, for example '2.0 / 0.5 / 0.5') and writer (text, for example 'us-east-1 (e1-1)'). For the read path, it gives queries, queriesOk, queriesFailed, queriesSkipped, centerOk, cubeOk, qapiPodsE1 and qapiPodsE2.

4.12Run the screenshot checks

The checks start their own Vite server on a free port. They open the app in headless Chromium.

3D check

1. Type npm run shoot.
2. To check only some tour steps, type npm run shoot -- --steps 3,4.
3. To skip the phone screenshots, add --no-mobile. To skip the view screenshots, add --no-views.
4. Read the last line. “OK: no errors or broken references” tells you that the check passed.
5. If the check fails, examine shots/report.json. It lists console errors, broken tour or part references and blank frames.

2D check

1. Type npm run shoot:2d.
2. To check only some tour steps, type npm run shoot:2d -- --steps 3,4.
3. Read the last line. “OK: no errors” tells you that the check passed.
4. Examine the screenshots in shots-2d/.

4.13Build and preview

1. Type npm run build. Vite writes dist/ with index.html, 2d.html, manual.html and assets/.
2. To serve the build with the Cloudflare runtime, type npm run preview:cf. This command builds the site and starts wrangler dev, with the same routing as production.
3. Open the URL that wrangler shows.
4. Make sure that /, /2d and /manual open.

4.14Deploy to Cloudflare

The app deploys as a Cloudflare Worker with static assets. The code in worker/index.js handles only /api/*, for the notes. Each reader has one Durable Object, with their email from Cloudflare Access as its key. The file wrangler.jsonc holds the configuration.

Deploy from your computer

1. Type npx wrangler login. Do this one time only.
2. Type npm run deploy. The command builds the site and starts wrangler deploy.
3. Open https://event-pipeline-v2.<your-subdomain>.workers.dev/.
4. Make sure that /, /2d and /manual open.

Make the notes sync

The API answers 401 if Cloudflare Access does not protect it. Notes then stay in the browser of the reader. You must do this procedure in the Cloudflare dashboard, because wrangler deploy does not change Access.

1. In Cloudflare Zero Trust, add a self-hosted Access application for event-pipeline-v2.tudum.workers.dev.
2. Set the path of the application to api/. The app and the manual stay public.
3. Add a policy that lets in your account. Do not use the email domain proton.me in a policy, because other people have that domain.
4. Type curl -I https://event-pipeline-v2.tudum.workers.dev/api/annotations. A 302 response to cloudflareaccess.com tells you that Access protects the API.
5. Open the manual. Click ✎ Notes, then Sign in to sync. After the login, the page comes back to the manual.

Optional: to use a custom domain, remove the comment marks from routes in wrangler.jsonc. The domain must be on your Cloudflare account.

To change the name of the Worker and its URL, change name in wrangler.jsonc.

5Fault isolation

Use this table when the app or a real pipeline shows a problem. Find the symptom, then do the action. Rows with “real system” apply only to a real pipeline.

6Reference data

6.1Documented values

These values come from the AWS, KEDA, Kubernetes, PostgreSQL and Artemis documentation. The numbers in brackets refer to Section 8.

6.2Simulation values

All values in this table are illustrative values of the app (src/content/sim.js). Simulation

6.3Scenario results

The read path results are in their own table.

These tables are the output of npm run scenarios -- --node on 2026-11-02. Each scenario has 60 simulated seconds of traffic at 2 events/s, if the scenario does not set a different rate. The run is seeded, so the numbers repeat. These scenarios run with the read path off, so they show ingestion only. Simulation

Messages

API and Aurora

• Scenario 1: the arrivals are not equal (52 / 8 / 0), but the stored messages are equal (40 / 40 / 40). The NLB pins connections, and ON_DEMAND spreads the messages.
• Scenario 9: 4 messages stay stranded on broker 1. This is two events, each with its mirror copy.
• Scenario 11: the one duplicate is the failover edge condition. us-east-1 processed the message, but its ack did not get to us-east-2 before the outage. One message goes to the Artemis DLQ: three fast 503 responses during the promotion. At the end, the writer is e2-1, because the script does not do a switchover.
• Scenario 13: 20 publishes fail. With one topic, the region of the topic is a single point of failure.
• Scenarios 2, 3 and 5: each duplicate is removed at the write. The rows stay equal to the unique events.
• Scenario 15: the writer gets to 8 ACU, the maximum. The API increases to 5 pods in us-east-1 and 6 pods in us-east-2.
• Scenario 17: 35 duplicates stop at the read and 85 stop at the write. The read on a reader that is 2 s behind misses most duplicates.
• Scenario 18: the global failover loses 3 rows (RPO). A later delivery of one of these events wrote its row again. The other 2 events have no row at the end, although a consumer acked them (acked, no row 2).
• Scenario 19: with no global failover, each write gets 503 for the full outage. 18 messages go to the Artemis DLQ, but no rows are lost.
• Scenarios 21 and 22: with redelivery delay 0, 8 messages go to the Artemis DLQ. With 3 s, no message does.
• Scenario 23: with the read path off, the write-only API does not use e2-1. Thus a stopped reader has no effect on the requests.
• Scenarios 25 and 26: the same spike of 1000 events. With capacity 10 / 10 / 10, the drain takes 212.7 s. With 50 / 50 / 50, it takes 68 s. At 50, the writer is at its maximum of 8 ACU, so Aurora is the next limit.

Artemis DLQ processing

These scenarios ran on 2026-11-02. With the processor on, each message that went to the Artemis DLQ is in S3 at the end. With it off, each one is still in the DLQ. Messages in the SQS DLQ always stay.

Read path

These rows are the output of the same command on 2026-11-03. Scenarios 1 to 31 run with the read path off, so the tables above stay the same. All 12 rows show ok in the column check. Last event done is the time until the last event finishes. Drain also includes the queries that wait in the Query API.

Compare scenarios 34 and 39, and scenarios 40 and 41. The reads increase the API pods and the ACUs of the reader. They have almost no effect on the time until the last event finishes.

Mirrored queues

These rows are the output of the same command on 2026-11-03. All 8 rows show ok in the column check. In scenarios 50 and 51, us-east-1 does not come back. Stuck in Artemis then counts the messages on its brokers. With mirroring on, this count includes copies of messages that us-east-2 already processed.

Compare scenarios 45 and 46: a short outage has almost the same result, because the messages of us-east-1 wait and then finish. Compare scenarios 50 and 51: if the region does not come back, mirroring saves the messages that were in Artemis (169 and 161 events done). It does not save the messages that were in the SQS queue of that region. Compare scenarios 3 and 47: with no copies, one shared queue name gives no duplicates.

7Glossary

8Sources

1. Apache ActiveMQ Artemis. AMQP broker connections and mirror. artemis.apache.org
2. The same page, earlier location. activemq.apache.org
3. Amazon SNS. Cross-region delivery. docs.aws.amazon.com
4. Amazon SNS. Raw message delivery; delivery retries; dead-letter queues.
5. Amazon SQS. Standard queues. docs.aws.amazon.com
6. Amazon SQS. Dead-letter queues; visibility timeout; SetQueueAttributes.
7. AWS PrivateLink. Endpoint services. docs.aws.amazon.com
8. Elastic Load Balancing. NLB introduction; Network Load Balancers; how Elastic Load Balancing works.
9. KEDA. aws-sqs-queue scaler; ScaledObject specification; Artemis scaler.
10. Apache Avro 1.12 specification. avro.apache.org
11. Confluent Schema Registry. Wire format. docs.confluent.io
12. Apache ActiveMQ Artemis. Address model (MULTICAST, ANYCAST) and mirror notes. artemis.apache.org
13. Apache ActiveMQ Artemis. Clusters: load balancing and redistribution. artemis.apache.org
14. Apache ActiveMQ Artemis. Address settings: dead-letter address and max-delivery-attempts. activemq.apache.org
15. Kubernetes. Horizontal Pod Autoscaling: sync period, algorithm, stabilization window and tolerance. kubernetes.io (examined 2026-10-23)
16. Amazon Aurora. How Aurora Serverless v2 works: ACU, 0.5-ACU steps, capacity ranges and promotion tiers. docs.aws.amazon.com (examined 2026-10-23)
17. Amazon Aurora. Serverless v2 auto-pause (scaling to 0 ACU). docs.aws.amazon.com (linked from [16])
18. Amazon Aurora. Using Amazon Aurora Global Database: storage-level replication, lag less than 1 s, read-only secondaries, up to 10. docs.aws.amazon.com (examined 2026-10-23)
19. Amazon Aurora. Global Database switchover and failover: RPO, RTO and rds.global_db_rpo. docs.aws.amazon.com (examined 2026-10-23)
20. Amazon Aurora. Write forwarding in an Aurora PostgreSQL global database. docs.aws.amazon.com (examined 2026-10-23)
21. Amazon Aurora. High availability: in-region failover and promotion tiers. docs.aws.amazon.com (examined 2026-10-23)
22. PostgreSQL. INSERT … ON CONFLICT. postgresql.org (examined 2026-10-23)
23. Amazon Aurora. Replication with Amazon Aurora: replica lag usually much less than 100 ms. docs.aws.amazon.com (examined 2026-10-23)
24. Amazon S3. Data consistency model.
25. Elastic Load Balancing. Target group attributes for an Application Load Balancer.
26. Amazon SQS. ReceiveMessage.
27. Apache ActiveMQ Artemis. Flow control; Core.
28. Apache ActiveMQ Artemis. Detecting dead connections.

Project files: research/notes.md (research notes and numbers), README.md (scenarios, controls, deploy), CONTRACT.md (design of the API and Aurora parts), src/content/topic.js (tour, panel, legend), src/content/info.js (parts), src/content/sim.js (simulation rules and timings) and scripts/scenarios.mjs (scenario list).