Operations

Health endpoint#

GET /health returns 200 while the process is up. It does not check the database, RabbitMQ or storage.

Finding an exchange#

The Exchanges page filters by status, subscription, partner, information type, exchange ids, correlation id, a promoted property value and a date range. Filters live in the URL, so a filtered view can be shared as a link.

• Search by promoted property to find a business document, such as an order number.
• Search by correlation id to see every exchange in one flow, including exchanges created from responses.
• Searching by id also finds retries of that exchange and the aggregation that collected it.
• Turn on Latest attempt only to hide attempts that were already retried, leaving only the failures that still need action.

Expanding an exchange shows each stage's file, the exception, its retry chain, the aggregation family, and buttons to retry or to run a waiting retry now. An exchange that was already retried links to its later attempt instead of offering Retry.

Reading an exchange's state#

Queue health#

The Queue health page reads the RabbitMQ management API every 5 seconds. It needs monitoring.view and the three RabbitMQ management keys.

• Tiles show lanes, queue depth, retry backlog, dead letters, and incoming and acknowledged rates.
• Lanes list each consumer with its nodes, in-flight, queued, retrying and dead messages, prefetch, rates and health. They are grouped as front doors, work, notifications, legacy and control.
• Alerts come from the bus library, such as backpressure and dead letters.
• Nobody is reading these lists queues with Bitween's prefix that no consumer reads. They are usually left behind by a deleted or renamed work group or information type, and RabbitMQ does not remove them.

The bus library retries a message whose consumer throws through the .retry queue, and parks it in the .bad queue when those retries run out. This is transport-level failure, separate from retry policies. An exchange that fails in its adapters still gets a result and does not go to .bad.

The same data is available from GET /api/ops/summary, /api/ops/consumers, /api/ops/queues, /api/ops/retries, /api/ops/deadletters, /api/ops/alerts and /api/ops/unattendedqueues.

Data sources#

• The Data sources page lists each connection with its state, last error and the node that holds it.
• Test connection runs the adapter's own checks and shows each stage. For databases it also checks every statement.
• The live panel refreshes every 3 seconds, but only when the page is served by the node running the adapter.
• A data source nobody runs keeps showing its last recorded state.

See Data sources.

Scheduled work#

• The Scheduled jobs and Aggregations pages show each subscription's last run, reliability, next run and schedule health.
• A subscription's overview lists its receive attempts and the exchanges each created.
• Stuck means a run was killed and left its running flag set, so later runs are skipped.
• Not scheduled means a trigger is missing. Saving the subscription again recreates it.

See Scheduling.

Dashboard#

The dashboard, opened from the logo, shows today's exchanges, the 7-day success rate, today's failures, waiting retries, queue alerts, a 14-day chart, the latest failures, unhealthy subscriptions and the busiest subscriptions. The browser computes it from up to 1,000 recent exchanges, so figures on a busy instance are approximate.

Two parts come from the server instead. Failures to act on counts failed exchanges that are the newest attempt of their chain, of any age, and links to that search. Chains that keep failing lists the five retry chains with the most attempts, with the success rate of retries over the last 7 days.

Audit trail#

The Audit trail page shows who changed which configuration entity, when, and the values before and after. Filter by entity type, member, entity id, date range, or everything changed in one save. See Security.

Logs#

Bitween logs to the console. When SwLogger:ElasticsearchUrl is set and the environment is listed in SwLogger:ElasticsearchEnvironments, logs also go to Elasticsearch, enriched with request context.

Messages worth watching for:

• Database migration failed on startup against {Host}:{Port}/{Database}
• A startup warning that the Microsoft sign-in redirect URI is not the sign-in landing page
• Auto-retry evaluation failed for xchange …
• Retry budget of subscription … could not be cleared after a success
• Bus gateway route references subscription …, which is not active; skipping.

Data retention#

Exchange rows outlive their files. Retrying an exchange whose input file has expired fails, and a waiting retry for it is dropped with a reason.

Common problems#