Deployment

Container image#

The Dockerfile builds the admin UI in a Node 22 stage, publishes SW.Bitween.Web with the .NET 10 SDK, and runs it on the ASP.NET 10 runtime image. The final image also carries:

• the .NET 6 shared runtime, copied from the ASP.NET 6 image
• python3, Ubuntu 24.04's Python 3.12, with pip, installed with apt, for Python adapters and the adapter editor
• node and npm from node:22-bookworm-slim, for JavaScript and TypeScript adapters and the adapter editor

Published adapters bring their own dependencies inside their packages. pip and npm are used only by the adapter editor, when Bitween:AdapterEditorDependencies lets it fetch a draft's packages. The container runs as the image's non-root user, which owns only /app/adapters, the cache of downloaded adapter packages.

docker build -t bitween:local .

docker run -p 8080:8080 \
  -e Bitween__DatabaseType=PgSql \
  -e ConnectionStrings__BitweenDb="Host=db;Database=bitween;Username=bitween;Password=..." \
  -e ConnectionStrings__RabbitMQ="amqp://user:password@rabbitmq:5672/" \
  -e Token__Key="..." -e Token__Issuer=bitween -e Token__Audience=bitween \
  -e Bitween__InitialAdminPassword="..." \
  -e Bitween__StorageProvider=S3 \
  -e CloudFiles__AccessKeyId=... -e CloudFiles__SecretAccessKey=... \
  -e CloudFiles__ServiceUrl=https://s3.example.com -e CloudFiles__BucketName=bitween \
  bitween:local

The container listens on port 8080. Released images are published to docker.io/simplify9/bitween.

Kubernetes with Helm#

The chart in charts/default creates a Deployment, a Service, a Secret, and either an ingress-nginx Ingress or a Gateway API HTTPRoute.

helm install bitween ./charts/default \
  --set db="Host=db;Database=bitween;Username=bitween;Password=..." \
  --set dbType=pgsql \
  --set global.bus.rabbitUrl="amqp://user:password@rabbitmq:5672/" \
  --set global.token.key="..." \
  --set global.token.issuer=bitween \
  --set global.token.audience=bitween \
  --set storageProvider=S3 \
  --set global.cloudFiles.accessKeyId=... \
  --set global.cloudFiles.secretAccessKey=... \
  --set global.cloudFiles.serviceUrl=https://s3.example.com \
  --set global.cloudFiles.bucketName=bitween \
  --set secrets.Bitween__SettingsEncryptionKey="..." \
  --set secrets.Bitween__InitialAdminPassword="..." \
  --set ingress.hosts[0]=bitween.example.com

Things to know about the chart:

• The image tag is the chart version, so a chart must be packaged with the version of an image that exists.
• The Service forwards port 80 to container port 8080.
• The Ingress exposes /api, /swagger and / by default, so the admin UI and /blank.html are served from the same host as the API. A deployment that overrides ingress.paths replaces the whole list, so it has to include / itself.
• Probes are on by default: a startup probe and a liveness probe on /health/live, which only checks that the process answers, and a readiness probe on /health/ready, which also checks the database and that the bus is connected to RabbitMQ. A pod whose broker is down starts and stays not ready until the bus connects; it does not restart. The exception is an installation's first start on a version that records the environment name its queues are made under: it won't start until RabbitMQ answers which name that was. The startup probe allows 40 minutes for migrations (probes.startupFailureThreshold).
• A stopping pod lets the exchanges it is running finish for up to 30 seconds before it closes its queues, so they are acknowledged instead of run again elsewhere. The app waits up to 45 seconds to stop, and the chart gives the pod 60 (terminationGracePeriodSeconds); keep that value above 45.
• The rabbitmq values are not used by any template.
• charts/default/README.md explains the two routing modes.

See Configuration for how each value becomes an environment variable.

The chart runs Bitween with a read-only root file system (securityContext.readOnlyRootFilesystem, on by default), so nothing in the image can be changed from inside the container. Bitween writes only to /tmp and to its cache of downloaded adapter packages, /app/adapters, which the chart mounts as emptyDir scratch space: created empty with the pod and gone with it, as the container's own layer was, and not persistent volumes. scratch.tmpSizeLimit and scratch.adapterCacheSizeLimit cap them, as 2Gi.

Databases#

Set Bitween:DatabaseType and ConnectionStrings:BitweenDb.

Migrations run automatically when the service starts. A failed migration stops startup and is logged with the targeted host and database. The scheduler's tables are created in the same database.

Azure managed identity#

Set Bitween__UseAzureManagedIdentity=true and leave the password out of the connection string.

• Azure SQL. Bitween appends Authentication=Active Directory Default to the connection string.
• Azure Database for PostgreSQL. Bitween requests an Entra ID token and refreshes it every 50 minutes, for both the application and the scheduler.

For a user-assigned identity, set Bitween__AzureManagedIdentityClientId or AZURE_CLIENT_ID. MySQL has no managed identity support.

Object storage#

Set Bitween:StorageProvider and the matching CloudFiles keys. See Configuration.

• At each start Bitween writes .access-check under the document prefix and reads it back, and won't start when it can't. The same file tells the Settings page whether the bucket is open to everyone.
• With the default temp30/ document prefix, the storage libraries expire exchange files after 30 days. Choose temp1/, temp7/ or temp365/ for other retention, or a prefix outside those to keep files. The prefix can be changed later on the Settings page.
• For Azure Blob, set CloudFiles__SubscriptionId and CloudFiles__ResourceGroupName, and give the identity a role that can manage the account's lifecycle policy. The rules are then created at startup. Without them the rules aren't created, and the Settings page can't show any set up by hand. See Configuration.
• Keep the bucket private: Bitween reads and writes with its own credentials and serves file links itself. On Azure it switches its container to private at startup; with a managed identity, give it a role that can change the container's access level, or set it to Private yourself.
• The chart sets Bitween's public address from its first host name (ingress.hosts, or gateway.hostnames), https unless the Ingress has no TLS for it. Exchange file links are built on it, so a partner handed a roll-up's links can open them. Without a host, roll-ups link through the instance's own address, which only adapters running in Bitween can open.
• Local writes files to disk and only starts in Development.

RabbitMQ#

• Bitween declares its queues at startup, and again whenever work groups or bus-enabled information types change.
• Enable the management plugin and set the three Bitween:RabbitMqManagement* keys to get queue health.
• Give each deployment that shares a broker its own Bitween:QueuePrefix.

Data sources#

Broker and database connections run only on nodes with Bitween__BusProvidersEnabled=true, and they need ConnectionStrings__RabbitMQ for leases.

• Install the adapter packages, bitween.bus.* and bitween.db.*, under {Bitween:AdapterPath} like custom adapters. The pipelines in this repository do not publish them.
• The chart has no dedicated values, so pass the settings through environmentVariables.
• Database pools are held on every enabled node, so size MaxPoolSize against the number of replicas.

See Data sources.

Scaling#

Run several replicas behind a load balancer for throughput and availability, and tune prefetch, parallel exchanges and priority per work group. Read the replica notes about the scheduler first.

When Bitween won't start#

Bitween checks its configuration and its database before it serves anything. When something is wrong it logs one entry, Bitween can't start until these are fixed:, with a numbered list of what to change, and exits. Fix everything on the list and redeploy.

• Without ConnectionStrings:BitweenDb or ConnectionStrings:RabbitMQ, the list is written to the console and holds only the configuration problems; the database is checked on the next start.
• A failed migration is logged on its own, as Database migration failed on startup against {Host}:{Port}/{Database}.

Upgrading#

• Migrations apply at startup, so back up the database before deploying a new version.
• Values already in the Settings table are not replaced by new configuration. Change them on the Settings page. A startup warning names each one configuration sets differently.
• Upgrading from a version without work groups needs nothing set: at each start Bitween reads the queues that version left behind until they are empty, then removes them.

Upgrading to parallel exchanges#

Before this version every lane ran one exchange at a time per node, whatever its prefetch. Now a lane runs up to its Parallel exchanges at once, and a lane that leaves it empty takes its prefetch: 12 by default. That is what the documentation always described, but not what deployments saw, so on upgrade:

• Partners receive concurrent calls. A partner that got one call at a time from each node can get up to the lane's parallel exchanges at once. One that limits its callers answers 429 and the circuit breaker opens, which is handled, but it is new for that partner.
• Order within a lane is not kept above 1. Two exchanges for the same order sent close together, a create and an update, can now run in either order on one node. Order was never kept across nodes.
• More of everything at once. Each running exchange can hold a database connection and its payload in memory; five lanes at 12 is 60 at once per node, against Npgsql's default pool of 100. Lanes that use classic (non-resident) adapters start that many processes.

To keep a lane as it was, set it to 1 before upgrading: on the Work groups page, set the group's Parallel exchanges to 1 and save (the field is stored with the group and ignored by older versions), or for the Ungrouped lane set Bitween__BusDefaultConcurrency=1 (busDefaultConcurrency: '1' in the chart). The results lane of every group runs one at a time either way.

Also in this version: a publish waits for the broker to confirm it (Bitween__BusPublisherConfirms=false publishes as before), a pod whose broker is down at boot starts and becomes ready when the bus connects, and a stopping pod drains its running exchanges.

CI/CD#

Releases are versioned 10.0.x.