Skip to content
Bitween AI is on the way
Adapters

Build integrations

Adapters

Every built-in adapter and its properties, custom adapters in .NET, Python and JavaScript, publishing, the adapter editor and the source viewer

Adapters do the work at each stage of a subscription. Bitween has four kinds.

Kind Stage Methods
Receiver Pulls items for a scheduled job Initialize, ListFiles, GetFile(id), DeleteFile(id), Finalize
Validator Checks API input before an exchange is created Validate(file) returns success or a list of errors
Mapper Transforms the input Handle(file) returns the output
Handler Delivers and returns an optional response Handle(file) returns the response

What Bitween calls and passes is the adapter contract. In .NET its interfaces are IBitweenReceiver, IBitweenValidator, IBitweenMapper and IBitweenHandler from SimplyWorks.Bitween.Adapters. Adapters written against the older IInfolinkReceiver, IInfolinkValidator and IInfolinkHandler from SimplyWorks.PrimitiveTypes, with XchangeFile, are called the same way. The payload has text Data, a Filename, a ContentType and a BadData flag. Notification channels reuse handler adapters, built in or custom, for subscription notifications and alerts.

Native and custom adapters#

  • Native adapters are compiled into Bitween and run in process. Their ids start with Native, such as NativeHttpHandler.
  • Custom adapters are separate programs, written in .NET, Python, JavaScript or TypeScript, that SimplyWorks.Serverless downloads from object storage and runs. A classic custom adapter runs as a new process for each call. A resident one runs as a long-lived process that keeps its connections open.
  • Data source adapters are resident adapters for brokers (bitween.bus.rabbitmq, bitween.bus.sqs) and databases (bitween.db.postgresql, bitween.db.mysql, bitween.db.sqlserver, bitween.db.oracle). A subscription binds them to a data source. See Data sources, External brokers and Databases.

An id starting with native, ignoring case, is a native adapter. For any other id, Bitween reads the package's metadata and runs it as resident or classic. Resident adapters only run on nodes with Bitween:BusProvidersEnabled. Classic adapters run on every node, including Python and Node ones.

Properties#

Each adapter declares its properties. The UI shows which are required, which are secret, and each one's description and default.

  • Tokens. Values can contain {{partner.KEY}} and {{globals.SET.KEY}}. Bitween substitutes them ignoring case, global values first, when an exchange is created. Unresolved tokens stay as written, so a value set can't be deleted while a subscription or notification channel references it. Receivers only get global values. Notification channels get both on each send, with the partner of the exchange being reported, and legacy notifiers get neither. A response or bus gateway subscription's handler can also use {{source.PATH}}, a value from the original document; see Response routing.
  • Connections. {{connections.NAME.accessToken}} is an OAuth2 access token Bitween keeps fresh, and {{connections.NAME.authorization}} is Bearer and the token, for a header. Unlike the tokens above, they are filled in as the adapter starts, for every adapter and every kind (handlers, mappers, validators, receivers, notification channels and legacy notifiers), so a token is never stored on an exchange. See Connections.
  • Secrets. Secret values are never sent to the browser. The API returns __private__ instead, and sending __private__ back keeps the stored value. If Bitween cannot describe an adapter, it masks every property. A property is secret only if its adapter says so: [Secure] on a native adapter, and on a published one isPrivate: true in Runner.Expect (.NET), secret=True in sw.expect (Python) or secret: true in expect (JavaScript and TypeScript), which bitween adapter build writes into the manifest. Anything else is returned as stored. A secret property's default is never sent either, but the adapter still applies it when the value is left empty.
  • Required properties are checked when a subscription is saved. A blank value counts as missing.
  • xchangeid is added to mapper and handler properties at run time.
  • A value that does not convert to the property's type, such as BatchSize=abc, silently falls back to the default.
  • The admin UI loads every adapter of a kind, with its properties, in one call. A published adapter is described from its manifest when the manifest lists its properties. Otherwise a .NET adapter is started and asked; an adapter in another runtime never is, and a manifest that lists none means it has none.
  • Descriptions of custom adapters are cached on each node. Publishing, promoting or withdrawing a version through Bitween tells every node to forget the adapter, over the bus. A version published straight to storage shows its old properties for up to a minute.

Connections#

A connection is an OAuth2 client whose tokens Bitween asks for, renews and rotates, so that an adapter, built in or published, in any language, needs no OAuth code of its own. They are on the Connections page, with the global values permissions.

How it gets tokens For
Approve once (authorization code) Someone chooses Authorize, approves access at the provider, and comes back to /oauth-callback. The refresh token that brings keeps the connection going. Salesforce, Zoho, Shopify and Dynamics apps.
Client ID and secret (client credentials) Nobody approves anything. ServiceNow and most partner APIs.
Service user (password grant) A service user's name and password, for providers that still allow it.

Register https://<bitween host>/oauth-callback as the redirect URI at the provider. The token and authorize URLs are https; plain http is accepted only to this machine itself (localhost, 127.0.0.1), to try a connection or a kit against a provider played locally. The approval uses PKCE and a state checked against an encrypted cookie. Extra approval parameters are added to the approval page's address, as access_type=offline&prompt=consent, which Google needs before it gives a refresh token; others want a scope such as offline_access or refresh_token. A connection approved without one says so, and needs approving again when its token expires.

Write {{connections.NAME.accessToken}} in any setting: the HTTP handler's LoginPassword with AuthType Bearer, or the Token of s9.openapi with Auth bearer. As the adapter starts:

  • a token good for at least two more minutes is used as it is; otherwise it is renewed first, with the refresh token or the client's credentials;
  • of several nodes or calls needing a renewal at once, one renews and the others wait for its token, because a provider that rotates refresh tokens honours only the first renewal;
  • a new refresh token the provider sends is kept, and the old one forgotten.

When a token can't be had, the adapter call fails with the reason, so retry policies and alerts work as for any failure, and the connection shows it. A provider that refuses the refresh token (invalid_grant) leaves the connection Needs authorizing. Test renews now and shows the result.

The client secret, the password and both tokens are encrypted at rest when Bitween:SettingsEncryptionKey is set, and masked in the API as __private__; a secret sent back as __private__ is kept. Changing the client, its token or authorize URL, its scopes or its user forgets the tokens it has. A connection settings refer to (a subscription's, a notification channel's, a notifier's, a retry policy's, a partner's or a global value set's) can't be deleted or renamed. Adding, changing and authorizing a connection are in the audit trail, with the secrets left out; renewing a token isn't.

Data source settings can't use connections: a data source's adapter runs for a long time, and a token handed to it when it starts would expire.

Rebex license#

The Rebex FTP/SFTP adapters and the Rebex POP3 receiver use the commercial Rebex library. They are hidden from the adapter pickers until a key is saved in the Rebex license key setting, which takes effect without a restart. Each has open-source counterparts that need no license: the SFTP adapters and the FTP / FTPS adapters below, and NativePop3Receiver.

Moved to the marketplace#

The Rebex adapters are no longer offered for new work. They are packages in the marketplace now, with the same properties plus a LicenseKey:

Built in Marketplace package
NativeRebexFtpReceiver s9.rebex.ftp.receiver
NativeRebexFtpUploadHandler s9.rebex.ftp.upload
NativeRebexPop3Receiver s9.rebex.pop3.receiver

A package doesn't read the Rebex license key setting, so keep the key in a global value set and write {{globals.<set>.<key>}} in its LicenseKey property.

Nothing that uses a built-in one changes: it keeps running, and its editor keeps it. Each stays in the pickers, with a licence key saved, while any subscription names it, active or not, and for the upload handler a notification channel, a notifier or a retry alert counts too. Once nothing does, it is no longer listed, and there is no picking it again.

Handlers#

NativeHttpHandler#

Sends the payload to an HTTP endpoint.

Property Default Notes
Url (required) When the payload is present and the URL contains {{, it is rendered as a Liquid template over the payload, so https://api.example.com/orders/{{ order.id }} works.
Verb post get, put or delete. Any other value, including patch, sends a POST. GET sends no body.
AuthType ApiKey, Bearer, Basic, Login or OAuth2, matched case-sensitively. Empty means no authentication.
ApiKey (secret) Used by ApiKey.
LoginUsername Used by Basic and Login.
LoginPassword (secret) Used by Basic and Login, and as the token for Bearer.
LoginUrl Token endpoint for Login and OAuth2.
ClientId, ClientSecret (secret) Used by OAuth2.
ContentType application/json application/x-www-form-urlencoded form-encodes a flat JSON object. multipart/form-data sends the payload as one part named file. Other types send the payload as text.
Headers (secret) Extra headers as Name:Value pairs separated by commas. A value cannot contain a colon.
CorrelationId Sent as the request-context-correlation-id request header.
DefaultRequest Body to send when the payload is empty.
AuthType Behaviour
ApiKey Adds the header ApiKey: {ApiKey}
Bearer Adds Authorization: Bearer {LoginPassword}
Basic Basic authentication with LoginUsername and LoginPassword
Login POSTs Email and Password as JSON to LoginUrl, reads Jwt from the reply, and sends it as a bearer token
OAuth2 Requests a client credentials token from LoginUrl, reads access_token, and sends it as a bearer token
Reply status Result
2xx or 3xx Success. The body becomes the response file.
4xx other than 408 and 429 Bad response. The body is kept and flagged, and retry policies see it as a BadResult.
5xx, 408, 429, or below 200 The exchange fails with the status and body, so retry policies see it as an Error. A Retry-After the partner sent is in the error text, and Bitween's next retry waits at least that long (details).

When a destination keeps failing

Each Bitween process keeps a circuit breaker per destination: scheme, host and port, so https://erp.example/orders and https://erp.example/invoices share one, and every subscription calling that destination shares it too. It is there so one partner that stops answering does not take every place in the processing queue for its whole timeout while flows to healthy partners wait behind it.

Counts as a failure Doesn't count
No answer within TimeoutSeconds 2xx and 3xx, which reset the count
Connection refused, reset or dropped 4xx other than 429: the partner rejected this message, and other messages may be fine. These reset the count, as the partner answered.
429 501: the endpoint will never do what was asked, and waiting won't change that
500, 502, 503, 504 and any other 5xx but 501 A failure before anything is sent, such as a URL template that doesn't render, or a sign-in (Login, OAuth2) that is refused

After Bitween:CircuitBreakerFailures failures in a row (5 by default) within Bitween:CircuitBreakerWindowSeconds (60), the circuit opens. For Bitween:CircuitBreakerOpenSeconds (60) every delivery to that destination fails at once, without signing in or sending anything, with:

503: ServiceUnavailable: circuit open for https://erp.example (5 failures in a row; trying again after 10:15:30Z)

The exception is a CircuitOpenException. The 503: prefix is the one the handler uses for a real 503, so a retry policy that retries 503 retries these too; see Retries and alerts. After the cool-down one delivery is let through to try. If it succeeds, the circuit closes and deliveries flow again; if it fails, the circuit opens for another cool-down. While that trial is under way, other deliveries still fail at once. A 500 from one message the partner can't handle counts like any other, but takes a run of them with no success between to open the circuit.

The circuit opening is logged as a warning, with the destination and the time it will try again, and closing as information. The bitween.circuits.opened counter, tagged with the destination, is exported with Bitween's other metrics.

Each replica learns on its own, from its own calls, and forgets on restart. Bitween:CircuitBreakerFailures set to 0 switches the breaker off. See Configuration.

NativeSmtpHandler#

Sends an email built from the payload.

Property Default Notes
Host (required)
Port 587 Port 465 uses implicit TLS. Other ports use STARTTLS when UseTls is on.
UseTls true TLS is required when on, never opportunistic.
Username From
Password (secret) Authenticates only when set, and only over a secure connection.
From (required), FromName
To (required), Cc, Bcc Comma-separated addresses.
Subject (required), Body (required) Scriban templates over the payload, with the same syntax as the legacy JSON mapper. They are rendered only when the payload is JSON, otherwise sent as written.
IsHtml true
CertificateThumbprint The relay certificate's SHA-256 thumbprint, in hex. When set, exactly that certificate is trusted, a self-signed one included, and any other is refused before anything is sent.

Connecting, which covers reaching the relay, its greeting and the TLS handshake, may take 30 seconds. A relay that expects TLS from the first byte on a port other than 465 never greets a client that starts plainly, so it fails there, saying so. Without a pinned thumbprint, server certificates must be valid. A chain whose only problem is that revocation could not be checked is accepted. The response file holds the rendered subject.

A relay that keeps failing is paused as the HTTP handler pauses a partner (When a destination keeps failing), keyed by smtp://{Host}:{Port}. Timeouts, dropped connections and a 421 reply count. A refused recipient, a bad template or a refused sign-in don't.

NativeS3UploadHandler#

Writes the payload to an S3-compatible bucket.

Property Default Notes
AccessKeyId, SecretAccessKey (secret), ServiceUrl, BucketName All required.
FolderName Ignored when FileName is set.
FileName Full object key. When empty, the key is {FolderName}/{yyyyMMddHHmmss}_{guid}.{FileExtension}.
FileExtension
ContentType text/plain
DataEncoding utf8 base64 decodes the payload into bytes before uploading, for a binary file such as an Excel workbook the mapper wrote.

The response file holds the object key.

NativeAzureBlobUploadHandler#

Writes the payload to an Azure Blob container.

Property Notes
ConnectionString (required, secret), ContainerName (required)
FileName Blob name. When empty, a timestamp and GUID name is generated.
FileExtension
DataEncoding utf8 by default. base64 decodes the payload into bytes before uploading, for a binary file such as an Excel workbook.

Existing blobs are overwritten. The response file holds the blob name.

NativeSftpUploadHandler and NativeFtpUploadHandler#

Upload the payload as a file, with open-source clients and no license: NativeSftpUploadHandler over SFTP (SSH.NET), NativeFtpUploadHandler over FTP or FTPS (FluentFTP). The file is named after the exchange file, or a UTC timestamp when it has no name, and a name from the message can't climb out of TargetPath.

A server that keeps failing is paused as the HTTP handler pauses a partner (When a destination keeps failing), keyed by sftp://{Host}:{Port} or ftp://{Host}:{Port}. Timeouts and refused, reset or dropped connections count. A refused sign-in, a missing directory or a refused write don't. The receivers, and the Rebex, S3 and Azure Blob adapters, have no breaker.

Property Default Notes
Host (required), Username (required)
TargetPath Remote directory.
FileName The file's name, in place of the message's own. For a JSON message it can use the message's fields, as order-{{ orderId }}.json, so each message gets its own file. Only the last path segment is kept. Empty keeps the message's name, or the time when it has none. Messages through a gateway are all named {urlName}.json, so without this each upload replaces the one before.
FileNamePrefix Prepended as {prefix}_.
DataEncoding utf8 base64 decodes the payload into bytes before uploading.

SFTP connection properties, shared with NativeSftpReceiver:

Property Default Notes
Port 22
Password (secret) Signs in, or is the private key's passphrase when a key is set.
PrivateKey (secret) Signs in with this key instead, PEM or OpenSSH format. A PEM key pasted onto one line is re-wrapped.
HostKeyFingerprint The server's SHA256:... host-key fingerprint. When set, any other key is refused before credentials are sent.
TimeoutSeconds 60 How long to wait for the server at each step: connecting, opening the SFTP session, each request. A server that never answers fails the run instead of holding it, and a place in the processing queue, indefinitely.

FTP connection properties, shared with NativeFtpReceiver:

Property Default Notes
Port 21, or 990 for implicit FTPS
Password (required, secret)
Encryption none none for plain FTP, explicit for FTPS with AUTH TLS, implicit for FTPS that is TLS from the start.
CertificateThumbprint The server certificate's SHA-256 thumbprint, in hex (colons and case don't matter). When set, exactly that certificate is trusted, a self-signed one included, and any other is refused before credentials are sent. When empty, the certificate must be valid.
PassiveMode true Turn off for active mode.

NativeRebexFtpUploadHandler#

Uploads over SFTP or plain FTP with Rebex, shown as "(Rebex)". Needs a Rebex license, and is offered only while in use: new work uses s9.rebex.ftp.upload. Its Protocol property picks the protocol, where the open-source adapters are split by protocol instead: sftp and sftpssh map to NativeSftpUploadHandler, ftp to NativeFtpUploadHandler with Encryption none. Every other property has the same name.

Property Default Notes
Host (required), Username (required)
Port 22 for SFTP, 21 for FTP
Protocol sftp sftp or ftp with a password, or sftpssh with a private key.
Password (secret) Required for sftp and ftp. The key passphrase for sftpssh.
PrivateKey (secret) Required for sftpssh. A PEM key pasted onto one line is re-wrapped.
TargetPath Remote directory.
FileName The file's name, in place of the message's own. For a JSON message it can use the message's fields, as order-{{ orderId }}.json, so each message gets its own file. Only the last path segment is kept. Empty keeps the message's name, or the time when it has none. Messages through a gateway are all named {urlName}.json, so without this each upload replaces the one before.
FileNamePrefix Prepended as {prefix}_.
DataEncoding utf8 base64 decodes the payload into bytes before uploading.

The file is named after the exchange file, or a UTC timestamp when it has no name.

Receivers#

BatchSize caps the items taken per run and defaults to 50. ResponseEncoding is utf8 or base64; use base64 for binary content.

NativeHttpReceiver#

Calls an HTTP endpoint once per run and turns the JSON reply into items. Its properties match the HTTP handler's, with these differences: Verb defaults to get, there is no URL templating or multipart, Login posts UserName rather than Email, and there is one extra property.

Property Notes
ArrayPath JSON path to the array of items. Without it, a root array is split into items and any other reply is a single item.
NextLinkPath For an API that pages its results: the path to the next page's address in each reply, such as next, links.next or @odata.nextLink. It can be relative. Pages are followed until it is empty or missing, and only on the Url's own server: a link elsewhere fails the run rather than be sent the credentials.
PageParameter For an API that numbers its pages: the query parameter that does it, such as page. Pages are asked for from PageStart (1) up, until one has no items.
MaxPages The most pages one run reads with either, 20 by default. The next run starts from the first page again.

The reply must be JSON, and a status of 400 or above fails the run. Without NextLinkPath or PageParameter it makes one request per run, as before. Items have no file name, and nothing is removed at the source.

NativeS3Receiver#

Property Notes
AccessKeyId, SecretAccessKey (secret), ServiceUrl, BucketName Required.
FolderName Key prefix. No / is added, so incoming also matches incoming-archive/.
BatchSize, ResponseEncoding
DeleteMovesFileTo When set, each processed object is copied under this prefix and then deleted. Otherwise it is deleted.

NativeAzureBlobReceiver#

Property Notes
ConnectionString (required, secret), ContainerName (required)
FolderName Prefix, with a / added.
BatchSize, ResponseEncoding, DeleteMovesFileTo As for S3. A blob that is already gone is skipped.

NativeSftpReceiver, NativeFtpReceiver and NativeRebexFtpReceiver#

Read files over SFTP (NativeSftpReceiver), FTP or FTPS (NativeFtpReceiver), with open-source clients and no license, or over SFTP or plain FTP with Rebex (NativeRebexFtpReceiver, shown as "(Rebex)", needs a license, and is offered only while in use). The connection properties are the upload handlers' for the same protocol.

Property Default Notes
TargetPath Directory to read.
BatchSize, ResponseEncoding
DeleteMovesFileTo When set, processed files are moved into this directory. Otherwise they are deleted.
CheckFileExistence true Skip the delete quietly when the file is already gone.
MinimumFileAgeSeconds 0 A file changed more recently than this is left for the next run, so one still being uploaded isn't taken half-written.

NativePop3Receiver and NativeRebexPop3Receiver#

Read email from a POP3 mailbox. NativePop3Receiver uses MailKit. NativeRebexPop3Receiver uses Rebex, needs a license, connects over implicit TLS on port 995 only, and is offered only while in use.

Property Notes
Host, Username, Password (secret) Required.
Encryption NativePop3Receiver: implicit (TLS from the first byte, the default) or explicit (STLS). The password is never sent unencrypted.
Port NativePop3Receiver: 995 with implicit TLS and 110 with explicit, unless set.
CertificateThumbprint NativePop3Receiver: the server certificate's SHA-256 thumbprint, in hex. When set, exactly that certificate is trusted, a self-signed one included, and any other is refused before the password is sent. When empty, the certificate must be valid.
BatchSize, ResponseEncoding

Each email becomes one item, named after its subject. When an email has attachments, only the first attachment is used. Otherwise its text body is used. Processed emails are deleted when the session closes.

Mappers#

Id Description
NativeMapper Rules-based mapper for JSON and XML, with a visual editor
NativeJSONMapper Legacy Scriban template mapper for JSON, offered only while a subscription still uses it

See Mapping. Bitween has no native validators.

Custom adapters#

What Bitween calls on an adapter of each kind, and what it passes, is the adapter contract. A custom adapter can be written in .NET, Python, JavaScript or TypeScript. The bitween CLI writes a new one of each kind, builds it into a package, checks it against the contract and publishes it.

Custom adapters in .NET#

A .NET adapter is a console application that references SimplyWorks.Serverless.Sdk, and SimplyWorks.Bitween.Adapters for the contract's interfaces. bitween adapter init AcmeOrders --kind handler (--lang dotnet is the default) writes one targeting .NET 10, declared with [AdapterKind("handler")] and [AdapterContract("bitween", 1)] and implementing IBitweenHandler.

Adapters written before SimplyWorks.Bitween.Adapters existed use the SimplyWorks.PrimitiveTypes interfaces and keep working. This is the repository's sample handler, written that way.

using SW.PrimitiveTypes;
using SW.Serverless.Sdk;

class Program
{
    static async Task Main(string[] args) => await Runner.Run(new Handler());
}

class Handler : IInfolinkHandler
{
    public Handler()
    {
        // Declares a property named ContentType with a default value.
        Runner.Expect("ContentType", "text/plain");
    }

    public Task<XchangeFile> Handle(XchangeFile xchangeFile)
    {
        var contentType = Runner.StartupValueOf("ContentType");
        return Task.FromResult(xchangeFile);
    }
}
  • Declare properties in the constructor with Runner.Expect. Overloads take a default value, whether the property is private, and a description.
  • Read values with Runner.StartupValueOf(name).
  • A validator implements IBitweenValidator and returns a ValidationResult, adding each failure with AddError(code, message); no errors means valid. With SimplyWorks.PrimitiveTypes it implements IInfolinkValidator and returns new InfolinkValidatorResult(errors).
  • A receiver implements IBitweenReceiver, or IInfolinkReceiver.
  • Runner.CorrelationId holds the exchange's correlation id. AdapterLogger forwards log lines to Bitween.
Sample project Shows
SW.Bitween.SampleHandler A handler with one property
SW.Bitween.SampleMapper A mapper, which is a handler
SW.Bitween.SampleValidator A validator using FluentValidation
SW.Bitween.SampleConfigurableAdapter A test double that can delay, fail or return fixed data
SW.Bitween.SampleResidentHandler A handler built as a resident adapter

Custom adapters in Python#

A custom adapter can also be written in Python 3.12 or later, with sw-serverless and the Bitween kinds in simplyworks-bitween (this repository's sdk/python). Subclass a kind and implement its methods; settings are declared with sw.expect as Runner.Expect declares them in .NET.

import sw_serverless as sw
from simplyworks_bitween import ExchangeFile, Handler


class Orders(Handler):
    def __init__(self):
        sw.expect("Url", description="Where orders go")
        sw.expect("ApiKey", secret=True)

    def handle(self, file: ExchangeFile) -> ExchangeFile:
        # A partner's rejection is returned with bad_data=True, not raised.
        return ExchangeFile(data=file.data, filename=file.filename)


if __name__ == "__main__":
    sw.run(Orders)
Kind Implement
Handler handle(file), returning an ExchangeFile
Mapper map(file), returning an ExchangeFile
Validator validate(file), returning a ValidationResult
Receiver list_files(), get_file(file_id), delete_file(file_id), and optionally initialize() and finalize()

The bitween CLI does the rest:

bitween adapter init AcmeOrders --lang python --kind handler
cd AcmeOrders
bitween adapter build                          # the package, with the SDKs and requirements.txt vendored
bitween adapter test --settings settings.json  # the Bitween contract checks
bitween login https://bitween.example.com      # once
bitween adapter publish bin/serverless/acme.orders-0.1.0.zip

A Python adapter is published to its versions and the catalog only, never to {Bitween:AdapterPath}/{id}, so a host older than SW-Serverless 10.1 never sees it. An id that already has a .NET package there can't be reused for a Python or Node adapter. Bitween runs it with python3, which the Docker image includes (Ubuntu 24.04's Python 3.12). Requirements are vendored into the package with pip; one with native code is vendored for linux-x64 and linux-arm64 unless adapter.json lists other platforms.

Python and Node adapters connect to Bitween over a Unix domain socket, so they run only on Linux and macOS hosts. They run through SW-Serverless's resident adapter host, which Bitween starts on every node, whether or not Bitween:BusProvidersEnabled is on.

Custom adapters in JavaScript or TypeScript#

The same, on Node 22 or later, with @simplyworks/sw-serverless and @simplyworks/bitween (sdk/node). Extend a kind and implement its methods, named as in Python but in camelCase: handle, map, validate, and listFiles, getFile, deleteFile.

import { expect, run, valueOf } from "@simplyworks/sw-serverless";
import { ExchangeFile, Handler } from "@simplyworks/bitween";

class Orders extends Handler {
  constructor() {
    super();
    expect("Url", { description: "Where orders go" });
  }

  handle(file: ExchangeFile): ExchangeFile {
    return new ExchangeFile({ data: file.data, filename: file.filename });
  }
}

run(Orders);

bitween adapter init AcmeOrders --lang typescript --kind handler (or --lang node for JavaScript) starts one. bitween adapter build needs no TypeScript compiler: Node strips the types, so only syntax that strips cleanly is allowed - no enums, namespaces or parameter properties. Dependencies in package.json are installed into the package with npm; one with native code has to be built on the platform the adapter runs on. Bitween runs it with node, which the Docker image includes (the Node 22 binary, without npm). It runs on the same hosts and nodes as a Python adapter.

Publishing a custom adapter#

Build the adapter with bitween adapter build, then publish the package:

bitween login https://bitween.example.com
bitween adapter publish bin/serverless/acme.orders-1.0.0.zip            # a new version, not current yet
bitween adapter promote acme.orders 1.0.0                               # make it the one that runs
bitween adapter versions acme.orders

Publishing through Bitween needs adapter-source.operate, so an author needs a Bitween account rather than the storage's keys, and every version is in the audit trail. Bitween checks the package's manifest and settles its version; it does not run the contract checks, so run bitween adapter test first. With -p and the storage flags, the CLI publishes straight to storage instead, as sw-serverless publish does - for CI that holds the keys.

What is published, and where:

{Bitween:AdapterPath}/{id}                       the current package, for hosts older than versions (.NET only)
{Bitween:AdapterPath}-versions/{id}/{version}    every version
{Bitween:AdapterPath}-catalog/{id}.json          versions, the current one, and each one's manifest

A version is published without being made current unless --current is given, so subscriptions keep running what they ran until it is promoted, or until a subscription pins it. Making an older version current again is how a release is rolled back. A withdrawn version stays listed but can't be pinned or made current; subscriptions already pinned to it keep running it.

On the Adapters page, someone with adapter-source.operate can do the same from the browser: Upload package publishes a .zip built by bitween adapter build (any language, with the version taken from the package or bumped, and made current only when ticked), and each version that isn't current has Make current and Withdraw.

The adapter's kinds come from its manifest - bitween adapter build writes them from what the code declares - so an adapter can be named anything. Packages published before manifests are found by the older convention, infolink6.{kind}s.{name}, and by the Kind metadata they were published with.

An adapter in another runtime than .NET, and a .NET version published without being made current, has no package at {Bitween:AdapterPath}/{id}. The picker finds it in the catalog instead, under the kinds its manifest declares, while it has a version that has not been withdrawn.

A manifest can name the oldest Bitween the adapter works with, in compatibility.applications.bitween (manifests written before SimplyWorks.Serverless 10.2 used compatibility.minBitweenVersion, which is still read). Saving a subscription that changes a stage to such an adapter, or pins such a version, on an older Bitween is refused with ADAPTER_NEEDS_NEWER_BITWEEN. A Bitween's own version is Bitween:Version, shown under Settings, About.

A custom adapter may run for Bitween:ServerlessCommandTimeout seconds, 300 by default.

The Adapters page names what each custom adapter runs on (.NET, Python, Node.js or a native binary), from its current version's manifest, with a Runtime column in its Versions table and a runtime filter once more than one is in use. Beside a subscription's adapter, the runtime of the version it runs is named, and the version picker names each version's runtime when they differ, since pinning one can move a step to another runtime.

Installing from the marketplace#

The Marketplace tab on the Adapters page lists the connectors a Bitween Registry offers, and installs them. It is on by default, with Simplify9's registry and its signing keys, and reaches the registry only when someone opens the tab or installs, and once a day, when something was installed from it, to check for updates and revocations. Settings, Marketplace turns it off, for a Bitween that must not reach the internet, or points it at another registry or a mirror: its Registry address and its signing keys, the registry's Ed25519 public keys, base64 or PEM, separated by commas or new lines.

Browsing needs subscriptions.view; installing needs adapter-source.operate. Installing a version:

  1. reads it from the registry, and refuses one that isn't published there, one from a Community publisher unless Allow Community connectors is on, a paid one without a licence (REGISTRY_LICENCE), and one that needs a newer Bitween (ADAPTER_NEEDS_NEWER_BITWEEN);
  2. refuses an id this Bitween already has from anywhere but a registry (REGISTRY_ID_TAKEN), so a registry package never takes the place of an adapter published here, and a version it already has (REGISTRY_INSTALLED);
  3. reads the registry's revocation list, which must be signed by a trusted key, and refuses a revoked version (REGISTRY_REVOKED);
  4. downloads the package and checks that its SHA-256 is the one signed, that the registry's countersignature verifies with one of the trusted keys, and that the publisher's signature verifies with the key the registry vouched for (REGISTRY_SIGNATURE);
  5. publishes it as Upload package does, made current only when Make it current is ticked, and records the registry in its catalog entry.

Every check fails closed: a registry that can't be reached, or a list that doesn't verify, refuses the install. Once installed, a connector runs from this Bitween's own storage like any published adapter, and never needs the registry again; pinning, Make current and Withdraw work as for any other. Each install is in the audit trail as an AdapterRelease with the action installed.

Installing from a bundle

A Bitween that must not reach the internet installs from a bundle instead: the zip bitween registry bundle downloads on a machine that can, with the packages named, both signatures on each, the registry's keys, its revocation list and the licences the downloader is entitled to. Install from bundle on the Marketplace tab (adapter-source.operate) takes the zip, with the marketplace on or off, and runs every check above on each package against the signing keys in Settings, never the keys the bundle carries: published, allowed here, not revoked by the bundled list, the bytes those signed, both signatures. Licences in the bundle are kept, so a paid connector installs when one covers it. Each package is installed, skipped (already here) or refused on its own, and the answer says which and why; the catalog entry records the registry the bundle came from. The API is POST /api/marketplace/bundle?current=true with the zip as the body.

A card says Installed when this Bitween has the registry's newest version, and Update available when it has an older one. Nothing is updated by itself. Besides the search and the publisher, the tab filters by type (connectors or kits), category, runtime and price.

Free connectors need nothing more than the above. A paid connector, marked so on its card, installs and runs only on a licence from the registry, and seeing a publisher's private connectors needs the same registration. Under Settings, Marketplace, Register takes a one-time registration code from Simplify9 (or from your customer page on the registry) and a name for this Bitween; the registry answers with an instance id and a token, kept as the settings Registered as and Instance token (a secret, so it needs Bitween:SettingsEncryptionKey). From then on every call to the registry carries the token. Clearing the two settings unregisters.

A licence is a document the registry signs with the same keys that countersign packages: the customer, the instances it names (one, or any), the packages it covers (ids, or a namespace such as s9.*), when it starts and ends, and days of grace. Bitween fetches them at registration, with Refresh, and once a day with the update check, keeps the ones that verify with the trusted keys, drops and reports the rest, and stores them in its own database, so every node reads the same and nothing is asked of the registry when a connector runs. The panel lists each licence as Live, In grace, Not yet or Expired, and the Marketplace tab says the same on each paid card.

Installing a paid connector fetches the licences first and refuses without one that is live or in grace. Running one checks the stored licences on every call, at no cost: a connector whose licence is past its grace fails with ADAPTER_LICENCE_EXPIRED, a failure the subscription's retry policy retries, so the calls go through once the licence is renewed. Nothing else on the instance is affected, and a free connector, or one not installed from the registry, is never asked.

Send usage reports, off unless turned on, sends the registry once a day, with the update check, one line per connector installed from it: the package id and version, how many subscriptions name it, and yesterday's calls and failures from the adapter metrics. No payloads, no partner or subscription names, no hostnames. Show yesterday's report on the panel says exactly what would go before it is turned on, and Send it now sends it. The report is /api/marketplace/usage with send true or false (settings.edit); registering is /api/marketplace/register, the licences held are /api/marketplace/licences (subscriptions.view), and /api/marketplace/refreshlicences fetches them now (adapter-source.operate).

A card marked Kit is connectors plus the configuration that makes them useful (see Kits). Opening it shows the kit's parameters as a form, with secrets as password fields and defaults filled in, and Install as, a name for a second installation of the same kit. Show plan says what it would make, changing nothing; Install then installs its connectors at the versions it names and applies its configuration. The kit's package is checked as a connector's is, every step above, before anything is read from it. It installs as you: Bitween runs the same planner as bitween kit install against its own API with your session, so it can make only what you could make yourself, and each change is in the audit trail under your name, with an install-kit operator action for the whole. Installing a kit as an adapter is refused (REGISTRY_KIT).

Updates and revocations

Once a day, on whichever node gets there first, Bitween asks the registry about every adapter installed from it: is there a newer version, with its release notes, and has an installed version been revoked. Check now on the Installed tab asks at once (adapter-source.operate). What it finds is kept in the database, so every node shows the same and the pages that show it never wait on the registry:

  • the Installed tab says when the registry was last asked and how many adapters have a newer or a revoked version; a row shows v1.2.0 available and, when its current version is revoked, Revoked, with the registry's reason on hover; its versions mark each revoked one and list the newer ones with their notes;
  • a subscription whose receiver, validator, mapper or handler runs a revoked version, pinned or following current, says so at the top of its page, with the reason.

Revocations are read only from the registry's signed list, checked against the trusted keys as installing does. A list that doesn't verify, or a registry that can't be reached, changes nothing: the last good check's flags stay, and the tab says why the check didn't finish. A revoked version keeps running; what to do about it, pin another version or make one current, is yours to decide.

The data source providers (Oracle, PostgreSQL, MySQL, SQL Server, RabbitMQ and Amazon SQS) are on the marketplace too, published by Simplify9 under their own ids, such as bitween.db.oracle, which data sources refer to. They are the one exception to REGISTRY_ID_TAKEN: one installed by hand from a CLI release was Simplify9's as well, so the registry's version upgrades it. scripts/publish-providers-to-registry.sh builds all six with a release's version and publishes them for review.

Simplify9's own connectors, from the Bitween-Connectors repository, include:

Package
s9.openapi A handler that calls one operation of a REST API, found in its OpenAPI 3.0 or Swagger 2 description, checking the body against the operation's schema before sending. Bearer (a connection works), Basic, API key or OAuth2 client credentials. A Retry-After the API sends is waited out.
s9.openapi.receiver A receiver that takes records in from a list operation: every page (Link header, next address, cursor, page number or offset), each record one exchange, and only what changed since the last run, by a bookmark.
s9.soap A handler that calls a SOAP 1.1 or 1.2 web service, with HTTP Basic or a WS-Security UsernameToken. A Client or Sender fault is a bad response; a Server or Receiver fault fails the delivery, so a retry policy retries it.
s9.rebex.ftp.receiver, s9.rebex.ftp.upload, s9.rebex.pop3.receiver The Rebex adapters that were built in.

And kits on those connectors: s9.salesforce.contacts, s9.zoho.contacts (any Zoho data centre) and s9.servicenow.incidents, each one job both ways between that system and yours.

Writing an adapter in Bitween#

Python, JavaScript and TypeScript adapters can be written in Bitween itself, without the CLI.

  • New adapter on the Adapters page starts one from the template bitween adapter init writes. Edit on a published Python or Node version starts a draft from the source that version carries.
  • A draft is saved in Bitween's database. The editor shows its files; Save and check builds it on the server and runs the Bitween contract checks bitween adapter test runs. Try calls a command with the settings and input given. Settings typed in the editor are sent with each call and never stored.
  • Publish builds and checks it again and publishes a new version. A version published from the editor is not current: subscriptions keep running what they ran until Make current is used, on the editor's Publish tab or on any version in the Versions table, or until a subscription pins it. Making an older version current is how a release is rolled back.

By default the editor builds adapters that need only the SDK; one whose requirements.txt or package.json names other packages is refused, with the CLI as the way to build it. With Bitween:AdapterEditorDependencies on, the server fetches them from PyPI and npm, or from the mirrors PIP_INDEX_URL and NPM_CONFIG_REGISTRY name. Only plain names and versions are accepted: no URLs, paths, git sources or pip options, wheels only for Python, and no install scripts for Node, so nothing a package brings runs while it is built. A package-lock.json in the draft is ignored, since it can name any source.

Building, checking and trying run the draft's code on the Bitween server, at most two at a time. Each call may take 30 seconds, and a run may use 256 MB and keep one core busy (Bitween:AdapterEditorMemoryMb, Bitween:AdapterEditorCpuCores); past either it is stopped and the editor says which limit it hit. A draft holds at most 200 files and 2 MB. Python and Node versions that carry their source can be opened in the editor; .NET versions are built with the CLI.

Two members can have a draft open at once. A save made after someone else saved is refused, naming who and when, and the editor offers to load their version or to save over it.

Permission Allows
adapter-source.view Reading a published version's source and comparing versions
adapter-source.edit Drafts: starting, editing, deleting, checking and trying them
adapter-source.operate Publishing a version from a draft or as a package through the API (bitween adapter publish), making any published version current, and withdrawing one

All three are Administration permissions: administrators have them, and a custom role can be granted them. Drafts are in the audit trail, each save recorded by a hash of its files, and so is every version published, made current or withdrawn through Bitween (AdapterRelease).

Reading an adapter's source#

A package built with bitween adapter build (or SW-Serverless's sw-serverless build) carries the adapter's source under source/, and its manifest lists each file with its SHA-256. On the Adapters page, a version that carries source has a View link in the Versions table. It opens the version's files; Compare with shows which files another version added, removed or changed, and a diff of each.

Each file is read from the package in storage and checked against the hash in its manifest. A file that does not match is not shown. A file larger than 1 MB, or that isn't UTF-8 text, is listed but its content is not shown. Versions published before packages carried source, or with --no-source, have no link.

Reading source needs the adapter-source.view permission, which only administrators have unless a custom role grants it. Every file read is recorded in the audit trail.

To write a native adapter instead, see Development.