Reference
Connecting Bitween to a database: a guide per engine
Written for a developer who knows SQL and has met a relational database before, but has not necessarily met this one. It assumes nothing about Bitween beyond the fact that you are pointing it at a…
Written for a developer who knows SQL and has met a relational database before, but has not necessarily met this one. It assumes nothing about Bitween beyond the fact that you are pointing it at a database and want it to work.
Each engine gets the same five questions answered:
- What must I set, and what happens if I leave the rest alone.
- How do I write a statement — placeholders, row limits, identifier case.
- What does the database hand back, and as what.
- How do I call a procedure, and will it return rows.
- What will go wrong, and what the message means when it does.
The terse settings tables live in database-adapters-api.md §2. This is the explanation behind them.
The parts that are the same everywhere#
A data source is a connection, held open. The adapter is a long-lived process with an ADO.NET
pool inside it, so a connect, a TLS handshake and an authentication round trip are paid once rather
than per message. That is the whole reason it exists, and it is why the pool settings matter more
than they would in a web app: MinPoolSize connections are held per node, so multiply by your
replica count before comparing against the server's session limit.
SQL lives on the data source, not in a message. You create statements — named pieces of SQL
belonging to the connection — and a subscription names one. A message supplies parameter values
and never SQL text. This is not ceremony: adapter property values have {{partner.X}} substituted
into them before the adapter sees them, so SQL in a subscription's properties would be steerable by
ordinary partner data.
A statement is checked when you save it. The engine parses and plans it — never runs it — so a typo is refused while you are still looking at the form. If the connection is not running at that moment the save says "Saved, but not checked" rather than pretending.
Only the engine's own answers are reported. The capability list (Describe) is what this
server can do, narrowed by version where it matters. If it says merge: false, writing a MERGE
will fail, and the list is telling you so first.
PostgreSQL — bitween.db.postgresql#
The easiest of the four to configure, and the one most likely to be right by default.
What you must set#
Host, Database, UserName, Password. That is genuinely all.
Database is required and has no default, because PostgreSQL connects to a database, not to a
server — there is no "log in and then pick one". It is the name you would pass to psql -d.
What the defaults do#
| Setting | Default | Leave it unless… |
|---|---|---|
Port |
5432 | |
SslMode |
Prefer |
Change this to Require for anything not on localhost. Prefer will silently fall back to an unencrypted connection if the server does not offer TLS — which is precisely the case you wanted to be told about. |
Schema |
server default | You want unqualified names to resolve somewhere other than public. Sets search_path; takes a list, e.g. sales, public. |
ApplicationName |
Bitween |
Keep it distinctive — it is how a DBA finds your connections in pg_stat_activity. |
AutoPrepare |
false |
You are running the same statement constantly and have measured a win. It caches a plan per connection, and a plan built against one shape of data can be worse than replanning. |
ConnectionIdleLifetimeSeconds |
300 |
Writing a statement#
- Parameters are
@name. Not:name— that collides with the::cast operator, andvalue::textwould be read as a parameter calledtext. - Row limit is
limit N, at the end. - Identifiers fold to lower case unless quoted. A table created as
"Orders"must always be written"Orders"; one created asOrdersisordersand can be written either way.
select id, order_no, total
from sales.orders
where customer_id = @customerId
order by id
limit 100
What comes back#
numeric → decimal, timestamptz → DateTime, uuid → Guid, jsonb → a string, bytea → bytes,
arrays → an array. A row count from Discover is reltuples, which is as fresh as the last
ANALYZE and is null on a table that has never been analysed — that is an honest "unknown",
not a zero.
Procedures and functions#
This is the one thing PostgreSQL does differently from the other three, and it catches people:
-
A
PROCEDUREcalled withCALLcannot return a result set. The capability list saysprocedureResultSets: falsefor exactly this reason. -
A set-returning
FUNCTIONcan, and it is queried withSELECT, notCALL:select * from orders_by_customer(@code)So a "stored procedure that returns rows" from an Oracle or SQL Server background is a function here, and it goes through Query rather than Call.
Receiving#
Any always-increasing column works as a cursor — a serial/identity id, or a timestamptz
updated on write. Use where id > @cursor order by id; the order by is not optional, because
without it the last row read is arbitrary and the cursor will skip rows.
What goes wrong#
| Message | What it means |
|---|---|
42P01: relation "x" does not exist |
Wrong name, wrong schema, or your search_path does not include it. Qualify it or set Schema. |
42703: column "x" does not exist |
Usually a case problem — the column was created quoted with capitals. |
syntax error at or near ":" |
You wrote :name. Use @name. The save-time check says this in words. |
merge: false in the capability list |
The server is older than 15. MERGE does not exist there; use insert … on conflict. |
MySQL — bitween.db.mysql#
Serves MariaDB too. Configuration is simple; the surprises are in what the engine does not have.
What you must set#
Host, Database, UserName, Password.
Database is also the schema. MySQL has no separate concept — CREATE SCHEMA is a synonym for
CREATE DATABASE — so this one value is both what you connect to and what an unqualified table name
resolves against. There is no search_path equivalent and no separate Schema setting, and
browsing the catalog with no schema filter means this database, not every database on the server.
What the defaults do#
| Setting | Default | Leave it unless… |
|---|---|---|
Port |
3306 | |
SslMode |
Preferred |
Required or above off localhost, same reasoning as PostgreSQL's Prefer. |
ConnectionIdleLifetimeSeconds |
180 | Check your server's wait_timeout. The server default is 28800, but a managed instance or a proxy in front of one is routinely far lower. A connection the server closed first surfaces as a broken pipe on the next message rather than as a timeout, so keep this comfortably below it. |
ServerPrepare |
true |
You connect through ProxySQL or similar, where prepared statements pin a backend and defeat the pooling the proxy exists to provide. Turning it off also weakens the save-time check — with IgnorePrepare on, nothing is sent to the server and every statement reports valid. |
AllowZeroDateTime |
false |
The schema contains 0000-00-00, which MySQL permits and no .NET date type can hold. Only old schemas have these. |
Writing a statement#
- Parameters are
@name. - Row limit is
limit N, at the end. - Identifier case follows the file system on the server: case-sensitive on Linux, case-insensitive on Windows and macOS. Write table names exactly as they were created and the question never comes up.
- Backticks quote an identifier, not double quotes (unless
ANSI_QUOTESis set).
What comes back#
| Column type | Arrives as | Worth knowing |
|---|---|---|
tinyint(1) |
bool | This is the boolean type — MySQL has no other. The driver returns a bool, so do not expect 0/1. |
bigint unsigned |
decimal | It does not fit in a signed 64-bit integer. |
int unsigned |
long | Same reason, one size up. |
decimal |
decimal | |
datetime, timestamp |
DateTime | timestamp is stored UTC and converted to the session time zone; datetime is not converted at all. |
json |
string | |
blob family |
bytes |
A row count from Discover is InnoDB's estimate from index statistics and can be out by a wide
margin on a table that has not been analysed. It is always null for a view.
Procedures and functions#
A procedure returns rows simply by SELECTing — no cursor to declare, nothing to bind. This is
the capability PostgreSQL declares false, and it means the obvious thing works:
create procedure orders_by_customer(in p_code varchar(20))
begin
select * from orders where code = p_code;
end
A statement meant to be called holds the procedure's name, not SQL — orders_by_customer,
not call orders_by_customer(...). The parameters are bound by the message.
What it does not have#
This is the part worth reading before you design against it:
- No sequences.
AUTO_INCREMENTbelongs to a column, not to an object. If you need a shared counter, a one-row table is the usual stand-in. - No
MERGE. The upsert isinsert … on duplicate key update, which has different semantics. - No
RETURNING. (MariaDB has it forINSERTandDELETE; MySQL does not, and the capability list says false so that a statement written against it fails here rather than in production.) - No array type. A JSON array is the stand-in and arrives as a string.
Receiving#
An AUTO_INCREMENT id is the natural cursor. A timestamp column with
on update current_timestamp works for timestamp mode — but note that timestamp has
second resolution unless you declare timestamp(3) or finer, and two rows written in the same
second can straddle a poll boundary. Prefer the id where you have one.
What goes wrong#
| Message | What it means |
|---|---|
Table 'db.x' doesn't exist |
Wrong database, or a case mismatch on a Linux server. |
You have an error in your SQL syntax … near ':name' |
You wrote :name. Use @name. |
You do not have the SUPER privilege and binary logging is enabled |
You are trying to create a function, not to run one. The server needs log_bin_trust_function_creators=1, which is a DBA switch — Bitween never creates routines. |
| Broken pipe / "server has gone away" on the first message after a quiet period | ConnectionIdleLifetimeSeconds is above the server's wait_timeout. |
SQL Server — bitween.db.sqlserver#
Serves Azure SQL too. The configuration surprise here is encryption; the SQL surprise is TOP.
What you must set#
Host, Database, UserName, Password — and very likely TrustServerCertificate.
Encrypt defaults to true, which is a change from the old System.Data.SqlClient everyone
learned on. The modern driver encrypts by default and then validates the certificate, and the
usual on-premises instance presents a self-signed one. That combination is why an instance that
worked for years starts failing the moment something is upgraded.
- Azure SQL: leave both alone. The certificate is real and validates.
- On-premises with a self-signed certificate: set
TrustServerCertificate = true, knowing that it means an attacker between you and the server could present their own. Installing the certificate properly is better; this is the pragmatic answer.
A named instance is addressed through Host, not Port. Write SERVER\SQLEXPRESS and leave
the port alone — a named instance is resolved by the SQL Server Browser service, and supplying both
is an error the driver reports obscurely.
What the defaults do#
| Setting | Default | Leave it unless… |
|---|---|---|
Port |
1433 | You are using a named instance, in which case it is ignored. |
Schema |
the login's own | You want unqualified names to resolve to something other than the login's default (usually dbo). |
MultipleActiveResultSets |
false |
Something specifically needs it. It stops the connection being reset cleanly between uses, which is the opposite of what a shared pool wants. |
SnapshotIsolation |
false |
You want readers not to block writers. Only has an effect where the database has snapshot isolation enabled. |
A note on Schema. SQL Server has no search_path: the default schema is a property of the
login, not of the connection. Bitween applies the setting per connection with EXECUTE AS, and
failure is deliberately not fatal — impersonating a schema's owner is a privilege many service
accounts do not have. If it cannot, the connection still works and unqualified names resolve as they
would have. Qualify your names (sales.orders) and the question never arises.
Writing a statement#
-
Parameters are
@name. -
Row limit is
top N, and it goes before the column list — not at the end. This is the dialect difference people hit first:select top 100 id, order_no, total from sales.orders where customer_id = @customerId order by idoffset … fetch next … rows onlyalso works and is what paging uses, but it requires anorder by. -
Schemas are real and worth using.
dbois a default, not a rule. -
Square brackets quote an identifier:
[order].
What comes back#
| Column type | Arrives as | Worth knowing |
|---|---|---|
bit |
bool | |
decimal, money |
decimal | |
datetime2, datetime |
DateTime | Prefer datetime2; datetime has ~3ms resolution and a 1753 floor. |
datetimeoffset |
DateTimeOffset | |
uniqueidentifier |
Guid | |
nvarchar, varchar |
string | |
varbinary, rowversion |
bytes |
A declared length you see in Discover is halved for n-types, because max_length in the
catalog is in bytes and an nvarchar stores two per character — so nvarchar(50) reports 50, as
you would write it, not 100.
Row counts come from the partition statistics, which the engine maintains — cheaper and closer to true than most engines' estimates, but still an estimate.
Procedures and functions#
A procedure returns rows by SELECTing, like MySQL. Put set nocount on at the top so the
"N rows affected" messages do not arrive as extra result sets.
Three kinds of function exist and all are listed: scalar (FN), inline table-valued (IF) and
multi-statement table-valued (TF). A table-valued function is the thing to SELECT from:
select * from sales.orders_for(@code)
A statement meant to be called holds the procedure's name — sales.orders_by_customer.
What it has that the others may not#
-
MERGE, the real statement. -
OUTPUT, which does whatRETURNINGdoes — and does it forMERGEtoo:insert into sales.orders (order_no, total) output inserted.* values (@orderNo, @total) -
Snapshot isolation, which is genuinely useful for a polling receiver: a long read does not block the writers it is reading from.
Receiving#
An identity column is the natural cursor. rowversion is the SQL-Server-specific answer and is
strictly monotonic across the database — but it arrives as bytes, not a number, so it does not fit
the incrementing mode; use an identity or a datetime2 column.
What goes wrong#
| Message | What it means |
|---|---|
A connection was successfully established … but then an error occurred during the login process / certificate chain errors |
Encrypt=true against a self-signed certificate. Set TrustServerCertificate. |
Invalid object name 'x' |
Wrong schema, most often — the login's default is not what you assumed. Qualify it. |
Incorrect syntax near ':' |
You wrote :name. Use @name. |
Invalid usage of the option NEXT in the FETCH statement |
offset/fetch without an order by. Add one, or use top. |
The multi-part identifier could not be bound |
Almost always a typo in an alias or a join. |
Oracle — bitween.db.oracle#
The most configuration of the four, and the most dialect to remember. Everything here is normal Oracle; none of it is Bitween being awkward.
What you must set#
Host, exactly one of ServiceName or Sid, UserName, Password.
Both or neither is refused with a message saying so rather than guessed at. ServiceName is what a
modern Oracle wants — it is what lsnrctl services lists, e.g. FREEPDB1 or ORCLPDB1. Sid is
for an older instance that has no service name.
For RAC, Data Guard, or Autonomous Database, use ConnectDescriptor instead of host/port/
service: a full TNS descriptor or an Easy Connect string. Pair it with WalletDirectory (the
folder holding cwallet.sso) for mTLS or cloud wallets.
What the defaults do#
| Setting | Default | Leave it unless… |
|---|---|---|
Port |
1521 | |
Schema |
the login's own | Only the schema browser's default today. It does not set CURRENT_SCHEMA, so qualify names in statements. |
BindByName |
true |
Never turn this off. With it off ODP.NET binds by position, so a statement using :id twice — or one whose parameters arrive in a different order than they appear — silently binds the wrong values. Silently. |
FetchSize |
100 | Sets the driver's fetch size for the whole adapter process. The adapter multiplies the value by 1024 before applying it. |
AsSysDba |
false |
Almost never right for an integration login. |
Writing a statement#
- Parameters are
:name. This is the one engine of the four that uses a colon, and it is the most common mistake when SQL is copied between data sources. The save-time check names it explicitly: "write:coderather than@code". - Row limit is
fetch first N rows only, at the end (12c and later). Do not userownumwith anorder by—rownumis applied before the sort, so you get an arbitrary N rows and then sort those. It is the classic Oracle trap. - Identifiers are stored UPPER CASE unless they were created quoted. A table created as
ordersisORDERSin the catalog, and the schema browser will show it that way. Unquoted SQL is case-insensitive, soselect * from ordersworks regardless. - Oracle has no
booleanin SQL before 23c — a flag isNUMBER(1)orCHAR(1). Compare with= 1or= 'Y'accordingly.
select id, order_no, total
from orders
where customer_id = :customerId
order by id
fetch first 100 rows only
What comes back#
| Column type | Arrives as | Worth knowing |
|---|---|---|
NUMBER, any precision |
decimal | Every NUMBER is reported as decimal, including NUMBER(10,0). The type hint is coarse on purpose — it tells a mapper author to expect a number rather than a string, and NUMBER genuinely has more range than any binary float. |
VARCHAR2, CLOB |
string | |
DATE |
DateTime | Oracle's DATE includes a time, unlike every other engine here. |
TIMESTAMP WITH TIME ZONE |
DateTime | |
RAW, BLOB |
bytes |
A row count from Discover is ALL_TABLES.NUM_ROWS, which is null until statistics are
gathered (DBMS_STATS.GATHER_TABLE_STATS). Null means unknown, not empty.
Procedures and functions#
A procedure returns rows through an explicit SYS_REFCURSOR out parameter — this is Oracle's
answer to "a procedure that returns a result set", and the adapter binds and reads it for you:
create or replace procedure orders_for_customer(p_code in varchar2, p_rows out sys_refcursor)
as
begin
open p_rows for select * from orders where code = p_code;
end;
A statement meant to be called holds the procedure's name — orders_for_customer. Because a
name is not SQL, the save-time check accepts it with a note saying its existence was not verified;
it is resolved when called.
Receiving#
An identity column (12c+) or a sequence-backed id is the natural cursor. Note that a sequence
does not guarantee gap-free or commit-ordered values — two sessions can take 5 and 6 and commit in
the other order, so a poll between the commits can miss one. Where that matters, use a
marker column (a processed flag) rather than a cursor, which is what the marker receive mode is
for.
What goes wrong#
| Message | What it means |
|---|---|
ORA-00942: table or view does not exist |
Wrong name, wrong schema, or no grant — Oracle reports "does not exist" for an object you cannot see, which is the same message either way. Check ALL_TAB_PRIVS before assuming a typo. |
ORA-00936: missing expression |
Often @name where :name was meant. The check says so. |
ORA-01017: invalid username/password |
As it says. Note that passwords are case-sensitive from 11g. |
ORA-12514: service not registered with the listener |
ServiceName is wrong, or the database is still starting. |
ORA-01722: invalid number |
An implicit string-to-number conversion failed — usually a parameter bound as text against a NUMBER column. |
Choosing a receive mode#
Independent of engine, and the thing most worth getting right:
| Mode | Finds new rows by | Needs | Use when |
|---|---|---|---|
incrementing |
an always-growing column | cursor column | There is an id. The default answer. |
timestamp |
a modified-at column | cursor column | Rows are updated as well as inserted and you want both. |
timestamp+incrementing |
both | cursor column | Intended for rows that share a timestamp. There is no tie-break yet, so it currently behaves exactly like timestamp. |
marker |
a processed-flag column | a mark-processed statement | There is no reliable ordering, or commits arrive out of order. |
bulk |
re-reading everything | a mark-processed statement | The table is a queue that is emptied. |
None of them can see a DELETE. That is a property of polling, not of this adapter. If rows
disappear and that matters, the source needs a soft delete or an outbox.
Always order by the cursor column. Without it the "last row read" is whatever the engine happened
to return last, and the cursor will skip rows silently.
A minimal grant#
Bitween needs to read what it reads and write what it writes, plus enough catalog access for the schema browser. It never creates objects.
| Engine | Minimum |
|---|---|
| PostgreSQL | connect on the database, usage on the schema, select/insert/update on the tables used. Catalog views are readable by default. |
| MySQL | select (plus insert/update where it writes) on the database, and execute on any routine it calls. information_schema is filtered by grant automatically. |
| SQL Server | connect on the database, select/insert/update on the objects, execute on the procedures, and view definition if you want the schema browser to show things the login has no other permission on. |
| Oracle | create session, select/insert/update on the objects, execute on the routines. The ALL_* catalog views show only what the login is granted, which is why a missing grant reads as "does not exist". |
Whatever you grant, press Test connection. It reports what it connected as, what the login is actually allowed to do, and then parses every statement you have configured against the live schema — which is the quickest way to find a missing grant, a dropped column, or a statement written in another engine's dialect.