The problem
Every custodial exchange starts the same way. You pick a custody provider, you integrate, you ship. The vendor's SDK is right there in your withdrawal service, its asset symbols are in your database, and its webhook payload shape is the thing your event handlers parse. It works, and for a while there is no reason to think about it again.
Then something forces the question. Operational-resilience rules start asking how a licensed venue would continue operating if a critical outsourced provider failed — VARA's rulebooks are explicit about this, and they are not unusual in that. Or the provider does not support an asset the desk needs. Or their pricing changes. At that point you discover that "add a second custody provider" is not an integration task. It is a refactor of everything that touches money.
Here is how to avoid that, and how to dig out if you are already in it
The approach
- The coupling is deeper than the SDK
The instinct is that coupling means import { CustodySdk } from '@vendor/sdk' appearing in too many files, and that the fix is a wrapper class. It is not. By the time a custody integration is a year old, the vendor has leaked into your system in at least five places, most of which are not code you would think to search:
Identity. The provider has its own account model — vaults, sub-accounts, wallets, whatever they call it — and you have probably stored their identifier on your user record. Your notion of "this user's account" is now defined by a vendor.
Asset identity. Providers do not agree on what an asset is called. One says USDC, another says USDC_ETH, a third wants a chain identifier plus a contract address. If a provider's symbol is a key anywhere in your ledger, you have made that vendor structural.
Transaction status. Every provider has its own status enum, its own confirmation thresholds, and its own idea of which states are terminal. Their success status is final in their model, which is not the same as final on the chain.
Fees. One provider quotes an estimate you pass through; another deducts from the transfer; another bills you out of band. This one reaches all the way into your pricing.
Webhooks. Payload shape, signature scheme, delivery guarantees, retry behaviour, ordering. Vendors differ on all five, and webhook handlers are usually the least-abstracted code in the whole system.
A wrapper class fixes the first of these and none of the others.
- Derive the interface from your ledger, not from their API
The common failure mode is a "custody abstraction" that is really a transcript of the first provider's API with the names changed. It looks like an abstraction. The moment you write the second adapter you find that half the methods do not translate, because they were never yours — they were vendor concepts wearing a generic name.
The interface has to come from the other direction. Ask what your ledger actually needs from a custodian, in the language of your own domain, and write only that. Call it the custody port:
export interface CustodyProvider {
readonly id: ProviderId;
readonly capabilities: ProviderCapabilities;
// Give me somewhere this account can receive this asset.
provisionDepositAddress(
account: AccountRef,
asset: AssetId,
): Promise<DepositAddress>;
// Move value out, keyed by an intent id we generated.
submitWithdrawal(intent: WithdrawalIntent): Promise<Transfer>;
// Recover our own submission when we don't know whether it landed.
findByIntentId(intentId: IntentId): Promise<Transfer | null>;
// The custodian's account of a transfer it already knows about.
getTransfer(ref: ProviderTransferRef): Promise<Transfer>;
// What the custodian says it is holding, for reconciliation.
getBalances(account: AccountRef): Promise<Balance[]>;
// Verify and normalise their webhook — raw body and headers, not JSON.
parseWebhook(raw: RawWebhook): Promise<CustodyEvent[]>;
}
Six methods. Notice what is absent: no vault creation, no policy configuration, no staking, no whitelist management. Those are real features, but they differ so much between vendors that abstracting them produces a lowest common denominator that serves nobody. Leave them out of the port and reach for the vendor SDK directly in the one operational module that needs them. Over-abstraction is a worse outcome than honest coupling in a corner you have chosen.
Where providers genuinely differ in ways your business logic must respect, do not hide it — declare it. A capabilities descriptor (supportsMemoTags, feeModel: 'deducted' | 'added' | 'billed', confirmationsBeforeCredit) lets the routing layer decide explicitly, which is far better than an adapter quietly pretending to support something it does not.
- Own the mappings
Three translation tables sit between your domain and every provider, and all three belong to you.
Accounts. A custody_accounts table mapping (account_id, provider_id) → provider_account_ref. Your user record never holds a vendor identifier again.
Assets. A canonical asset registry — your ID, the chain, the contract address, decimals, your confirmation threshold — with a per-provider symbol column. Every adapter translates at its boundary. USDC_ETH never appears in a ledger row.
Status. One state machine that you define, that the adapters map into:
type TransferStatus =
| 'INTENT_RECORDED' // ours, before the provider knows anything
| 'SUBMITTED' // provider accepted it
| 'BROADCAST' // on-chain, has a hash
| 'CONFIRMED' // enough confirmations by our policy, not theirs
| 'SETTLED' // ledger posted, terminal
| 'FAILED' // terminal
| 'UNKNOWN'; // we asked and could not tell — recover, then page
Two of those carry weight. CONFIRMED is set by your threshold from the asset registry, because the vendor's threshold was chosen for their risk appetite and not yours. And UNKNOWN earns its place: providers time out, and an adapter that maps ambiguity onto FAILED will eventually cause a double spend. Ambiguity should be loud, and it should be a state rather than an exception.
Nothing reverses out of SETTLED. If a deep reorg orphans a settled transfer, the answer is a compensating ledger entry, not a status change — the same as any other posted error in a book of account.
- Idempotency is the actual hard part
This is where abstraction stops being an architecture exercise and starts protecting money.
The dangerous window is between "we told the provider to send funds" and "we know whether it worked." A network timeout there does not mean the withdrawal did not happen. It means you do not know. If a retry is a fresh API call with a fresh identifier, you have just sent the customer's money twice.
It is worth being precise about what custodians actually give you here, because the naive mental model is wrong. Vendor external-reference fields — Fireblocks' externalTxId, BitGo's sequenceId — do not replay the original response on a duplicate submission. They reject it. Recovery is a separate lookup by your own key. Generic Idempotency-Key headers, where offered, do replay a stored response, but typically only for around 24 hours — and a sweeper finding a stuck transfer on day three has no vendor protection left at all.
Which is exactly the argument for your ledger being the record. Your intent outlives their idempotency window.
So: the intent is written down before the provider is contacted, the identifier is one the caller generated, and the provider is pinned to the intent so that a retry cannot route somewhere new.
async function withdraw(cmd: WithdrawCommand): Promise<Transfer> {
// 1. Our record, committed before any network call. Upsert, so a
// retry returns the existing row instead of a duplicate-key error.
const intent = await ledger.upsertWithdrawalIntent({
intentId: cmd.idempotencyKey, // deterministic, supplied by the caller
accountId: cmd.accountId,
asset: cmd.asset,
amount: cmd.amount,
destination: cmd.destination, // address plus memo/tag
travelRule: cmd.travelRule, // beneficiary data, required before submit
status: 'INTENT_RECORDED',
});
// 2. Pin the provider to the intent. Re-running the router on a retry
// is how a failover turns into a double send.
const providerId = intent.providerId
?? await ledger.pinProvider(intent.intentId, router.select(intent));
const provider = registry.get(providerId);
// 3. Replay protection is a read, not a hope.
if (intent.providerTransferRef) {
return provider.getTransfer(intent.providerTransferRef);
}
try {
const transfer = await provider.submitWithdrawal(intent);
await ledger.recordSubmission(intent.intentId, transfer); // → SUBMITTED
return transfer;
} catch (err) {
// 4. We do not know. Record that, and let recovery resolve it.
await ledger.markUnknown(intent.intentId, err);
throw new WithdrawalOutcomeUnknown(intent.intentId, { cause: err });
}
}
The UNKNOWN rows are then resolved out of band, never in the request path:
const found = await provider.findByIntentId(intent.intentId);
found
? await ledger.recordSubmission(intent.intentId, found)
: await ledger.releaseIntent(intent.intentId); // safe to resubmit
If a provider offers no way to look up a transfer by your own reference, that is not something an adapter can paper over. It is a fact the routing policy needs to know, and it may be a reason not to send withdrawals through that provider at all.
- Webhooks are hints; the provider's read is the attestation
Treat every inbound webhook as an unreliable, unordered, possibly duplicated, possibly forged suggestion that something changed. Verify its signature in the adapter — against the raw body, before parsing, which is the classic way this goes wrong — normalise it into your event type, and use it only to trigger a fetch. The state transition is then driven by getTransfer, not by the payload.
That is more work, and it removes an entire class of incident: replayed webhooks, out-of-order delivery, a "confirmed" event arriving after a "failed" one. It also means a provider that drops webhooks entirely degrades into a system that is merely slower, rather than one that is wrong.
Behind that, run a sweeper over anything non-terminal past a threshold. It will find things — stuck transfers are a property of distributed systems talking to chains, not evidence of a bad integration.
- Reconciliation is the accounting part
I came to engineering from accounting, and the habit that transferred most directly is this: the ledger is the system of record. The custodian's balance is an external attestation you reconcile against it, never a source you copy from.
Run a scheduled job. Per account and asset, it compares your ledger balance against getBalances() and writes a reconciliation record with the difference. In-flight transfers explain most breaks; anything else is an alert. It is unglamorous, and it is the single control that catches adapter bugs early — a mis-mapped decimal or a missed status transition shows up as a break the same day, rather than as a support ticket a month later.
- Routing, and the migration you can actually run
With the port in place, choosing a provider becomes one policy decision rather than a hundred conditionals. A router takes the asset, the flow, the amount and the capability flags, and returns a provider. Two rules matter:
Deposits are sticky. An address belongs to the provider that generated it. With an MPC custodian the key material cannot leave, so there is no migrating it — you can only stop issuing new addresses from that provider and keep watching the old ones indefinitely.
Withdrawals are routable. This is where a second provider earns its keep: failover, liquidity, asset coverage.
The migration follows. Write adapter two. Run it in shadow — reconcile against it, submit nothing. Then move a low-volume asset's new deposit addresses to it while adapter one keeps serving its existing ones. Then withdrawals for that asset. Then widen. Balances drain rather than cutting over, and there is no flag day — though it is worth being honest that the old relationship never fully ends, because those addresses stay live for as long as customers have them saved.
What makes this safe is a conformance suite: one set of tests written against the port, run against every adapter, plus an in-memory fake for CI and local development. Real adapters run it against the vendor's sandbox, never mainnet.
describe.each(adapters.map((a) => [a.id, a] as const))(
'CustodyProvider conformance: %s',
(_id, provider) => {
it('resolves a submitted intentId through findByIntentId', ...);
it('never reports terminal below our confirmation threshold', ...);
it('rejects a webhook whose signature fails against the raw body', ...);
it('round-trips every registry asset it claims to support', ...);
},
);
A new provider is production-ready when it passes. That is a better answer to give a regulator, or a board, than "we think so."
Ledgers must balance. State must reconcile. The custodian is just one more counterparty you check your books against.
Written from public vendor documentation, published regulation, and general industry practice. It does not describe the architecture, controls or operations of any employer or client. Views are my own.