Context
We’ve been building UBLP’s Incoterms Escrow module ( UBLP/modules/incoterms-escrow at main · ekacin/UBLP · GitHub ) — a Compact contract that holds shielded NIGHT in escrow for trade deals (buyer locks funds, an independent party attests a milestone, seller gets paid — amount and parties stay private on-chain). Getting the release step working end-to-end against a real devnet surfaced a real, undocumented gap. Posting it here in case it saves someone else the same trial-and-error, or in case someone already has a better answer than ours.
The gap: a contract-owned coin’s real mtIndex is unobtainable through any documented API
When a contract holds a shielded coin (via receiveShielded) and a later circuit needs to spend it (sendShielded), you have to supply a QualifiedShieldedCoinInfo with the coin’s real Merkle-tree position (mtIndex). The obvious way to get this is publicDataProvider.queryZSwapAndContractState(contractAddress), which returns a ZswapChainState whose firstFree field is supposed to reflect the next free/appended tree position.
For a contract-owned coin, firstFree comes back as 0 — always, no matter how long you wait — even though the coin is genuinely sitting at a real, non-zero index elsewhere in the tree (we observed real indices of 46, 51, 56, 61, 66, 75, 79, 82 across separate runs, deployment after deployment). Trusting firstFree fails at proof time with:
“invalid index into sparse merkle tree: 0”
We checked the winning submission for the related content bounty ([Tutorial] Accepting Token Deposits into a Contract: ReceiveShielded & Escrow Patterns · Issue #288 · midnightntwrk/dApp-idea-board · GitHub) (a tutorial on exactly this receiveShielded/escrow pattern) — even that explicitly says “you cannot set mt_index manually… the blockchain assigns that index when the transaction is confirmed” and then never explains how to retrieve it afterward. So this doesn’t look like something we’re just missing in the docs — it looks like a genuine, still-open gap.
Only working method we found: deserialize the raw indexer state and dump it with ZswapChainState.toString(true) (a debug method, not a public API), then regex the contract’s own commitment entry out of the human-readable output. Filed as servicedesk#187 ( [Bug]: queryZSwapAndContractState's firstFree is always 0 for contract-owned shielded coins, making the real mtIndex unobtainable through any documented API · Issue #187 · midnightntwrk/servicedesk · GitHub ), with a full runnable repro against a real devnet if anyone wants to dig in or has a cleaner answer.
Related, already-tracked issue (not new — just a pointer)
If you’re also trying to convert unshielded NIGHT into shielded NIGHT via WalletFacade.initSwap and hitting InvariantViolation(NightBalance(…)) — that’s already a known issue, tracked at midnight-wallet#554 ( WalletFacade.initSwap returns partial transactions for mixed shielded/unshielded swaps · Issue #554 · midnightntwrk/midnight-wallet · GitHub ). We hit it too, worked around it by funding test wallets shielded-to-shielded from an already-shielded wallet instead of converting. Just flagging it here since it doesn’t seem widely known outside the tracker — saw the same question go unanswered in Discord a few times.
An open question, unrelated to the above
Is ZswapSecretKeys’s spend/view key split real — i.e. independently derivable, the way Zcash Sapling’s spending/viewing keys are? Checked up @midnight-ntwrk/ledger-v9@1.0.0-rc.4 — CoinSecretKey and EncryptionSecretKey are separate classes now, but both have private constructor(), so ZswapSecretKeys.fromSeed() is still the only way to get either, always together. Curious if independent derivation is planned, or if the Sapling comparison is more about the selective-disclosure story than a literal wallet-key capability.