Marketplace Fan-in

The obvious endpoints of this flow — the ERP/WMS inventory master on one side, the Amazon Seller API on the other — are the two places least likely to surprise you, and that is exactly why they deserve your attention last. The inventory master is authoritative by definition; whatever it says a quantity is, it is. The Seller API is a published contract with documented behavior. Neither of those is where a marketplace flow goes wrong. The real complexity lives in the Marketplace Sync Hub, because it is the only component in this chain doing two structurally different jobs in opposite directions: pushing inventory state outward to Amazon, and pulling incoming marketplace orders back inward for import into the ERP. Those two jobs share a component but not a rhythm. Inventory pushes are a projection of a moving internal number onto an external listing; order pulls are a claim against that same number made by a buyer who acted on whatever the listing said at the moment they looked. The hub is where a stale push and a fresh order meet, and it is the only place in the flow that can be internally inconsistent with itself.

That means a QA engineer's first questions belong to the hub's timing and its handling of repetition, not to endpoint correctness. What is the window between a quantity changing in the ERP/WMS master and that change being reflected through the Seller API — and what happens to orders that arrive inside that window against stock that no longer exists? What does the hub do when the same incoming marketplace order is fetched twice, or when order import into the ERP succeeds downstream but the hub never registers the acknowledgment? Those failures do not present as errors at either endpoint; the inventory master will look correct, the Seller API will report a valid order, and the defect will surface as a duplicated or unfulfillable record inside the ERP. Test the seam that reconciles two directions of truth, and the endpoints largely take care of themselves; test the endpoints first, and you will confirm that both halves work while the flow between them quietly does not.

The failure most likely to blindside a team is the per-channel rate limit that's hit while every other channel stays current. The assumption underneath it — the one that feels obviously true right up until it isn't — is that the hub has a single health state, so if the pipeline is up and data is flowing, all channels are flowing. Fan-in architecture quietly breaks that assumption: health is per-channel, not per-hub. A rate-limited channel doesn't throw the errors people watch for. It doesn't exhaust the connection pool, it doesn't hold a long transaction and create lock contention, it doesn't land anything in the DLQ to accumulate un-alerted. It just falls behind, and the aggregate looks fine because the other channels are answering for it. The same shape shows up in the parser that mis-maps a field for one marketplace after its response format changes, and in the price/stock push that partially succeeds and leaves one channel out of sync with the master: single-channel damage hidden inside a healthy-looking whole.

What that means for testing is that any test which exercises the hub with uniform load across channels will pass while the real defect goes undetected. You have to test asymmetrically — drive one channel past its rate limit while the others stay under, change one marketplace's response format and leave the rest alone, let one push partially succeed — and then assert per-channel, not on the aggregate. The assertions that matter are freshness and correctness scoped to a single channel measured against the master: is this channel's data as current as the master, and does each field still map to what it mapped to before. Compare that to replication lag producing stale reads or messages expiring on TTL during consumer downtime, which degrade everything at once and are correspondingly easier to notice. A failure that spares most of the system is the one that survives to production.

Picture SKU `WID-4417-BLK` in the ERP inventory master: `on_hand: 42`, `allocated: 6`, `available: 36`, `list_price_cents: 2999`, `updated_at: 2024-06-11T09:14:02Z`. The Marketplace Sync Hub reads that row, projects it into Amazon's shape — `SellerSKU: WID-4417-BLK`, `Quantity: 36`, `StandardPrice: {Amount: 29.99, CurrencyCode: USD}` — and submits it through the Amazon Seller API as a feed submission, getting back an acknowledgment the hub records against the push. Separately and asynchronously, orders come back the other way into the Incoming Marketplace Orders queue: Amazon posts `AmazonOrderId: 113-4429981-7752233` with a line for `WID-4417-BLK`, quantity 2, which lands alongside orders arriving from the other channel's feed on the same queue.

The dangerous version is the quiet one. Suppose Amazon's release renames or renests the price node, and the hub's parser for that channel now reads a null and maps `StandardPrice` to zero, or the submission for `WID-4417-BLK` partially succeeds — quantity applied at 36, price not applied — so Amazon keeps selling at a stale figure while the master says 29.99 and nobody sees an error. Layer on the per-channel rate limit: the hub gets throttled mid-batch, the remaining SKUs in that push silently fall behind while the second marketplace stays current, and a replica read during that window returns the pre-update `available: 42`, re-pushing a number that was already wrong. On the inbound side, if the consumer times out mid-processing, Amazon's redelivery of `113-4429981-7752233` can land twice, and because the other channel numbers its orders independently there is no guarantee its ID space won't collide with Amazon's. Test it by pushing `WID-4417-BLK` and then reading back the live Amazon offer to confirm both quantity **and** price landed as a pair, not just that the API returned 200 — assert the applied price is `29.99` and fail loudly on zero or null rather than accepting the parse. Force a throttle by driving a batch past the channel's rate limit and assert the hub surfaces the lag for that channel specifically instead of reporting success; re-read `available` from the replica immediately after the master update and confirm the hub doesn't push stale `42`. For the queue, submit `113-4429981-7752233` twice and assert exactly one order is created, then submit a second-channel order deliberately carrying the string `113-4429981-7752233` as its native ID and assert both survive as distinct orders — duplicate suppression must key on channel plus ID, not ID alone.

Follow `113-4429981-7752233` past the queue and into the ERP itself. The consumer picks the message up, maps it to an ERP sales order — `channel: amazon`, `external_order_id: 113-4429981-7752233`, one line for `WID-4417-BLK` at quantity 2, unit price 2999 cents — and posts it into the ERP's order staging table, where it sits as `status: STAGED` waiting for the ERP's own import job to promote it into a real order that decrements `allocated` from 6 to 8 and `available` from 36 to 34. The ERP returns a staging id, the hub records it against the message, acks the queue, and the message is gone. That ack is the whole risk: from the hub's point of view the order is delivered, and everything after it belongs to a process the hub never watches.

What goes wrong is that `STAGED` is a resting state, not a transient one. If the import job chokes on the line — say the ERP has no price field on the inbound schema and quietly stamps its own `list_price_cents: 2999` from the item master, which happens to be right today and wrong the moment Amazon honors a promo price, or it defaults a missing warehouse code to the primary DC when the unit ships from the overflow site — the record either sits at `STAGED` forever or promotes with values nobody sent. Meanwhile `allocated` never moves off 6, so the next push tells Amazon `Quantity: 36` when only 34 are really sellable, and the oversell traces back to an order that the hub logged as successfully delivered. And because the queue redelivery from CP1 can hand the consumer `113-4429981-7752233` twice, two staged rows can exist under one external id, waiting to promote into two orders for four units. Test it by importing `113-4429981-7752233` and then asserting, on a timer, that it leaves `STAGED` and reaches a promoted status — a test that only checks "ERP accepted the record" passes on a row that never moves. Read back the promoted order and assert the line price and warehouse match what was sent rather than whatever the item master would have supplied, seeding a deliberately non-default price so a target-side default is visibly wrong instead of coincidentally right. Then assert `allocated` on `WID-4417-BLK` actually went 6 → 8 as proof the promotion had its inventory effect. Finally, submit `113-4429981-7752233` twice through to staging and assert exactly one staged row survives keyed on channel plus external id, and that only one promotion occurs — deduplication at the queue is not the same guarantee as deduplication in the staging table.