rabbitmq - Message Ordering Across Queues -
i have scenario in rabbitmq setup i'm curious how solve. diagram below illustrates (exchanges , queues removed succinctness):
scenario
- producer creates message a(1), received top consumer, begins processing message.
- producer creates message a(2), received bottom consumer (assuming both consumers on round-robin exchange).
- the bottom consumer publishes message b(2), put message b consumer's queue
- the poor slow top consumer finishes , emits message b(1).
problem
if assume b consumer cannot made idempotent, how ensure result of both b messages applied in correct order?
i had thought of using timestamp applied initial publish of message a, , having consumer maintain timestamp of last change, rejecting timestamps before time, works if each message causes exact same kind of change , requires lot of tracking.
other ideas how approach appreciated. thanks!
i not sure specific rabbitmq here, idea timestamps sounds start if have single producer.
the producer attaches timestamp messages a, each message b take same timestamp of respective message a.
with approach messages not processed, eg, message b(1). if messages should processed consumer b, should processed in deterministic order, can deterministic merge:
consumer b equipped 2 queues, 1 queue each consumer a. consumer b checks top of both queues:
- if both queues non-empty, consumer b pops message lowest timestamp.
- if @ least 1 queue empty, consumer b waits.
with approach order in consumer b processes messages given timestamps of producer , no message discarded. assumptions are:
- queues fifo
- no process crashes
- always case each consumer processes message
- consumer b can check top of queues in non-blocking fashion