rabbitmq - Message Ordering Across Queues -


i have scenario in rabbitmq setup i'm curious how solve. diagram below illustrates (exchanges , queues removed succinctness):

enter image description here

scenario

  1. producer creates message a(1), received top consumer, begins processing message.
  2. producer creates message a(2), received bottom consumer (assuming both consumers on round-robin exchange).
  3. the bottom consumer publishes message b(2), put message b consumer's queue
  4. the poor slow top consumer finishes , emits message b(1).

problem

if assume b consumer cannot made idempotent, how ensure result of both b messages applied in correct order?

i had thought of using timestamp applied initial publish of message a, , having consumer maintain timestamp of last change, rejecting timestamps before time, works if each message causes exact same kind of change , requires lot of tracking.

other ideas how approach appreciated. thanks!

i not sure specific rabbitmq here, idea timestamps sounds start if have single producer.

the producer attaches timestamp messages a, each message b take same timestamp of respective message a.

with approach messages not processed, eg, message b(1). if messages should processed consumer b, should processed in deterministic order, can deterministic merge:

consumer b equipped 2 queues, 1 queue each consumer a. consumer b checks top of both queues:

  • if both queues non-empty, consumer b pops message lowest timestamp.
  • if @ least 1 queue empty, consumer b waits.

with approach order in consumer b processes messages given timestamps of producer , no message discarded. assumptions are:

  • queues fifo
  • no process crashes
  • always case each consumer processes message
  • consumer b can check top of queues in non-blocking fashion

Popular posts from this blog