java - A Scalable Architecture for Reconstructing events -

i have been tasked develop architecture data transformation pipeline. essentially, data comes in @ 1 end , routed through various internal systems acquiring different forms before ending in destination.

the main objectives -

• fault tolerant. message should recoverable if 1 of intermediate systems down.
• replay/ resequence - message can replayed stage , should possible recreate events in idempotent manner.

i have few custom solutions in mind address

• implement checkpoint system message can logged @ both entry , exit points @ each checkpoint know failure happens.
• implement recovery mechanism can go logged storage ( database, log file etc.. ) , reconstruct events programmatically.

however, have feeling standard problem defined solutions.

so, welcome thoughts on suitable architecture go with, tools/packages/patterns refer etc..

thanks

akka obvious choice. of course scala version more powerful, java bindings can achieve lot.

i think can follow cqrs approach , use akka persistence module. in case it's easy replay sequence of events, because have persistent journal.

generally actor model provides fault-tolerance using supervision.

akka clustering give scalability need.

really awesome example of using akka clustering akka persistence , cassandra - https://github.com/boldradius/akka-dddd-template (only scala unfortunately).