Recently I have started with the design and implementation of a new tool which basically is nothing more than a Data import - export application.
From functionality point of view, this application reads data from various types of resources (being cvs files, xml files, databases, ftp feeds), stores them in an internal datastore after running a set of simple business rules on them, and offers this data to external systems that fetch data from this application (using the API this tool provides to offer data to customer applications).
So simply set, following are the core functionality this application provides:
1- Read data from various resources
2- Store data in internal data store after running business rules
3- Offers the stored data to other external systems
From design point of view, following are important:
1- loose coupling between the Import and Export modules.
2- It should be easy to extend the Import Module with new data sources (new databases, data feeds based, etc...)
3- It should be easy to attach new subscribers to the Export module.
I am planning to use Spring to setup the project (project configuration) and also make use of Spring components where possible /applicable (for example AOP to define a business rule engine, do the logging and exception handling)
Any suggestions on the technologies, design patterns, and project setup? Has anyone had experience with something similar and can provide some suggestions, remarks, possible bottlenecks, tips and tricks?
Any suggestion is appreciated,