Sending a lot of data from one system to another...


XML & Web services: Sending a lot of data from one system to another...

  1. We have a functional need where n records from a table needs to be sent from one system to another. I'm sure that wrapping the data in XML is the way to go. Beyond that, a nice "shove" would be nice. :-)

    Would you use SOAP to get it from one system to the other? Would you just use JDom's ability to output to a stream to get it from one system to another? What would you do?
  2. really depends on what kind of data that needs to be shared between two systems, It can just be a XML stream rather than a entire document(DOM Stream) or a object mapped to XML Data..
    If data is not very complex in structure with nodes and stuff like that, then send a simple ansi text which can be read in all kinds of system.

    SOAP is basically for invoking methods over the wire just like XML over rpc.

    Its depends what kind of data really.

  3. Although SOAP is for RPC calls, it also has a way of attaching messages. I never used it. May be worth looking into.
    The simplest solution that comes to mind is to export data into a text file in tabular format, ftp the file and let the other system import the data into the database.


  4. I concur with Pranab, the easiest way is for you to just FTP your data over (EDI style).

    Now with that out of the way, I am sure that you have a very compeling reason for doing it the XML way using SOAP. One thing to keep in mind when dealing with large volumes of data is to make sure that your SOAP/WS container is able to digest large datasets without dying. In the past, I've seen catalogue updates in 100 MB range that required changes in our underlying container. We using messaging-style because we wanted to defer processing until the entire payload was received and do some preliminary validation before processing. (e.g. why process a huge document if the document is invalid due to signature problems or the message digest doesn't sum up properly).