Java database driver design

Question

I have this problem where I need to design a Java package which is used for:

• Getting data from different data sources. For example, Class A will retrieve customer data from an Oracle database, while Class B will retrieve the same information from a web service data source (via SOAP ).
• The results will need to be combined, the rule for combination is quite complex, so ideally I should hide this from the users (other developers) of this package.
• When one data sources fails, I need to still return the result from other data sources. However, I also need to let the caller know one of the data sources failed to respond.

Right now I'm doing it by having a boolean value inside the Class A and Class B indicating whether there's an error, and another object for storing the actual error message. The caller will have to check this boolean value after making a call to see whether an error has occurred.

What is a good design model for this?

Answer 1

The answer would be very broad, so I would suggest you to use the:

• The Data Access Object (DAO) design pattern to abstract the source of the data (database or webservice)
• The strategy pattern to abstract the algorithm by which the data is merged (when both sources are available and one there is only one)
• And finally the state design pattern to change the way your application works depending on which source is available.
• All this wrapped (why not) in a nice facade .

This psuedo code has similar syntax as UML and Python :

// The data implements one interface
Data {interface}

// And you implement it with DatabaseData
DbData -> Data
   ...

// Or WebServiceData
WsData -> Data
   ...

// -- DAO part
Dao {interface}
   + fetch(): Data[]

// From database
DatabaseDao -> Dao
    - data: Data[0..*]
    // Query database and create dbData from rows...
    + fetch(): Data[]
        self.status = "Not ok"
        self.status = connectToDb()
        if( self.status == ok ,
            performQuery()
            forEach( row in resultSet,
                data.add( DbData.new( resultSet.next() ) )
            )
            disconnect()
        )
    ...

// From web service
WebServiceDao -> Dao
    - data: Data[0..*]
    // Execute remote method and create wsData from some strange object
    + fetch(): Data[]
        remoteObject: SoapObject = SoapObject()
        remoteObject.connect()
        if (remoteObject.connected?(),
            differentData: StrangeObject = remoteObject.getRemoteData()
            forEach( object in differentData ,
                self.data.add( WsData.new( fromElement ))
            )
        ).else(
           self.status = "Disconnected"
        )
    ....
// -- State part
// Abstract the way the data is going to be retrieved
// either from two sources or from a single one.
FetcheState { abstract }

    - context: Service
    - dao: Dao // Used for a single source

    + doFetch(): Data[] { abstract }

    + setContext( context: Service )
        self.context = context
    + setSingleSource( dao: Dao)
        self.dao = dao

// Fetches only from one DAO, and it doesn't quite merge anything
// because there is only one source after all.
OneSourceState -> FetcheState
   // Use the single DAO and fetch
   + doFetch(): Data[]
       data: Data[] =  self.dao.doFetch()
       // It doesn't hurt to call "context's" merger anyway.
       context.merger.merge( data, null )

// Two sources, are more complex, fetches both DAOs, and validates error.
// If one source had an error, it changes the "state" of the application (context),
// so it can fetch from single source next time.
TwoSourcesState -> FetcheState
    - db: Dao = DatabaseDao.new()
    - ws: Dao = WebServiceDao.new()

    + doFetch(): Data[]
        dbData: Data[] =  db.doFetch()
        wsData: Data[] =  ws.doFetch()

        if( ws.hadError() or db.hadError(),
            // Changes the context's state
            context.fetcher = OneSourceState.new()
            context.merger = OneKindMergeStrategy.new()
            context.fetcher.setContext( self.context )
            // Find out which one was broken
            if( ws.hadError(),
                context.fetcher.setSingleSource( db )
            )
            if( db.hadError(),
                context.fetcher.setSingleSource( ws )
            )
        )
        // Since we have the data already let's 
        // merge it with the "context's" merger.
        return context.merger.merge( dbData, wsData)

// -- Strategy part --
// Encapsulate algoritm to merge data
Strategy{ interface }
    + merge( a: Data[], with : Data[]  )

// One kind doesn't merge too much, just "cast" one array
// because there is only one source after all.
OneKindMergeStrategy -> Strategy
    + merge( a: Data[], b: Data[]  )
         mergedData: Data[]
         forEach( item, in( a ),
            mergedData = Data.new( item ) // Take values from wsData or dbData
         )
         return mergedData

// Two kinds merge, encapsulate the complex algorithm to
// merge data from two sources.
TwoKindsMergeStrategy -> Strategy
    + merge( a: Data[], with: Data[] ): Data[]
        forEach( item, in( a ),
            mergedData: Data[]
            forEach( other, in(with ),
                 WsData wsData = WsData.cast( item )
                 DbData dbData = DbData.cast( other )
                 // Add strange and complex logic here.
                 newItem = Data.new()
                 if( wsData.name == dbData.column.name and etc. etc ,
                    newItem.name = wsData+dbData...e tc. etc
                    ...
                    mergedData.add( newItem )
                 )
            )
        )
        return mergedData

// Finally, the service where the actual fetch is being performed.
Service  { facade }

    - merger: Strategy
    - fetcher: FetcheState

    // Initialise the object with the default "strategy" and the default "state".
    + init()
        self.fetcher  = TwoSourcesState()
        self.merger = TwoKindsMergeStrategy()
        fetcher.setContext( self )

    // Nahh, just let the state do its work.
    + doFetch(): Data[]
        // Fetch using the current application state
        return fetcher.doFetch()

Client usage:

     service: Service = Service.new()
     service.init()
     data: Data[] = service.doFetch()

Unfortunately, it looks a bit complex.

OOP is based a lot on polymorphism.

So in Dao , you let the subclass fetch data from whatever place and you just call it dao.fetch().

In Strategy the same, the subclass performs one algorithm or the other (to avoid having a lot of strange if 's, else 's, switch 's, etc.).

With State the same thing happens. Instead of going like:

if isBroken and itDoesntWork() and if ImAlive()

etc., etc. you just say, "Hey, this will be the code one. There are two connections and this is when there is only one.".

Finally, facade say to the client "Don't worry, I'll handle this.".

Answer 2

Do you need to write a solution, or do you need a solution? There's plenty of free Java software that does these things - why re-invent the wheel. See:

• Pentaho Data Integration (Kettle)
• CloverETL
• Jitterbit

Answer 3

I would suggest a Facade that would represent the object as a whole (the customer data) and a factory which creates that object by retrieving from each data source and passing those to the Facade (in the constructor or as a builder, depending on how many there are). The individual class with the specific data source would have a method (on a common interface or base class) to indicate if there was an error retrieving the data. The Facade (or a delegate) would be responsible for combining the data.

Then the Facade would have a method that would return a collection of some sort indicating which data sources the object represented, or which ones failed - depending on what the client needs to know.