在运行时配置“管道”

Question

This is kind of a design-patterns question in Java.

I am designing a java .jar file to act as a frameworks for managing and processing a certain form of data. I want the end user to be able to dictate the "plumbing" configuration, within certain constraints, in a certain way. The pieces are producers and/or consumers, and I know how to implement them, but the connections are confusing to me... here's a nonsense example that sort of parallels my application.

Suppose I have implemented these elements:

• AppleTree => produces apples
• ApplePieMaker => consumes apples, produces apple pies
• ApplePress => consumes apples, produces apple cider
• AppleSave => stores apples, apple pies, or apple cider into a file
• AppleLoad => "reconstitutes" apples, apple pies, or apple cider from a file that was produced by AppleSave
• ApplePieMonitor => displays apple pies on the screen in a GUI format, as they are produced

Now I want the user to be able to specify things like:

• AppleTree | ApplePress | AppleSave cider1.sav AppleTree | ApplePress | AppleSave cider1.sav (produce apples, make them into cider, save them to a file)
• AppleTree | AppleSave apple1.sav AppleTree | AppleSave apple1.sav (produce apples, save them to a file)
• AppleLoad apple1.sav | ApplePieMaker | ApplePieMonitor AppleLoad apple1.sav | ApplePieMaker | ApplePieMonitor (take saved apples, make them into pies, display the results on the screen in a GUI)

• (not sure how to illustrate this, but might be specified as follows)

  AppleTree tree1, ApplePieMaker piemaker1 < tree1, AppleSave apples.sav < tree1, AppleSave @select(*.sav) < piemaker1, ApplePress press1 < tree1, AppleSave cider.sav < press1, ApplePieMonitor piemon1 < piemaker1

(produce apples, make them into pies and cider, save the apples, pies and cider to separate files, with the pies going to a file selected by the user at runtime and the others going to predetermine files, and also display the pies on the screen in a GUI)

So I have a rough idea of how to architect a configuration file: namely to structure the thing into elements that have at most 1 input and at most 1 output, and then for each instantiated element, name it, and if it has an input, specify the name of the element providing the input.

What I'm unclear is how to go about coupling the elements of the program when they run. Maybe the way to go is to have a number of interfaces, like AppleConsumer , ApplePieConsumer , etc. so an ApplePieMaker would implement the AppleConsumer interface (incl. method consumeApple() ), and an AppleTree would implement an AppleProducer interface which can register consumers at startup, so that every time the AppleTree produces an apple, it has a list of its consumers and calls consumeApple() on each of them, which then does the right thing without the AppleTree having to know what they are doing with the apples....

Any suggestions? Does this sort of thing have a name? I'm not really that experienced in design patterns.

edit: my end users don't know and don't care about Java. They just need to be able to set up a config file (which I'm trying to make as simple as possible so I can give them some good examples) and run my program which will read the config file, construct the elements, hook them together, and go. All the elements are under my control, I don't need to support plugins, so I don't have to be super-general.

Answer 1

I had that problem a while ago, and it was a bit hard to specify cleanly in Java, since my input and output could be plural. However, when you are sure to have a single input and output (since a file is a specific kind of output, right?), you could try to use checked generics.

A processing step implementation (I will call that a filter ) has two checked type arguments: input and output (lets for instance suppose they all extend a common interface, and for every new type you inject in the system, you will subclass that interface).

public interface Filter<Input extends Type, Output extends Type> {

  public Class<Input> getInputType();

  public Class<Output> getOutputType();

  public void process(Input in, Output out);

}

A filter chain is then just an array of (compatible) Filter s. By compatible, I intend that for each filter, its Output is the same type as its follower Input , the first filter has an Input that match your overall input type, and the last filter has an Output that matches the expected result type. This is easy to validate, in practice, since we are using checked generics.

A filter chain is thus an other (compound) filter. The compatibility of compounded filters should be checked in the constructor, and the array of compounds final. There is no accurate way to express the "chaining" property (compatibility) of the arguments of that constructors with generics, so you are going to have to do that with bare types, which is a bit unclean.

An other way to do it that gets around this limitation, at the cost of more cumbersome writing, is to change the definition of a filter like this:

public interface Filter<Input extends Type, Output extends Type> {

   public Class<Input> getInputType();

   public Class<Output> getOutputType();

   public Output out process(Input in);

}

We will then have to define a compound filter as an imbrication of filter pairs, thus defined:

public class CompoundFilter<Input extends Type, Output extends Type>
       implements Filter<Input extends Type, Output extends Type> {

  private final Filter<Input extends Type, ? extends Type> l;
  private final Filter<Input extends Type, ? extends Type> r;

  public <Median extends Type> CompoundFilter(
         Filter<Input, Median> r,
         Filter<Median, Output> l
      ) {
      this.l = l;
      this.r = r;
  }

  @SuppressWarnings("unchecked")
  public Output out process(Input in) {
      // Compute l(r(in)) = (l o r) (in)
      return ((Output<Input,Type>) l).process(r.process(in));
  }
}

Thus, composing filters is just a matter of writing:

Filter<A,B> f1 = new FilterImpl<A,B>;;
Filter<B,C> f2 = new FilterImpl<B,C>;
// this is mathematically f2 o f1
Filter<A,C> comp = new CompoundFilter<A,C>(f1,f2);

Answer 2

I could not help it, I have to workout something for this.

So here it is.

You already have the idea about the apple producer/consumer so this is how I would do it.

Create three interfaces and implement as follows:

• Product - Either Apple, AppleCider, ApplePie,
• Producer - AppleTree, ApplePress, ApplePieMaker, AppleLoad,
• Consumer - ApplePress ( consumes Apples ), ApplePieMaker ( consumes Apples ) , AppleMonitor, AppleSave.

The idea is to have a generic Product, produced by generic Producers and consumed by generic Consumers.

Once you have that, you can create the configuration file pretty much the way you describe it, and parse it to create a new instance for each element.

 element1 | element2 | element3 <parameters> | element 4

In a map you create the element name and map it to the class that will create the new instance.

Let's say

map.put( "AppleTree", YouAppleTreeClass.class );

So each time you read an element in the configuration you create the instance:

for( String item: line )  { 
    Object o = map.get( item ).newInstance();
}

Finally you have to validate the structure of your configuration, but basically it could be like this:

• The first element should be a producer
• The last should be a consumer
• Any intermediate should be producer-consumers
• You can parse arguments needed ( file to save data for instance )

Once you have all your objects created and chained, you start producing.

There are some thing you have to workout but they're pretty easy:

1. Argument passing ( the file where they will be saved/loaded from)
2. Object re-use in different configurations ( use the same AppleTree always )

Final notes: The following code, is just an scratch, you may really consider a dependency injector to do the job, but of course it will take you a little while to learn it.

The configuration parsing should be made by hand, for the format you're using will be unique for the end-user and should be pretty simple. Still you can deliver as much complexity you want inside your jar ( using any number of frameworks you need ).

You can also take a look to the following design patterns:

• Composite
• Observer
• Interpreter

The implementation below, is a kind of monster the these three ( I didn't compile it, just throw some code to show how the idea would look like )

I hope this helps.

/**
 * Anything. An apple, cider, pie, whatever.
 */
interface Product{}

// The kinds of products. 
class Apple     implements Product{}
class ApplePies implements Product{}
class AppleCider implements Product{}

/**
 * This indicates the class will do something.
 **/
interface Producer { 
    // adds a consumer to the list.
    public void addConsumer( Consumer c );
    // removes the consumer from the list.
    public void removeConsumer( Consumer c );
    // let know eveytone a product has been created.
    public void notifyProductCreation( Product someProduct );
    // You're producer? Produce then... 
    public void startProduction();
}

// To avoid copy/paste all around
class AbstractProducer implements Producer { 
    private List<Consumer> consumers = new ArrayList<Consumer>();
    // adds a consumer to the list.
    public void addConsumer( Consumer c ) {
        consumers.add( c );
    }
    // removes the consumer from the list.
    public void removeConsumer( Consumer c ) {
        consumers.remove( c );
    }
    public void notifyProductCreation( Product someProduct ) { 
        for( Consumer c : list ) { 
            c.productCreated( someProduct );
        }
    }
}

interface Consumer { 
    // Callback to know a product was created
    public void productCreated( Product p );
}


class AppleTree extends AbstractProducer { 
    public void startProduction() { 
        // do something with earh, sun, water.. 
        // and from time to time:
        Product ofThisNewApple = new Apple();
        notifyProductCreation( ofThisNewApple );
    }

}    
class ApplePieMaker extends AbstractProducer implements Consumer { 

    // Ok, a product was created, but
    // is it the product I care?
    // check first and consume after.
    public void productCreated( Product p ){
        // Is this the kind of product I can handle..
        // well do handle
        if( p instanceof Apple ) {
            /// start producing pies..
        }
    }
    public void startProduction() { 
        // collect the needed number of apples and then...
        Product ofPie = new ApplePie();
        notifyProductCreation( ofPie );
    }

}
class ApplePress extends AbstractProducer implements Consumer { 
    // Yeap, something gots produced.
    // Just handle if it is an apple
    public void productCreated( Product p ) { 
        if( p instanceof Apple ) { 
            // start producing cider
        }
    }


    public void startProduction() { 
        // collect the needed number of apples and then...
        Product ofCiderBottle = new AppleCider();
        notifyProductCreation( ofCiderBottle );
    }


}
class AppleSave implements Consumer { 
    public void productCreated( Product p ) { 
        file.append( p );// any one will do.
    }
}

class AppleLoad extends AbstractProducer { 
    public void startProduction() { 
        readFromFile();
    }
    private readFromFile() { 
        for( Product p : file ) { 
            notifyProductCreation( p );  
        }
    }
}


class Main { 
    public static void main( String [] args ) { 
        Configuration conf = new Configuration();
        List<Producer> producers conf.read();
        for( Producer p : producers ) { 
            // fasten your seat belts.... 
            p.startProduction();
        }
    }
}

/// Ahhh, pretty ugly code below this line.
// the idea is:
// Read the configuration file
// for each line split in the "|"
// for each element create a new instance
// and chain it with the next.
// producer | consumer | etc...  
// Becomes....
// new Producer().addConsumer( new Consumer() );
// Return the list of create producers.  
class Configuration { 
    List<Producer> producers
    // read the file 
    // create the instances
    // let them run.
    public List<Producer> read() { 
        File file = new File(....
        // The format is: 
        // producer | consumer-producer <params> | consumer 
        String line = uniqueLineFrom( file );

        String [] parts = line.split("|");

        if( parts.length == 1 ) { 
            System.err.println("Invalid configuration. use element | element | etc. Only one element was....");
            System.exit( 1 );
        }



        int length = parts.length;
        for( int i = 0 ; i < parts.length ; i++ ) { 
            Object theInstance = implementationMap.get( parts[i] ).newInstance();
            validatePosition( i, length, theInstance , parts[i] );
        }

        List<Producer> producers = new ArrayList<Producer>();
        for( int i = 0 ; i < parts.length ; i++ ) { 
            Object theInstance = getInstance( parts[i] );
            if( not( isLast( i, length ) && isProducer( theInstance ) ) { 
                // the next is its consumer
                Producer producer = ( Producer ) theInstance;
                producer.addConsumer( ( Consumer )  getInstance( parts[i+1] ));
                producers.add( producer );
            }
        }
        return producers;

    }
    // creates a new instance from the implementation map.
    private Object getInstance( String key ) { 
        return implementationMap.get( part[i] ).newInstance();        
    }
    // validates if an element at the given position is valid or not.
    // if not, prints the message and exit.
    // the first element most be a producer
    // the last one a consumer 
    // all the middle elements producer-consumer
    // 
    private void validatePosition( int i, int length, Object theInstance, String element  ) { 
        if( isFirst( i ) && not(isProducer(( theInstance ) ))) {  
            System.err.println( "Invalid configuration: " + element + " most be a producer ( either Ap...");
            System.exit( 2 );
        } else if ( isLast( i, length ) && not( isConsumer( theInstance  ))) { 
            System.err.println( "Invalid configuration: " + element + " most be a consumer ( either Ap...");
            System.exit( 3 );
        } else if ( isMiddleAndInvalid( i, length , instance ) ) { 
            System.err.println( "Invalid configuration: " + element + " most be a producer-consumer ( either Ap...");
            System.exit( 4 );
        }
    }
    private static Map<String,Class> implementationMap = new HashMap<String,Class>() static { 
        implementationMap.put( "AppleTree", AppleTree.class );
        implementationMap.put( "ApplePieMaker ", ApplePieMaker .class );
        implementationMap.put( "ApplePress", ApplePress.class );
        implementationMap.put( "AppleSave", AppleSave.class );
        implementationMap.put( "AppleLoad", AppleLoad.class );
        implementationMap.put( "ApplePieMonitor", ApplePieMonitor.class );
    };

    // Utility methods to read better ( hopefully ) the statements 
    // If you could read the validations above you may ignore these functions.


    private boolean not( boolean value ) { 
        return !value;
    }
    private boolean isFirst( int i  ) { 
        return i == 0;
    }
    private boolean isLast( int i, int l ) { 
        return i == l -1 ;
    }
    private boolean isProducer( Object o ) { 
        return o instanceof Producer;
    }
    private boolean isConsumer( Object o ) { 
        return o instanceof Consumer;
    }
    private boolean isMiddleAndInvalid( int index, int length, Object instance ) { 
        return not( isFirst( index ) ) && not( isLast( index, length ) ) && not( isProducer( instance ) && isConsumer( instance ));
    }
}

Answer 3

I believe what you are trying to do can be done within the Spring framework. It uses dependency injection to say "In order to create XI need Y, so find something that produces Y and see what you need in order to create it".

I may be wrong, but I suggest you have a look.

Answer 4

Try the Java Beanshell .

BeanShell is a small, free, embeddable Java source interpreter with object scripting language features, written in Java. BeanShell dynamically executes standard Java syntax and extends it with common scripting conveniences such as loose types, commands, and method closures like those in Perl and JavaScript.

Answer 5

You would need some sort of Registry where producers could register (Hi, I'm an apple tree and I produce apples) and then the consumers could look up whom ever produces apples. This could also be done in reverse where the consumers register interest and the producers look up. I did something similar using JMX where an Object could query the JMX Server for an Object that produced a certain type of Message and then register with that Object (Publish/Subscribe). I am now porting that app to use OSGi which has a similar capability