How can I use stream API to load data from the DB?

Question

For example, I'd like to load data from the DB, based on some filter, convert(map) it to some objects, limit the data and collect it to a list:

List<Person> people = new DbStream("PERSON").filter(...).limit(20l).map(...).collect(...);

Can I do it against a DB, using the stream API (probably with JDBC unseen in the background)? How?

* I know it bypasses the SQL language. Just thought to check if there's a way...

Answer 1

You can do it by using an external library called Speedment . They allow you to generate code from your database's metadata, and then use the generated classes to stream over the database's tables.

In this example, we use a database Users which have three fields:

-----------------------------------
- UserName - Trusted - TimeOnline -                                -
-  VARCHAR -   INT   -  BIGINT    -
-----------------------------------

Example streaming (finds the 10 alpabetically first users):

List<String> userNames = new LinkedList<>();
userDatabase.stream()
    .sorted( Users.USER_NAME.comparator() )
    .limit( 10 )
    .forEach( p ->  {
        users.add( p.getUsername() );
    } );
return users;

Example insert:

userDatabase.newEmptyEntity()
             .setUsername( "Gikkman" )
             .setIsTrusted( 1 )
             .setTimeOnline( 0 )
             .persist();  //This command persists the entry to the DB

Answer 2

Doing exactly what you need to do

An interesting library that does what you're looking to do is JINQ , which performs symbolic bytecode execution to translate ordinary Java code to SQL / JPQL or jOOQ queries. Here's an example taken from the documentation:

cityStream
   .where( c -> c.getPopulation() > 10000
         && c.getLandArea() > 300 );

The above will translate (roughly) to the following SQL query:

SELECT * FROM city c WHERE c.population > 10000 AND c.land_area > 300

Or, if it is backed by JPQL, it might just as well generate an entirely different SQL query, including necessary joins, depending on how you specify your entities and relationships.

JINQ takes inspiration from LINQ, I recommend reading this interesting interview with the author .

Doing almost what you need to do

Another library that was mentioned in the comments is jOOQ . It doesn't map stream calls to SQL, but lets you write SQL with a fluent API and then fetch / transform results with streams. Eg

try (Stream<CityRecord> stream =
    DSL.using(configuration)
       .selectFrom(CITY)
       .where(CITY.POPULATION.gt(10000))
       .and(CITY.LAND_AREA.gt(300))) {

    stream.map(...).limit(...).collect(...);
}

However, do note that your database will usually profit from each individual clause like .map() or .limit() if you push it into your SQL statement.

(disclaimer: I work for the company behind jOOQ).