subquery inside an object constructor in JPQL

Question

I am having issues in converting SQL query to JPQL. There is a table CASE_FORM whose status can be something like 'NEW', 'Draft', etc. There are further 3 columns: category, case_type and region(ie location). My task is to show the number of cases for each case_status for every combination of category, case_type and region.

For example, for some category: A , case_type: 1 and region: San Francisco the number of cases with status New could be some integer value like 12 and number of cases with status Draft could be 3.

For this purpose, I intend to use a group by on category, case_type and region and then take a count for the number of cases with case_status Draft and another count with status, New .

A working SQL query for the above mentioned approach is given below:

SQL:

select cf.CATEGORY_ID, cf.CASE_TYPE_ID, cf.REGION,
(select count(*) from KM_ECM.CASE_FORM where CATEGORY_ID=cf.CATEGORY_ID AND CASE_TYPE_ID=cf.CASE_TYPE_ID AND REGION=cf.REGION AND CASE_STATUS='New') AS NEW,
(select count(*) from KM_ECM.CASE_FORM where CATEGORY_ID=cf.CATEGORY_ID AND CASE_TYPE_ID=cf.CASE_TYPE_ID AND REGION=cf.REGION AND CASE_STATUS='Draft') AS DRAFT
from KM_ECM.CASE_FORM cf
group by cf.CATEGORY_ID, cf.CASE_TYPE_ID, cf.REGION;

Since the query will be hit inside a java based web service therefore I am using JPA and hence JPQL to write the above query. For this specific task, the query in JPQL looks like this:

JPQL:

@NamedQuery(name="CaseForm.getMainReportData", query="SELECT new com.techlogix.sbp.casemanagement.model.MainReport(c.category, c.caseType, c.region, "
        + "(SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='New'),"
        + "(SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='Draft')"
        + ") "
        + "FROM CaseForm c "
        + "GROUP BY c.category, c.caseType, c.region")

The idea behind the JPQL query is exactly same as the SQL one mentioned above. The only addition is that the JPQL query stores the data in a custom Bean class, MainReport .

However, upon execution the query breaks and following exception is thrown,

org.apache.openjpa.persistence.ArgumentException : Your query is invalid. Your select and having clauses must only include aggregates or values that also appear in your grouping clause

The complete stack-trace of the exception is:

[3/5/19 12:17:05:718 PKT] 000000c0 BusinessExcep E   CNTR0020E: EJB threw an unexpected (non-declared) exception during invocation of method "getMainReport" on bean "BeanId(casemanagement-ear#casemanagement-business-service-impl-1.0.jar#FEODReportService, null)". Exception data: <openjpa-2.2.3-SNAPSHOT-r422266:1764177 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Your query on type "class com.techlogix.sbp.casemanagement.model.CaseForm" with filter "" is invalid.  Your select and having clauses must only include aggregates or values that also appear in your grouping clause.
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor$ValidateGroupingExpressionVisitor.enter(ExpressionStoreQuery.java:563)
at org.apache.openjpa.kernel.exps.Val.acceptVisit(Val.java:119)
at org.apache.openjpa.jdbc.kernel.exps.CompareEqualExpression.acceptVisit(CompareEqualExpression.java:138)
at org.apache.openjpa.jdbc.kernel.exps.AndExpression.acceptVisit(AndExpression.java:80)
at org.apache.openjpa.jdbc.kernel.exps.SubQ.acceptVisit(SubQ.java:241)
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor$ValidateGroupingExpressionVisitor.validate(ExpressionStoreQuery.java:544)
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor.validate(ExpressionStoreQuery.java:335)
at org.apache.openjpa.kernel.QueryImpl.compile(QueryImpl.java:591)
at org.apache.openjpa.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:1038)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:116)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:50)
at org.apache.openjpa.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:1017)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:127)
at com.ibm.ws.jpa.management.JPATxEmInvocation.createNamedQuery(JPATxEmInvocation.java:340)
at com.ibm.ws.jpa.management.JPAEntityManager.createNamedQuery(JPAEntityManager.java:543)
at com.techlogix.sbp.casemanagement.dao.impl.CaseFormDaoImpl.getMainReportData(CaseFormDaoImpl.java:495)
at com.techlogix.sbp.casemanagement.service.impl.FEODReportServiceImpl.getMainReport(FEODReportServiceImpl.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.EJSContainer.invokeProceed(EJSContainer.java:5798)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:569)
at org.apache.webbeans.ejb.common.interceptor.OpenWebBeansEjbInterceptor.callInterceptorsAndDecorators(OpenWebBeansEjbInterceptor.java:559)
at org.apache.webbeans.ejb.common.interceptor.OpenWebBeansEjbInterceptor.callToOwbInterceptors(OpenWebBeansEjbInterceptor.java:222)
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:227)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:549)
at org.apache.webbeans.ejb.WSEJBInterceptor.callToOwbInterceptors(WSEJBInterceptor.java:152)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:227)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:549)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.doAroundInvoke(InvocationContextImpl.java:230)
at com.ibm.ejs.container.EJSContainer.invoke(EJSContainer.java:5689)
at com.techlogix.sbp.casemanagement.service.EJSRemote0SLFEODReportService_81b95711.getMainReport(EJSRemote0SLFEODReportService_81b95711.java)
at com.techlogix.sbp.casemanagement.service._FEODReportService_Stub.getMainReport(_FEODReportService_Stub.java:1)
at com.techlogix.sbp.casemanagement.soap.FEODReportsWebService.getMainReport(FEODReportsWebService.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.apache.axis2.jaxws.server.dispatcher.JavaDispatcher.invokeTargetOperation(JavaDispatcher.java:120)
at org.apache.axis2.jaxws.server.dispatcher.JavaBeanDispatcher.invoke(JavaBeanDispatcher.java:118)
at org.apache.axis2.jaxws.server.EndpointController.invoke(EndpointController.java:111)
at org.apache.axis2.jaxws.server.JAXWSMessageReceiver.receive(JAXWSMessageReceiver.java:161)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:212)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at com.ibm.ws.websvcs.transport.http.WASAxis2Servlet.doPost(WASAxis2Servlet.java:1632)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1114)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:949)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)


[3/5/19 12:17:05:733 PKT] 000000c0 AxisEngine    E org.apache.axis2.engine.AxisEngine receive null
                             org.apache.axis2.AxisFault
at org.apache.axis2.jaxws.server.dispatcher.JavaBeanDispatcher.createFaultResponse(JavaBeanDispatcher.java:407)
at org.apache.axis2.jaxws.server.dispatcher.JavaBeanDispatcher.invoke(JavaBeanDispatcher.java:138)
at org.apache.axis2.jaxws.server.EndpointController.invoke(EndpointController.java:111)
at org.apache.axis2.jaxws.server.JAXWSMessageReceiver.receive(JAXWSMessageReceiver.java:161)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:212)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at com.ibm.ws.websvcs.transport.http.WASAxis2Servlet.doPost(WASAxis2Servlet.java:1632)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1114)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:949)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.apache.axis2.jaxws.server.dispatcher.JavaDispatcher.invokeTargetOperation(JavaDispatcher.java:120)
at org.apache.axis2.jaxws.server.dispatcher.JavaBeanDispatcher.invoke(JavaBeanDispatcher.java:118)
... 30 more
Caused by: javax.ejb.EJBException: See nested exception; nested exception is: <openjpa-2.2.3-SNAPSHOT-r422266:1764177 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Your query on type "class com.techlogix.sbp.casemanagement.model.CaseForm" with filter "SELECT new com.techlogix.sbp.casemanagement.model.MainReport(c.category, c.caseType, c.region, (SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='New'), (SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='Draft')) FROM CaseForm c GROUP BY c.category, c.caseType, c.region" is invalid.  Your select and having clauses must only include aggregates or values that also appear in your grouping clause.
at com.ibm.ejs.container.util.ExceptionUtil.EJBException(ExceptionUtil.java:475)
at com.ibm.ejs.container.util.ExceptionUtil.EJBException(ExceptionUtil.java:365)
at com.ibm.ejs.container.BusinessExceptionMappingStrategy.mapException(BusinessExceptionMappingStrategy.java:392)
at com.ibm.ejs.container.BusinessExceptionMappingStrategy.setUncheckedException(BusinessExceptionMappingStrategy.java:554)
at com.ibm.ejs.container.EJSDeployedSupport.setUncheckedLocalException(EJSDeployedSupport.java:570)
at com.techlogix.sbp.casemanagement.service.EJSRemote0SLFEODReportService_81b95711.getMainReport(EJSRemote0SLFEODReportService_81b95711.java)
at com.techlogix.sbp.casemanagement.service._FEODReportService_Stub.getMainReport(_FEODReportService_Stub.java:1)
at com.techlogix.sbp.casemanagement.soap.FEODReportsWebService.getMainReport(FEODReportsWebService.java:120)
... 36 more
Caused by: <openjpa-2.2.3-SNAPSHOT-r422266:1764177 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Your query on type "class com.techlogix.sbp.casemanagement.model.CaseForm" with filter "SELECT new com.techlogix.sbp.casemanagement.model.MainReport(c.category, c.caseType, c.region, (SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='New'), (SELECT count(v) FROM CaseForm v where v.category=c.category AND v.caseType=c.caseType AND v.region=c.region AND v.caseStatus='Draft')) FROM CaseForm c GROUP BY c.category, c.caseType, c.region" is invalid.  Your select and having clauses must only include aggregates or values that also appear in your grouping clause.
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor$ValidateGroupingExpressionVisitor.enter(ExpressionStoreQuery.java:563)
at org.apache.openjpa.kernel.exps.Val.acceptVisit(Val.java:119)
at org.apache.openjpa.jdbc.kernel.exps.CompareEqualExpression.acceptVisit(CompareEqualExpression.java:138)
at org.apache.openjpa.jdbc.kernel.exps.AndExpression.acceptVisit(AndExpression.java:80)
at org.apache.openjpa.jdbc.kernel.exps.SubQ.acceptVisit(SubQ.java:241)
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor$ValidateGroupingExpressionVisitor.validate(ExpressionStoreQuery.java:544)
at org.apache.openjpa.kernel.ExpressionStoreQuery$AbstractExpressionExecutor.validate(ExpressionStoreQuery.java:335)
at org.apache.openjpa.kernel.QueryImpl.compile(QueryImpl.java:591)
at org.apache.openjpa.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:1038)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:116)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:50)
at org.apache.openjpa.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:1017)
at com.ibm.ws.persistence.EntityManagerImpl.createNamedQuery(EntityManagerImpl.java:127)
at com.ibm.ws.jpa.management.JPATxEmInvocation.createNamedQuery(JPATxEmInvocation.java:340)
at com.ibm.ws.jpa.management.JPAEntityManager.createNamedQuery(JPAEntityManager.java:543)
at com.techlogix.sbp.casemanagement.dao.impl.CaseFormDaoImpl.getMainReportData(CaseFormDaoImpl.java:495)
at com.techlogix.sbp.casemanagement.service.impl.FEODReportServiceImpl.getMainReport(FEODReportServiceImpl.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.EJSContainer.invokeProceed(EJSContainer.java:5798)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:569)
at org.apache.webbeans.ejb.common.interceptor.OpenWebBeansEjbInterceptor.callInterceptorsAndDecorators(OpenWebBeansEjbInterceptor.java:559)
at org.apache.webbeans.ejb.common.interceptor.OpenWebBeansEjbInterceptor.callToOwbInterceptors(OpenWebBeansEjbInterceptor.java:222)
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:227)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:549)
at org.apache.webbeans.ejb.WSEJBInterceptor.callToOwbInterceptors(WSEJBInterceptor.java:152)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:227)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:549)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.doAroundInvoke(InvocationContextImpl.java:230)
at com.ibm.ejs.container.EJSContainer.invoke(EJSContainer.java:5689)
... 39 more

Answer 1

I do recommend you don't run two very similar correlated subqueries like that. It will be much faster to calculate both counts in one go in SQL. Given that your subqueries select from the same table as the outer query, you can even do this with a single access to your table:

SELECT
  cf.CATEGORY_ID, 
  cf.CASE_TYPE_ID, 
  cf.REGION,
  COUNT(CASE WHEN cf.CASE_STATUS = 'New' THEN 1 END) AS NEW,
  COUNT(CASE WHEN cf.CASE_STATUS = 'Draft' THEN 1 END) AS DRAFT
FROM KM_ECM.CASE_FORM cf
GROUP BY 
  cf.CATEGORY_ID, 
  cf.CASE_TYPE_ID, 
  cf.REGION

I have blogged about this technique more in detail here .

With this simplified version that no longer relies on correlated subqueries, you might be able to work with a DTO projection in JPQL now, or you just use a native query instead.