Query working in SQL server, but not in RODBC

Question

I'm trying to run the following query in RODBC :

library(RODBC)

sql <- paste("SELECT * INTO #DataQuery FROM ( SELECT * FROM [DSMM_PPCreporting].[dbo].[DSMM_FACT_TABLE] WITH (NOLOCK)  WHERE [DSMM_FACT_TABLE].Client_ID = 85 AND VenueSource=1   AND TimeStamp BETWEEN  '10/1/2014' AND '10/13/2014' AND Venue_ID in (1,14)) xxxx") 
sql <- paste(sql, "SELECT Timestamp as Date, isnull(FlexCategory.FlexCategoryName,'UNASSIGNED') AS [Flex Category], isnull(Venue.Venue,'UNASSIGNED') AS [Venue], isnull(CategoryName,'UNASSIGNED') AS [Campaign], isnull(Device,'UNASSIGNED') AS [Device], ")
sql <- paste(sql, "round(SUM(isNull(cost,0)),2) AS [Cost ($)],SUM(isNull(VenueConversions1PC,0)) +SUM(isNull(VenueConversions,0))  AS [Venue Leads] ")
sql <- paste(sql, "INTO #FlexCategory FROM #DataQuery DSMM_FACT_TABLE WITH (NOLOCK) ")
sql <- paste(sql, "LEFT JOIN Device WITH (NOLOCK) ON DSMM_FACT_TABLE.Device_ID = Device.Device_ID") 
sql <- paste(sql, "LEFT JOIN Venue WITH (NOLOCK) ON DSMM_FACT_TABLE.Venue_ID = Venue.Venue_ID")
sql <- paste(sql, "LEFT JOIN Category WITH (NOLOCK) ON DSMM_FACT_TABLE.Category_ID = Category.Category_ID")
sql <- paste(sql, "LEFT JOIN FlexCategory WITH (NOLOCK) ON Category.FlexCategory_ID = FlexCategory.FlexCategory_ID")
sql <- paste(sql, "GROUP BY Timestamp, isnull(FlexCategory.FlexCategoryName,'UNASSIGNED') ,")
sql <- paste(sql, "isnull(CategoryName,'UNASSIGNED'), ")
sql <- paste(sql, "isnull(Venue.Venue,'UNASSIGNED'),")
sql <- paste(sql, " isnull(Device,'UNASSIGNED')")
sql <- paste(sql, "ORDER BY [Venue Leads] DESC")
sql <- paste(sql, "SELECT * FROM #FlexCategory WITH (NOLOCK)")
##sql <- paste(sql, "DROP TABLE #FlexCategory")
##sql <- paste(sql, "DROP TABLE #DataQuery")

# Connecting R to DSMM and running the SQL query there
connect <- odbcConnect("DSMM")
system.time(Data <- sqlQuery(connect, sql))
close(connect)

this query works perfectly inside microsoft SQL server:

SELECT * INTO #DataQuery FROM ( SELECT * FROM [DSMM_PPCreporting].[dbo].[DSMM_FACT_TABLE] WITH (NOLOCK)  WHERE [DSMM_FACT_TABLE].Client_ID = 85 AND VenueSource=1   AND TimeStamp BETWEEN  '10/1/2014' AND '10/13/2014' AND Venue_ID in (1,14)) xxxx SELECT Timestamp as Date, isnull(FlexCategory.FlexCategoryName,'UNASSIGNED') AS [Flex Category], isnull(Venue.Venue,'UNASSIGNED') AS [Venue], isnull(CategoryName,'UNASSIGNED') AS [Campaign], isnull(Device,'UNASSIGNED') AS [Device],  round(SUM(isNull(cost,0)),2) AS [Cost ($)],SUM(isNull(VenueConversions1PC,0)) +SUM(isNull(VenueConversions,0))  AS [Venue Leads]  INTO #FlexCategory FROM #DataQuery DSMM_FACT_TABLE WITH (NOLOCK)  LEFT JOIN Device WITH (NOLOCK) ON DSMM_FACT_TABLE.Device_ID = Device.Device_ID LEFT JOIN Venue WITH (NOLOCK) ON DSMM_FACT_TABLE.Venue_ID = Venue.Venue_ID LEFT JOIN Category WITH (NOLOCK) ON DSMM_FACT_TABLE.Category_ID = Category.Category_ID LEFT JOIN FlexCategory WITH (NOLOCK) ON Category.FlexCategory_ID = FlexCategory.FlexCategory_ID GROUP BY Timestamp, isnull(FlexCategory.FlexCategoryName,'UNASSIGNED') , isnull(CategoryName,'UNASSIGNED'),  isnull(Venue.Venue,'UNASSIGNED'),  isnull(Device,'UNASSIGNED') ORDER BY [Venue Leads] DESC SELECT * FROM #FlexCategory WITH (NOLOCK)

I know that my SQL connection works perfectly, and a different query that i'm using works great, for example the following code works great in R and loads a data frame into the global environment:

sql <- "SELECT  TK_ID, Client_ID, tk.searchterm_id, SearchTerm,  RegionName, CityName, Timestamp FROM [DSMM_PPCreporting].[dbo].[tk] "
sql <- paste(sql, "left join [DSMM_PPCreporting].[dbo].searchterm (nolock) on [DSMM_PPCreporting].[dbo].tk.searchterm_id=[DSMM_PPCreporting].[dbo].searchterm.searchterm_id ")
sql <- paste(sql, "left join [DSMM_PPCreporting].[dbo].Region (nolock) on [DSMM_PPCreporting].[dbo].tk.Region_ID=[DSMM_PPCreporting].[dbo].Region.Region_ID ")
sql <- paste(sql, "left join [DSMM_PPCreporting].[dbo].City (nolock) on [DSMM_PPCreporting].[dbo].tk.city_id=[DSMM_PPCreporting].[dbo].city.city_ID ")
#sql <- paste(sql, "where convert(Date, [LastUpdate]) = dateadd(day, datediff(day, 1, GETDATE()), 0)") 
sql <- paste(sql, "where convert(Date, [LastUpdate]) >= '2014-10-15' and convert(Date, [LastUpdate]) <= dateadd(day, datediff(day, 1, GETDATE()), 0)")
# Connecting R to DSMM and running the SQL query there
connect <- odbcConnect("DSMM")
system.time(Data <- sqlQuery(connect, sql))
close(connect)

I know that some SQL server commands don't work in RODBC , could that be the problem?, if so can some one point out to me which SQL server functions won't work in RODBC , so ill try to change the query (if possible).

I tried to find some info on which SQL Server commands, don't work in RODBC , without any success.

Answer 1

I've had a similar problem using RJDBC on an ingres database and I believe the cause was a memory problem.

I was running a large query from R with an inner select (which would also have a large amount of data). It did not work and returned an error like

Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for "..

Running the same query directly on the database worked perfectly fine (it took a while though). At first I also thought it could be the inner select. However, I had another query with an inner select which was working fine, and when I only ran the part inside the inner select it did not work either.

I have concluded it was a memory issue, since later, when there were less users using the database, the same query did run from R (and it still does). Certainly it is not so that RODBC or RJDBC does not support inner selects. I have worked with inner selects with both configurations and they work fine.

For me this query had a particularly large amount of data obtained by the inner select. And I think that this is the reason why at the same time I could run another similar query, using only a smaller timeframe (and hence there was less data obtained by the inner select). Hope this helps with figuring out what is wrong.

Answer 2

It seems like RODBC does not support inner select statements:

This code works:

sql <- paste("SELECT TimeStamp FROM [DSMM_PPCreporting].[dbo].[DSMM_FACT_TABLE]  WHERE [DSMM_FACT_TABLE].Client_ID = 85 AND VenueSource=1   AND TimeStamp BETWEEN  '10/1/2014' AND '10/13/2014' AND Venue_ID in (1,14)") 
connect <- odbcConnect("DSMM")
Data <- sqlQuery(connect, sql)
close(connect)

While this doesn't:

sql <- paste("SELECT *  FROM (SELECT TimeStamp FROM [DSMM_PPCreporting].[dbo].[DSMM_FACT_TABLE]  WHERE [DSMM_FACT_TABLE].Client_ID = 85 AND VenueSource=1   AND TimeStamp BETWEEN  '10/1/2014' AND '10/13/2014' AND Venue_ID in (1,14))") 
connect <- odbcConnect("DSMM")
Data <- sqlQuery(connect, sql)
close(connect)