Query with NOT IN subquery returning 0 rows

Question

I am having issues with a SQL query that ideally should return all the comments to a thread in a forum.

Right now i'm having the following query:

    SELECT p.*, 'BBCode' AS Format,
        FROM_UNIXTIME(TIME) AS DateInserted,
        FROM_UNIXTIME(editTime) AS DateUpdated
    FROM et_post p
    LEFT JOIN et_conversation c ON c.conversationId = p.conversationId
    WHERE c.private = 0
    AND p.postId NOT IN (
        SELECT p.postId
        FROM et_conversation c
        LEFT JOIN et_post p ON p.conversationId = c.conversationId WHERE c.private = 0
        GROUP BY p.conversationId
        ORDER BY p.TIME
    )

This, however, returns 0 rows. I expect it to return around 8800 rows.

If I run the first part alone:

    SELECT p.*, 'BBCode' AS Format,
        FROM_UNIXTIME(TIME) AS DateInserted,
        FROM_UNIXTIME(editTime) AS DateUpdated
    FROM et_post p
    LEFT JOIN et_conversation c ON c.conversationId = p.conversationId
    WHERE c.private = 0

Output:

# postId, conversationId, memberId, time, editMemberId, editTime, deleteMemberId, deleteTime, title, content, attributes, Format, DateInserted, DateUpdated
'12', '5', '1', '1436600657', NULL, NULL, NULL, NULL, '', 'Content1', ?, 'BBCode', '2015-07-11 09:44:17', NULL
'13', '5', '1', '1436600681', NULL, NULL, NULL, NULL, 'Testing area', 'Content2', ?, 'BBCode', '2015-07-11 09:44:41', NULL
'14', '5', '1', '1436600698', NULL, NULL, NULL, NULL, 'Testing area', 'Content 3', ?, 'BBCode', '2015-07-11 09:44:58', NULL
'15', '5', '19', '1436602065', NULL, NULL, NULL, NULL, 'Testing area', 'More content', ?, 'BBCode', '2015-07-11 10:07:45', NULL
'16', '5', '19', '1436602093', NULL, NULL, NULL, NULL, 'Testing area', 'Even more content', ?, 'BBCode', '2015-07-11 10:08:13', NULL
'17', '5', '1', '1436602137', NULL, NULL, NULL, NULL, 'Testing area', 'Will it ever stop?', ?, 'BBCode', '2015-07-11 10:08:57', NULL
'54', '5', '1', '1436617274', NULL, NULL, NULL, NULL, 'Testing area', 'Ah, final one..', ?, 'BBCode', '2015-07-11 14:21:14', NULL

It returns 9304 rows like the above which sounds right.

Running the subquery alone:

        SELECT p.postId
        FROM et_conversation c
        LEFT JOIN et_post p ON p.conversationId = c.conversationId WHERE c.private = 0
        GROUP BY p.conversationId
        ORDER BY p.TIME

Output:

# postId
'12'
'18'
'19'
'44'
'70'
'73'
'75'

And it gives me 412 rows like the above which also sounds right.

Ideally, my output of the final query should look like this:

# postId, conversationId, memberId, time, editMemberId, editTime, deleteMemberId, deleteTime, title, content, attributes, Format, DateInserted, DateUpdated
'13', '5', '1', '1436600681', NULL, NULL, NULL, NULL, 'Testing area', 'Content2', ?, 'BBCode', '2015-07-11 09:44:41', NULL
'14', '5', '1', '1436600698', NULL, NULL, NULL, NULL, 'Testing area', 'Content 3', ?, 'BBCode', '2015-07-11 09:44:58', NULL
'15', '5', '19', '1436602065', NULL, NULL, NULL, NULL, 'Testing area', 'More content', ?, 'BBCode', '2015-07-11 10:07:45', NULL
'16', '5', '19', '1436602093', NULL, NULL, NULL, NULL, 'Testing area', 'Even more content', ?, 'BBCode', '2015-07-11 10:08:13', NULL
'17', '5', '1', '1436602137', NULL, NULL, NULL, NULL, 'Testing area', 'Will it ever stop?', ?, 'BBCode', '2015-07-11 10:08:57', NULL
'54', '5', '1', '1436617274', NULL, NULL, NULL, NULL, 'Testing area', 'Ah, final one..', ?, 'BBCode', '2015-07-11 14:21:14', NULL

(Notice postId 12 is gone)

[EDIT] From some quick head calculations I came up to the fact that the following query sounds right according to the number of rows returned:

    SELECT p.*, 'BBCode' AS Format,
        FROM_UNIXTIME(TIME) AS DateInserted,
        FROM_UNIXTIME(editTime) AS DateUpdated
    FROM et_post p
    INNER JOIN et_conversation c ON c.conversationId = p.conversationId
    WHERE c.private = 1
    AND p.postId NOT IN (
        SELECT DISTINCT po.conversationId
        FROM et_post po
    );

[EDIT2] Now with an sqlfiddle

Basically, I want the rows with id 12, 15 and 18 to be gone since they are the original posts created by the one who started the conversation.

[EDIT3] Now with an updated sqlfiddle

• I dug some more into the database and figured out the first sqlfiddle wasn't 100% correct regarding how the data is in the database - therefore this updated version.

Answer 1

Based on the provided SQLFiddle in the edited question, this works.

SELECT p.*, 'BBCode' AS Format,
FROM_UNIXTIME(TIME) AS DateInserted,
FROM_UNIXTIME(editTime) AS DateUpdated
FROM et_post p
INNER JOIN et_conversation c 
ON c.conversationId = p.conversationId
and c.private = 0
join (
select conversationId,min(postId) as m
from et_post
group by conversationId
) r
on r.conversationId = c.conversationId
where p.postId<>r.m

12,15,18 disappear as requested in your edit ... so too does NOT IN madness

Answer 2

first of all , i get no idea what you are trying to achieve with this query, but i am going to tell you whats wrong with your query here ..

In the second part you are using the following query :

SELECT p.postId
    FROM et_conversation c
    LEFT JOIN et_post p ON p.conversationId = c.conversationId WHERE c.private = 0
    GROUP BY p.conversationId
ORDER BY p.TIME

If you run it separately it gives following :

12,15,18

Now remove the Order by and the Group by clause . The query is now:.

SELECT p.postId
FROM et_conversation c
LEFT JOIN et_post p ON p.conversationId = c.conversationId WHERE c.private = 0

If you run this query , you get the result :

12,13,14,15,16,17,18,19,20

which are all of the postid's . the sql is ignoring all of these post id's before grouping them.. because your Group by clause is logically wrong. you are grouping on a column and selecting another column which is not unique in this case.

whatever. the thing is , NOT IN clause is checking the postids before group by.

You need to change logic for your script.

Answer 3

The issue here appears very simple. Lets break down your statement:

 SELECT p.*, 'BBCode' AS Format,
        FROM_UNIXTIME(TIME) AS DateInserted,
        FROM_UNIXTIME(editTime) AS DateUpdated
    FROM et_post p
    LEFT JOIN et_conversation c ON c.conversationId = p.conversationId
    WHERE c.private = 0
    AND p.postId NOT IN (
        SELECT p.postId
        FROM et_conversation c
        LEFT JOIN et_post p ON p.conversationId = c.conversationId WHERE c.private = 0
        GROUP BY p.conversationId
        ORDER BY p.TIME
    )

In the first section you are pulling ALL rows from the et_post table and et_conversation table where c.private = 0

There is no other clauses.

Then in your NOT IN section, you are saying:

"Return ALL results where c.private=0 "

And then of course it removes them from the outer result.

So what is happening here is:

a) You are returning all records in the outer statement b) the NOT IN is returning ALL statements based on the SAME WHERE conditions c) With every row matching, of course you get zero results

Sounds like you need to modify your subquery to what it is exactly that you don't want to see.

Answer 4

omitting the unused et_conversations :

SELECT p.*
    , 'BBCode' AS Format
    , FROM_UNIXTIME(TIME) AS DateInserted
    , FROM_UNIXTIME(editTime) AS DateUpdated
FROM et_post p
WHERE EXISTS ( -- suppress the first one
   SELECT 1
   FROM et_post x
   WHERE x.postid = p.postid
   AND x.TIME < p.TIME
   );