How to DROP a table or stream in KSQL when topic was dropped first

Question

Using KSQL (Confluent: Version: 5.0.1) I'm able to drop a table / stream normally (using DROP [TABLE|STREAM] ) <NAME> when the linked topic exist and when it is registered (Registered=true).

However, if the topic is dropped first (Registered=false) then the associated stream or table can't be dropped with KSQL pointing out that "No topic with name Foo was registered".

The problem is that the stream / table still shows in the list and no new stream / table can be added using the same name.

Is there any way to delete them after the topic was dropped?

Answer 1

From the official documentation, I would do it like this:

SHOW QUERIES;

Write down the QueryID;

TERMINATE <QueryID>;

DROP TABLE <tablename>;

[See documentation...]

• https://docs.ksqldb.io/en/latest/developer-guide/create-a-table/#delete-a-ksqldb-table
• https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/terminate/

Answer 2

I think I found an answer, or at least in the right direction.

You need to re-register the topic with the stream. After a bit of hacking around, I managed to reattach my topics to streams, the drop the streams.

REGISTER TOPIC <ksql_topic_name> WITH (KAFKA_TOPIC='<kafka_topic_name>', VALUE_FORMAT='<format>');

Then,

DROP STREAM <stream_name>;

I don't know if this works in all cases, but it got me back to where I want to be. If anyone else can iterate on this and find a more robust solution I would like to hear about it.

Answer 3

I've been wrestling with these type issues for a while. Here's where I'm at, from my notes:

Having trouble deleting streams/tables/topics? Delete them in zookeeper:

• Shut down any producers or consumers associated with topics you'll be deleting
• Attempt to drop your tables and streams from KSQL CLI
• Shut down KSQL server
• Delete topics with 'kafka-topic' CLI

• Delete in zookeeper like this:

  zookeeper-shell localhost:2181

  ls /brokers/topics

  _confluent-controlcenter-5-0-0-1-TriggerEventsStore-changelog, _confluent-controlcenter-5-0-0-1-error-topic, ...

  rmr /brokers/topics/yourTopicName

Now do a " kafka-topics --list --zookeeper localhost:2181' and they should be gone for good.

OR, for the ABSOLUTE NUCLEAR OPTION:

• Attempt to drop your tables and streams from KSQL CLI
• Shut down KSQL server
• Shut down your brokers and zookeepers
• Delete eveything in the kafka logs dir on each broker node (location of dir is defined in your properties file).
• Start up your zookeepers again and delete the /brokers/topics from zookeeper CLI

Start everything back up and it should all be gone. But be careful, you'll lose everything, all your data.

Answer 4

You would have to terminate any running queries first.

SHOW QUERIES;  --> that will show you the active queries (COPY QUERY ID)

Then:

TERMINATE [QUERY ID];

Then: DROP [TABLE OR STREAM];