Adding custom stopwords to IBM Watson Discovery

Question

I'm trying to add a custom stopwords on a colletion of Watson Discovery, but I only get the error 500 "Error when creating 'stopwords'.". Same on both web and api (curl).

I've tried:

• file with 99 lines,
• empty file,
• and an IBM file ( https://watson-developer-cloud.github.io/doc-tutorial-downloads/discovery/custom_stopwords_en.txt )

I've checked ( https://cloud.ibm.com/docs/services/discovery?topic=discovery-query-concepts&locale=en ):

• Advanced plans - ok
• The size limit is one million characters - ok
• Only one custom stopword list per collection - ok
• All stopwords should be lowercase. - ok
• Delete and create a new collection - ok

Also, I ran curl with invalid collection and environment to check the api validation (unnecessary I know), and it returned 404 "Could not find listed collection" as expected (ok, it's working).

Am I missing something? What more can I check?

curl command:

curl -X POST -u "apikey":"..." --data-binary @custom_stopwords_pt.txt "https://gateway.watsonplatform.net/discovery/api/v1/environments/.../collections/.../word_lists/stopwords?version=2019-04-30"

Thank's

Answer 1

I am using python sdk and the following code to upload a custom stopword list, discovery query seems to use the custom stopwords and update the search query results without any error, I did not even need to Delete and create a new collection.

authenticator = IAMAuthenticator('<your api key>')
discovery = DiscoveryV1(
    version='2020-01-15',
    authenticator=authenticator)
discovery.set_service_url('<service url>')

# get the writable environment id (as eid) and the collection id (as cid) 
# where you want to upload the stopwords list

discovery.create_stopword_list(environment_id=eid,
                               collection_id=cid,
                               stopword_file=stopwords,
                               stopword_filename ='custom_stopwords.txt')

# it takes some time to update the sotpword list
# once it's done, the 'status' returned by the next api should be 'active'
discovery.get_stopword_list_status(environment_id=eid, 
                                   collection_id=cid).get_result()
# {'status': 'active', 'type': 'stopwords'}