logstash cron schedule to run every 12 hours starting at certain time

Question

I am trying 0 1/12 * * * cron expression but it only fires once a day. Below is 1 of my configuration.

input {
    jdbc {
        jdbc_connection_string => "jdbc:redshift://xxx.us-west-2.redshift.amazonaws.com:5439/xxx"
        jdbc_user => "xxx"
        jdbc_password => "xxx"
        jdbc_validate_connection => true
        jdbc_driver_library => "/mnt/logstash-6.0.0/RedshiftJDBC42-1.2.10.1009.jar"
        jdbc_driver_class => "com.amazon.redshift.jdbc42.Driver"
        schedule => "0 1/12 * * *" #01:00,13:00, tried from https://crontab.guru/#0_1/12_*_*_*
        statement_filepath => "conf/log_event_query.sql"
        use_column_value => true
        tracking_column => dw_insert_dt
        last_run_metadata_path => "metadata/logstash_jdbc_last_run_log_event"
    }
}
output {
    elasticsearch {
        index => "logs-ics_%{+dd_MM_YYYY}"
        document_type => "log_event"
        document_id => "%{log_entry_id}"
        hosts => [ "x.x.x.x:xxxx" ]
    }
}

I also tried below 0 0 1/12 ? * * * 0 0 1/12 ? * * * from https://www.freeformatter.com/cron-expression-generator-quartz.html but lostash does not support this type. Original cron used.

Please help me get a cron expression which works in logstash according to following dates and also is there a online page where I can test my future logstash cron expressions?

1st  at 2018-08-01 01:00:00
then at 2018-08-01 13:00:00
then at 2018-08-02 01:00:00
then at 2018-08-02 13:00:00
then at 2018-08-03 01:00:00

Answer 1

It looks like your scheduling format is wrong.

To do a once-every-twelve hours task, you would use */12 , not 1/12 :

0 */12 * * * # Every twelve hours at minute 0 of the hour.

Your schedule looks more like an attempt to run the task at one and 12, but to do that you would use a comma, like this:

0 1,12 * * * # Run at one and twelve hours at minute 0.

The rufus extension also allows for adding the timezone (like Asia/Kuala_Lumpur), if you need this to run scheduled on a specific timezone that is not the default machine clock.

Your code above doesn't show us the SQL query you are running. The query could be firing, but if there are no results from the query, you aren't going to get any input in logstash. In any case, your scheduling syntax needs to change from 1/12 to */12 to do what you want it to.

More generally, according to the logstash jdbc input plugin documentation, the scheduling format is considered to be cron-"like." The logstash jdbc input plugin uses the ruby Rufus scheduler. The docs on that scheduling format are here: https://github.com/jmettraux/rufus-scheduler#parsing-cronlines-and-time-strings

Logstash 6.0 JDBC plugin docs are here: https://www.elastic.co/guide/en/logstash/6.0/plugins-inputs-jdbc.html

Hope this helps.