Singer.io 增量加载不工作

Question

Working with singer.io trying to get data from mysql using xampp on Ubuntu and able to fetch historical data but then i tried to get the incremental load the code is not working.

Not able to get the replication_key_value if anyone can help i will be really thankful to you.

config.json:

{
  "host": "localhost",
  "port": "3307",
  "user": "root",
  "password": ""
}

properties.json:

{
  "streams": [
    {
      "tap_stream_id": "test_1-company",
      "table_name": "company",
      "schema": {
        "properties": {
          "COMPANY_ID": {
            "inclusion": "automatic",
            "maxLength": 6,
            "type": [
              "null",
              "string"
            ]
          },
          "COMPANY_NAME": {
            "inclusion": "available",
            "maxLength": 25,
            "type": [
              "null",
              "string"
            ]
          },
          "COMPANY_CITY": {
            "inclusion": "available",
            "maxLength": 25,
            "type": [
              "null",
              "string"
            ]
          }
        },
        "type": "object"
      },
      "stream": "company",
      "metadata": [
        {
          "breadcrumb": [],
          "metadata": {
            "selected-by-default": false,
            "database-name": "test_1",
            "row-count": 7,
            "is-view": false,
            "table-key-properties": [
              "COMPANY_ID"
            ],
            "replication-method": "INCREMENTAL",
            "replication-key": "COMPANY_ID"
          }
        },
        {
          "breadcrumb": [
            "properties",
            "COMPANY_ID"
          ],
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "varchar(6)"
          }
        },
        {
          "breadcrumb": [
            "properties",
            "COMPANY_NAME"
          ],
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "varchar(25)"
          }
        },
        {
          "breadcrumb": [
            "properties",
            "COMPANY_CITY"
          ],
          "metadata": {
            "selected-by-default": true,
            "sql-datatype": "varchar(25)"
          }
        }
      ]
    }
  ]
}

Console log:

tap-mysql -c config.json --properties properties.json
INFO Server Parameters: version: 10.4.14-MariaDB, wait_timeout: 2700, innodb_lock_wait_timeout: 2700, max_allowed_packet: 1048576, interactive_timeout: 28800
INFO Server SSL Parameters (blank means SSL is not active): [ssl_version: ], [ssl_cipher: ]
{"type": "STATE", "value": {"currently_syncing": null}}

Answer 1

Can you share your executable? Your problem may be in the state.json file. State.json needs the form:

{
   "bookmarks":{
      "123po-employee":{
         "replication_key":"Id",
         "replication_key_value":45,
         "version":1609927544221
      }
   },
   "currently_syncing":null
}