在Postgres日志记录机制中更改定界符和换行符

Question

I have enabled logging in postgres. It logs in CSV. The ninth column is the query that's actually executed. I want to run a cut command to pull that 9th column, but the issue is that the commas and newlines in the query would break the parser. Is there any way to change the delimiter and the newline character?

Answer 1

As suggested in PostgreSQL Manual you can use postgres to parse its own logs:

• create temporary table;
• import log file into it;
• select required data.

Here is simple bash script to automate the process.

#!/usr/bin/env bash

psql -t << EOF
CREATE TEMPORARY TABLE postgres_log
(
  log_time timestamp(3) with time zone,
  user_name text,
  database_name text,
  process_id integer,
  connection_from text,
  session_id text,
  session_line_num bigint,
  command_tag text,
  session_start_time timestamp with time zone,
  virtual_transaction_id text,
  transaction_id bigint,
  error_severity text,
  sql_state_code text,
  message text,
  detail text,
  hint text,
  internal_query text,
  internal_query_pos integer,
  context text,
  query text,
  query_pos integer,
  location text,
  application_name text,
  PRIMARY KEY (session_id, session_line_num)
);

COPY postgres_log FROM '$1' WITH csv;

SELECT query from postgres_log;
EOF

You can run it as: ./fetch-queries.sh /full/path/to/logfile.csv