Poor query performance PostgreSQL

Question

I have a query:

SELECT
        '{\"nombre\":\"'||c.column_name||'\",\"type\":\"'|| c.data_type
        ||'\",\"is_nullable\":\"'|| c.is_nullable||'\",\"is_pk\":\"'|| CASE WHEN constraint_type = 'PRIMARY KEY' THEN 'SI' ELSE 'NO' END
        ||'\",\"max_length\":\"'||COALESCE(c.character_maximum_length::VARCHAR,'')||'\",\"FK_schema\":\"'||COALESCE(ccu.table_schema,'')||'\",\"FK_tabla\":\"'||
       COALESCE(ccu.table_name,'')||'\",\"FK_columna\":\"'||COALESCE(ccu.column_name,'')||
       '\"}' as id, c.column_name
FROM information_schema.columns AS c
LEFT JOIN  information_schema.key_column_usage i ON I.table_name=c.table_name AND I.table_schema=c.table_schema AND c.column_name = I.column_name 
LEFT JOIN information_schema.table_constraints tc ON TC.constraint_name = I.CONSTRAINT_NAME AND constraint_type IN ('FOREIGN KEY','PRIMARY KEY')
LEFT JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name = tc.constraint_name 
AND constraint_type IN ('FOREIGN KEY')

WHERE c.table_schema = '".$_POST['schema']."' AND c.table_name='".$_POST['tabla']."'
AND NOT EXISTS (
        SELECT q.column_name FROM information_schema.constraint_column_usage  q
        inner join information_schema.table_constraints USING(constraint_name )
        WHERE c.table_schema = q.table_schema AND q.table_name=c.table_name AND q.column_name = c.column_name 
        AND tc.constraint_name > q.constraint_name AND TC.constraint_type <> 'PRIMARY KEY') 
ORDER BY c.ordinal_position;

It works, but in a database with 97 schemas it takes an entire minute to run.

How can I fix this?

Without this line:

LEFT JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name = tc.constraint_name AND constraint_type = 'FOREIGN KEY'

it takes less than 1 second.

Answer 1

Since the query all appears to be about Postgres metadata regarding your database, presumably those do not change nearly as often as perhaps a regular set of tables in a query would.

Therefore, I would recommend that you use a materialized view which is set to refresh in some interval that you deem acceptable for data currency. Perhaps that's every hour, or daily -- it all depends on how often this data changes, and how current you need it to be (which are also interrelated).

So you would do:

CREATE MATERIALIZED VIEW mat_view AS
SELECT
        '{\"nombre\":\"'||c.column_name||'\",\"type\":\"'|| c.data_type
        ||'\",\"is_nullable\":\"'|| c.is_nullable||'\",\"is_pk\":\"'|| CASE WHEN constraint_type = 'PRIMARY KEY' THEN 'SI' ELSE 'NO' END
        ||'\",\"max_length\":\"'||COALESCE(c.character_maximum_length::VARCHAR,'')||'\",\"FK_schema\":\"'||COALESCE(ccu.table_schema,'')||'\",\"FK_tabla\":\"'||
       COALESCE(ccu.table_name,'')||'\",\"FK_columna\":\"'||COALESCE(ccu.column_name,'')||
       '\"}' as id, c.column_name
FROM information_schema.columns AS c
LEFT JOIN  information_schema.key_column_usage i ON I.table_name=c.table_name AND I.table_schema=c.table_schema AND c.column_name = I.column_name 
LEFT JOIN information_schema.table_constraints tc ON TC.constraint_name = I.CONSTRAINT_NAME AND constraint_type IN ('FOREIGN KEY','PRIMARY KEY')
LEFT JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name = tc.constraint_name 
AND constraint_type IN ('FOREIGN KEY')

WHERE c.table_schema = '".$_POST['schema']."' AND c.table_name='".$_POST['tabla']."'
AND NOT EXISTS (
        SELECT q.column_name FROM information_schema.constraint_column_usage  q
        inner join information_schema.table_constraints USING(constraint_name )
        WHERE c.table_schema = q.table_schema AND q.table_name=c.table_name AND q.column_name = c.column_name 
        AND tc.constraint_name > q.constraint_name AND TC.constraint_type <> 'PRIMARY KEY') 
ORDER BY c.ordinal_position;

Which will essentially create a snapshot of the results of this view at the time it's first run. Then it can be refreshed via REFRESH MATERIALIZED VIEW mat_view;

You may also wish to use the CONCURRENTLY option, which prevents blocking, although can take longer.

The REFRESH MATERIALIZED VIEW command can be put on a cron to ensure it is done on the interval you wish.

Note: Materialized views were a new feature in Postgres 9.3. CONCURRENTLY was added in Postgres 9.4.

Edit in response to comment from OP:

Regarding Postgres 8.4 , it sounds like it's out of your control, but you may want to lobby the server manager and DBA with this information to see if it changes their minds:

1. Postgres 8.4 was released almost 7 years ago (July 2009)
2. Postgres 8.4 was EOL'ed (ie not supported, no more updates) almost 2 years ago (July 2014).

As for what you can do on 8.4, it's definitely much more limited... One thing you could try which may buy you some improvement (although not nearly as much as the materialized view could in later versions) would be to try to break the JOIN up into a couple of different temp table s. That may allow the query planner to better reason about each individual query and deal with the parts faster than the whole, although certainly it's a highly variable thing dependent on data, server tuning, and other factors.

Also, try creating a temp table for the sub-select in your NOT EXISTS expression within your WHERE before the query and use that instead.

Make sure it (and the other temp tables) has any indexes you may need so it can do index scans as much as possible and not full ones. Refer to the EXPLAIN statement ( PgAdmin provides a visual one as well) to help see what the query planner thinks about the query to start and how that changes as you break it down into components.

Answer 2

i made a new query reading this postgressql ...

    select 
        '{\"nombre\":\"'||columna.attname||'\",\"tipo\":\"'|| tipo.typname
        ||'\",\"is_nullable\":\"'|| CASE WHEN columna.attnotnull THEN 'NO' ELSE 'SI' END||'\",\"is_pk\":\"'|| CASE WHEN pk.conname IS NULL THEN 'NO' ELSE 'SI' END 
        ||'\",\"max_length\":\"'||COALESCE(character_maximum_length::varchar,'')||'\",\"FK_schema\":\"'||COALESCE(fk_schema.nspname,'')||'\",\"FK_tabla\":\"'||
       COALESCE(fk_tabla.relname,'')||'\",\"FK_columna\":\"'||COALESCE(fk_columna.attname,'')||
       '\"}' as id, columna.attname,columna.*
from pg_catalog.pg_namespace pgschema
inner join pg_catalog.pg_class clase  ON PGSCHEMA.OID = clase.relnamespace and relname ='".$_POST['tabla']."'
inner join  pg_catalog.pg_attribute columna ON ATtReLID = clase.oid
inner join pg_catalog.pg_type tipo ON tipo.oid = columna.atttypid
inner join information_schema.columns ON table_schema = pgschema.nspname AND table_name = clase.relname AND column_name = columna.attname
left join pg_catalog.pg_constraint pk ON pk.contype = 'p' AND PK.conrelid = clase.oid and array[columna.attnum] && pk.conkey
left join pg_catalog.pg_constraint Fk ON FK.contype = 'f' AND FK.conrelid = clase.oid and array[columna.attnum] && fk.conkey
left join pg_catalog.pg_class Fk_tabla ON FK.confrelid = fk_tabla.oid -- AND FK.conrelid = clase.oid and array[columna.attnum] && fk.conkey
left join pg_catalog.pg_namespace Fk_schema ON FK_tabla.relnamespace = fk_schema.oid
left join  pg_catalog.pg_attribute fk_columna ON fk_tabla.OID = fk_columna.attrelid AND ARRAY[fk_columna.attnum] && fk.confkey
where pgschema.nspname = '".$_POST['schema']."' and columna.attnum > 0;

it takes less than 1 second.