跨PG版本的Postgresql扩展pg_regress测试

Question

I am writing a PostgreSQL extension which needs to work on PostgreSQL 9.4, 9.5, 9.6, and 10. I am using the standard pg_regress regression tests with make installcheck for the tests. I am trying to set up tests across versions with Travis-CI.

Because this is a security-critical extension, I have a lot of statements in my test cases that are supposed to throw errors, for example checks against whitelisted keywords.

pg_regress works by comparing the output of the test script with the expected output. For ordinary, successful queries this works fine, but for errors I have a problem.

For 9.6, the output of an error seems to always have a CONTEXT line while for 9.4 it never does. In other words, 9.6 is sending troubleshooting information missing in 9.4, and this causes the tests to be incompatible across versions.

The relevant portions of my test script are:

set client_min_messages to warning;
set log_error_verbosity to terse;
create extension roleman;

-- whitelist errors

-- no tables so pick a random number and cast to oid
select roleman.grant_table('postgres', 1::oid, array['execute', 'drop']);
select roleman.grant_schema('postgres', 'foo', array['execute']);
select roleman.grant_schema_all('postgres', 'foo', 'tables', array['execute', 'drop everything']);
select roleman.grant_database('postgres', 'foo', array['execute']);
select roleman.grant_function('postgres', 1::OID, array['wheeee']);
select roleman.grant_seq('postgres', 1::oid, array['execute']);

The output on 9.6 is:

set client_min_messages to warning;
set log_error_verbosity to terse;
create extension roleman;
-- whitelist errors
-- no tables so pick a random number and cast to oid
select roleman.grant_table('postgres', 1::oid, array['execute', 'drop']);
ERROR:  bad database permissions for postgres,  table 1, perms execute, drop
CONTEXT:  PL/pgSQL function grant_table(text,regclass,text[]) line 4 at RAISE
select roleman.grant_schema('postgres', 'foo', array['execute']);
ERROR:  bad permissions for postgres,  schema foo, perms execute
CONTEXT:  PL/pgSQL function grant_schema(text,text,text[]) line 4 at RAISE
select roleman.grant_schema_all('postgres', 'foo', 'tables', array['execute', 'drop everything']);
ERROR:  bad database permissions for postgres,  schema foo, type tables, perms execute, drop everything
CONTEXT:  PL/pgSQL function grant_schema_all(text,text,text,text[]) line 5 at RAISE
select roleman.grant_database('postgres', 'foo', array['execute']);
ERROR:  bad database permissions for postgres, dbname foo, perms execute
CONTEXT:  PL/pgSQL function grant_database(text,text,text[]) line 4 at RAISE
select roleman.grant_function('postgres', 1::OID, array['wheeee']);
ERROR:  bad database permissions for postgres,  function 1, perms wheeee
CONTEXT:  PL/pgSQL function grant_function(text,regprocedure,text[]) line 4 at RAISE
select roleman.grant_seq('postgres', 1::oid, array['execute']);
ERROR:  bad database permissions for postgres,  sequence 1, perms execute
CONTEXT:  PL/pgSQL function grant_seq(text,regclass,text[]) line 4 at RAISE

On 9.4 the output is the same except the CONTEXT lines are missing. I tried to disable the lines by setting the error verbosity at the beginning of my script but that had no effect. The lack of the CONTEXT line is treated by pg_regress a test failure.

Is there any way around this?

Answer 1

This sort of thing is a right pain. pg_regress supports alternate output files, but they're clumsy to use and maintain.

What BDR and pglogical does is \\set VERBOSITY terse , which helps with some such issues.

In more complex cases sometimes you can use a DO block. Something like

DO LANGUAGE plpgsql $$
BEGIN
    BEGIN
        PERFORM the_statement;
    EXCEPTION WHEN ... THEN ...
        ... check exception matches expected here ...
    END;
END;
$$