Generate a Properties File using Shell Script and Results from a SQL Query

Question

I am trying to create a properties file like this...

firstname=Jon
lastname=Snow
occupation=Nights_Watch
family=Stark

...from a query like this...

SELECT 
  a.fname as firstname, 
  a.lname as lastname, 
  b.occ as occupation... 
FROM 
  names a, 
  occupation b, 
  family c... 
WHERE... 

How can I do this? As I am aware of only using spool to a CSV file which won't work here?

These property files will be picked up by shell scripts to run automated tasks. I am using Oracle DB

Answer 1

Perhaps something like this?

psql -c 'select id, name from test where id = 1' -x -t -A -F = dbname -U dbuser

Output would be like:

id=1
name=test1

(For the full list of options: man psql .)

Answer 2

It is possible to to this from your command line SQL client but as STTLCU notes it might be better to get the query to output in something "standard" (like CSV) and then transform the results with a shell script. Otherwise, because a lot of the features you would use are not part of any SQL standard, they would depend on the database server and client application. Think of this step as sort of the obverse of ETL where you clean up the data you "unload" so that it is useful for some other application.

For sure there's ways to build this into your query application: eg if you use something like perl DBI::Shell as your client (which allows you to connect to many different servers using the DBI module) you can jazz up your output in various ways. But here you'd probably be best off if could send the query output to a text file and run it through awk .

Having said that ... here's how the Postgresql client could do what you want. Notice how the commands to set up the formatting are not SQL but specific to the client.

~/% psql -h 192.168.2.69 -d cropdusting -u stubblejumper
psql (9.2.4, server 8.4.14)
    WARNING: psql version 9.2, server version 8.4.
         Some psql features might not work.
You are now connected to database "cropdusting" as user "stubblejumper".

cropdusting=# \pset border 0 \pset format unaligned \pset t \pset fieldsep =
Border style is 0.
Output format is unaligned.
Showing only tuples.
Field separator is "=".
cropdusting=# select year,wmean_yld from bckwht where year=1997 AND freq > 13 ;
1997=19.9761904762
1997=14.5533333333
1997=17.9942857143
cropdusting=# 

With the psql client the \\pset command sets options affecting the output of query results tables. You can probably figure out which option is doing what. If you want to do this using your SQL client tell us which one it is or read through the manual page for tips on how to format the output of your queries.

Answer 3

My answer is very similar to the two already posted for this question, but I try to explain the options, and try to provide a precise answer.

When using Postgres, you can use psql command-line utility to get the intended output

psql -F = -A -x -X <other options> -c 'select a.fname as firstname, a.lname as lastname from names as a ... ;'

The options are:

-F : Use '=' sign as the field separator, instead of the default pipe '|'
-A : Do not align the output; so there is no space between the column header, separator and the column value.
-x : Use expanded output, so column headers are on left (instead of top) and row values are on right.
-X : Do not read $HOME/.psqlrc, as it may contain commands/options that can affect your output.
-c : The SQL command to execute
<other options> : Any other options, such as connection details, database name, etc.

Answer 4

Since you mentionned spool I will assume you are running on Oracle. This should produce a result in the desired format, that you can spool straight away.

SELECT
    'firstname=' || firstname || CHR(10) ||
    'lastname=' || lastname || CHR(10) -- and so on for all fields
FROM your_tables;

The same approach should be possible with all database engines, if you know the correct incantation for a litteral new line and the syntax for string concatenation.

Answer 5

You have to choose if you want to maintain such a file from shell or from PL/SQL. Both solutions are possible and both are correct.

Because Oracle has to read and write from the file I would do it from database side.

You can write data to file using UTL_FILE package.

DECLARE
    fileHandler UTL_FILE.FILE_TYPE;
BEGIN
    fileHandler := UTL_FILE.FOPEN('test_dir', 'test_file.txt', 'W');
    UTL_FILE.PUTF(fileHandler, 'firstname=Jon\n');
    UTL_FILE.PUTF(fileHandler, 'lastname=Snow\n');
    UTL_FILE.PUTF(fileHandler, 'occupation=Nights_Watch\n');
    UTL_FILE.PUTF(fileHandler, 'family=Stark\n');

    UTL_FILE.FCLOSE(fileHandler);
EXCEPTION
    WHEN utl_file.invalid_path THEN
        raise_application_error(-20000, 'ERROR: Invalid PATH FOR file.');
END;

Example's source: http://psoug.org/snippet/Oracle-PL-SQL-UTL_FILE-file-write-to-file-example_538.htm

At the same time you read from the file using Oracle external table.

CREATE TABLE parameters_table
(
    parameters_coupled VARCHAR2(4000)
)
ORGANIZATION EXTERNAL
(
    TYPE ORACLE_LOADER
    DEFAULT DIRECTORY test_dir
    ACCESS PARAMETERS
    (
        RECORDS DELIMITED BY NEWLINE
        FIELDS
        (
            parameters_coupled VARCHAR2(4000)
        )
    )
    LOCATION ('test_file.txt')
);

At this point you can write data to your table which has one column with coupled parameter and value, ie: 'firstname=Jon'

You can read it by Oracle

You can read it by any shell script because it is a plain text.

Then it is just a matter of a query, ie:

SELECT  MAX(CASE WHEN INSTR(parameters_coupled, 'firstname=')  = 1 THEN REPLACE(parameters_coupled, 'firstname=')  ELSE NULL END) AS firstname
,       MAX(CASE WHEN INSTR(parameters_coupled, 'lastname=')   = 1 THEN REPLACE(parameters_coupled, 'lastname=')   ELSE NULL END) AS lastname
,       MAX(CASE WHEN INSTR(parameters_coupled, 'occupation=') = 1 THEN REPLACE(parameters_coupled, 'occupation=') ELSE NULL END) AS occupation
FROM    parameters_table;