How to incorporate SQL with my Python .txt parser

Question

Currently I have hard coded the fields for my jobs I need my code to get information from a .txt file. Rather then having it hard coded I want to put the fields in a table in a database and write a script to just read it from the table instead.

Current Code:

import pprint
import re


def field_Extract(fileLines, fieldsArray, delimit):
    for line in fileLines:
        for field in fieldsArray:
            if line.startswith(field):
                key, value = line.split(delimit)
                print(key.rstrip(), " : ", value.strip())


test_file = open('/parse.txt', 'r+')

********************************************************************

THE PART BELOW IS WHERE I WANT THE PROGRAM TO GET IT FROM A TABLE 
ALREADY CREATED BY ME AND FILLED IN WITH THE REQUIRED FIELDS. I'LL LIKE 
TO USE ONE TABLE FOR THIS ALTHOUGH THE FIELDS WILL VARY IN COLUMN FOR 
EACH DATA

*********************************************************************

job1 = ['NUMBER OF INPUT RECORDS', 'RECORDS WRITTEN TO ADJUSTMENT FILE', 'RECORDS WRITTEN TO ERROR FILE',
        'COD RECORDS PASSED CEV VALIDATION',
        'COD RECORDS FAILED CEV VALIDATION', 'ADS RECORDS PASSED CEV VALIDATION', 'ADS RECORDS FAILED CEV VALIDATION',
        'PAR ACCESSORIAL REFUND RECORDS']
job2 = ['6010 TOTAL DELIVERY RECORDS READ', '6050 TOTAL ERROR RECORDS WRITTEN',
        '7025 TOTAL PKG DERIVED DATA ROWS INSERTED', '7035 TOTAL PKG DERIVED DATA ROWS UPDATED',
        '7027 TOTAL ACC DERIVED DATA ROWS INSERTED', '7030 TOTAL DELIVERY ROWS UPDATED', 'TSOURCE DERIVED ZONE',
        'SLI INVALID FOR SAT', 'COUNTS FOR COUNTRY',
        'RECORDS READ', 'ERRORS WRITTEN', 'TKPUGPD INSERTED', 'TPKGUPD UPDATED', 'TACCUPD INSERTED',
        'CEV ASY DROP & WRITTEN', 'ZONE NOT FOUND ERRORS']
job3 = ['6010 TOTAL DELIVERY RECORDS READ', '6050 TOTAL ERROR RECORDS WRITTEN', '6030 TOTAL FCB RECORDS WRITTEN',
        '2317 TOTAL NON GENUINE DUP DELIV SCANS', '2270 TOTAL INVALID RESID/COMMER INDICATOR',
        '8130 TOTAL NON LTR CNTNR WITH 0 WEIGHT',
        '1010 TOTAL SHIPPERS NOT IN CRIS', '1080 TOTAL SHIPPERS ADDRESS NOT IN CRIS',
        '1070 TOTAL SHIPPERS CENTER NOT IN CRIS',
        '2310 TOTAL COD INVALID FOR ORIG CNY/ZONE ', '2308 TOTAL EAM SERVICE DOWNGRADED',
        '2309 TOTAL EAM SAT ACCESSORIALS DROPPED']
job4 = ['6010 SHIPMENT RECORDS READ','6030 FCB RECORDS WRITTEN']
job5 = ['TOTAL PACKAGE RECORDS READ', 'TOTAL TPKGUPD RECORDS UPDATED', 'TOTAL TACCUPD RECORDS INSERTED']

# Jobs start and end strings
jobStartStr = ['BEGINNING N260RV30', 'PROGRAM N260CV10 BEGINNING', 'N260CV08 BEGINNING',
               'PROGRAM N260GV12 BEGINNING', 'PROGRAM N260CW13 BEGINNING']
jobEndStr = ['END OF    N260RV30', 'SUCCESSFUL END N260CV10', 'SUCCESSFUL END N260CV08',
             'SUCCESSFUL END N260GV12', 'SUCCESSFUL END N260CW13']

currentJob = -1
currentJobData = []
startAppending = False
for line in test_file:
    # If job start found, gathar job lines
    if startAppending == True:
        currentJobData.append(line)

    # Get the current job
    for jobStart in jobStartStr:
        if jobStart in line:
            currentJob = jobStartStr.index(jobStart) + 1
            # Set the flag to start gathering job lines
            startAppending = True

    # Set the correct job
    if currentJob == 1:
        job = job1
    elif currentJob == 2:
        job = job2
    elif currentJob == 3:
        job = job3
    elif currentJob == 4:
        job = job4
    elif currentJob == 5:
        job = job5
    else:
        currentJob = -1

    # Check job end using End string list
    for jobEnd in jobEndStr:
        if (currentJob != -1) and (jobEnd in line):
            # As a job end found, stop gathering lines
            startAppending = False
            print('########============ NEW JOB STARTS HERE ===========#########')
            # Execute valid jobs
            if currentJob != -1:
                field_Extract(currentJobData, job, ':')
            # Erase completed job lines
            currentJobData = []
            # Set job to invalid job
            currentJob = -1

My end goal is to store all the values into another access table after figuring out my first issue.

Answer 1

To get the contents of the job1 ... job5 lists from a database into Python lists, create a table with the following format:

create table jobdata ( 
    job integer not null,
    joblabel text not null,
    constraint pk_jobdata primary key (job, joblabel)
    );

After you populate this table, it should look like this:

+------+------------------------------------+
|  job |              joblabel              |
+------+------------------------------------+
|    1 | NUMBER OF INPUT RECORDS            |
|    1 | RECORDS WRITTEN TO ADJUSTMENT FILE |
|    1 | RECORDS WRITTEN TO ERROR FILE      |
|    1 | COD RECORDS PASSED CEV VALIDATION  |
|    ... etc ...                            |
+------+------------------------------------+

To pull the data into Python lists, create your database connection, and use a function like the following:

def get_job_labels(db, jobno):
    curs = db.cursor()
    curs.execute("select joblabel from jobdata where job = %d;" % jobno)
    return [row[0] for row in curs.fetchall()]

where the function arguments are the database connection object and the job number, as in the job column of the table. The returned list can be assigned directly to the job1 ... job5 variables.

If you want or need to read the jobStartStr and jobEndStr from a database, you can use the same approach. You should use a different table, however, because the tables contain logically different data and the identifiers have different data types.