How to output the twilio sms logs in a csv file using Python API

Question

I am using this script for downloading the sms log file from the Twilio.

https://github.com/asplunker/twilio-app/blob/master/bin/get_sms_logs.py

In the first run its downloading the file properly but in the second run it throws " index out of range error "

So the error is suspected in the function :

def write_records():
# avoid duplicates

data = []
if os.path.exists(LOG_FILE):
    with codecs.open(LOG_FILE) as d:
        file_data = d.readlines()
        for line in file_data:
            print line
            date = line.split(',')[1]
            if  date == LAST_ENTRY:
                    data.append(date)

with codecs.open(LOG_FILE, 'a') as f:
    for record in reversed(RECORDS):
        if not record.split(',')[1] in data:
            f.write(record)
            f.write('\n')

I am not sure if the output csv file in the first run doesn't specify each record in a single line.

Any pointers would be greatly appreciated.

Answer 1

I would add some error handling to find the state of objects at the time of failure.

Have you ever used try/except? https://docs.python.org/2/tutorial/errors.html

Basically you could set it up like this

def write_records():
# avoid duplicates
    try:
        data = []
        if os.path.exists(LOG_FILE):
            with codecs.open(LOG_FILE) as d:
                file_data = d.readlines()
                for line in file_data:
                    print line
                    date = line.split(',')[1]
                    if  date == LAST_ENTRY:
                            data.append(date)

        with codecs.open(LOG_FILE, 'a') as f:
            for record in reversed(RECORDS):
                if not record.split(',')[1] in data:
                    f.write(record)
                    f.write('\n')

    except IndexError:
        #log variables here and examine the issue closely

Answer 2

First thing I can think of is that your CSV file is not properly read.

The out of index error is probably happening here:

date = line.split(',')[1]

Try adding a conditional for the date there:

date_ = line.split(',') date = date_[1] if len(date_) >= 1 else ""

But you REALLY REALLY should have a look at CSV from standard library

https://docs.python.org/2/library/csv.html

Answer 3

The FAQ on how to export your SMS/Call logs shows a CSV example in PHP. I'm a Python person myself, but I would use this to test it out quick and dirty depending on the situation.

<!--?php <br ?-->/**
 * Download the library from: https://github.com/twilio/twilio-php
 * Copy the 'Services' folder into a directory containing this file.
 */
require('Services/Twilio.php');

$account_sid = "ACXXXXXXXXX"; // Your Twilio account sid
$auth_token = "YYYYYYYYYYYY"; // Your Twilio auth token

// Download data from Twilio API
$client = new Services_Twilio($account_sid, $auth_token);
$messages = $client->account->sms_messages->getIterator(0, 50, array(
    'DateSent>' => '2012-09-01',
    'DateSent<' => '2012-09-30',
    //'From' => '+17075551234', // **Optional** filter by 'From'...
    //'To' => '+18085559876', // ...or by 'To'
));

// Browser magic
$filename = $account_sid."_sms.csv";
header("Content-Type: application/csv") ;
header("Content-Disposition: attachment; filename={$filename}");

// Write headers
$fields = array(
    'SMS Message SID', 'From', 'To', 'Date Sent',
    'Status', 'Direction', 'Price', 'Body'
);
echo '"'.implode('","', $fields).'"'."\n";

// Write rows
foreach ($messages as $sms) {
    $row = array(
        $sms->sid, $sms->from, $sms->to, $sms->date_sent,
        $sms->status, $sms->direction, $sms->price, $sms->body
    );
    echo '"'.implode('","', $row).'"'."\n";
}