NetSuite restlet write performance is poor

Question

Edit: I've redacted actual numbers and replaced them with pseudoswears because I've been told sharing performance data is against Netsuite's TOS.

I'm integrating our accounting system with NetSuite using restlets, and overall it has gone pretty well with the glaring exception of performance. I have figured out that nlapiLoadRecord is Satan's own child from a performance perspective, so I avoid it whenever possible favoring the search api and now my read restlets are pretty snappy. However, whenever I ask a restlet to write anything it is as slow as a tortoise stuck in cold tar. I'm generally profiling the nlapiSubmitRecord itself at between SLOW and REALLY DAMN SLOW seconds. This seems insane to me. No one would use NetSuite if the performance were always this poor for writing. I'll include a couple of code examples below. Any tips on speeding up NetSuite restlet write performance would be appreciated.

In this first one receivedInvoice is the incoming data, and findCurrencyCode and findCustomerByCustomerNumber are well performing functions that do those things. I just clocked this at a nearly unbelievable HOLY MONKEYS THAT IS SLOW seconds for a simple invoice with one line item, nearly all of the time passing while I waited for nlapiSubmitRecord to finish.

var createdInvoice = nlapiCreateRecord('invoice');

createdInvoice.setFieldValue('customform', Number(receivedInvoice.transactionType));
createdInvoice.setFieldValue('memo', receivedInvoice.message);
createdInvoice.setFieldValue('duedate', receivedInvoice.dateDue);

createdInvoice.setFieldValue('currency', findCurrencyCode(receivedInvoice.currencyUnit));
createdInvoice.setFieldValue('location', Number(receivedInvoice.location));
createdInvoice.setFieldValue('postingperiod', findPostingPeriod(receivedInvoice.datePosted));
var customer = findCustomerByCustomerNumber(receivedInvoice.customerNumber);

createdInvoice.setFieldValue('entity', customer.customerId );
createdInvoice.setFieldValue('custbody_end_user', customer.customerId );
createdInvoice.setFieldValue('department', customer.departmentId);

var itemCount = receivedInvoice.items.length;
for(var i = 0; i < itemCount; i++)
{
  createdInvoice.selectNewLineItem('item');
  createdInvoice.setCurrentLineItemValue('item', 'item',receivedInvoice.items[i].item);
  createdInvoice.setCurrentLineItemValue('item', 'quantity', receivedInvoice.items[i].quantity);
  createdInvoice.setCurrentLineItemValue('item', 'rate',receivedInvoice.items[i].price);
  createdInvoice.setCurrentLineItemValue('item', 'custcol_list_rate',receivedInvoice.items[i].price);
  createdInvoice.setCurrentLineItemValue('item', 'amount',receivedInvoice.items[i].totalAmount);
  createdInvoice.setCurrentLineItemValue('item', 'description',receivedInvoice.items[i].description);
  createdInvoice.commitLineItem('item');
}

var recordNumber = nlapiSubmitRecord(createdInvoice,false,true);

In this one I think I'm committing a performance heresy by opening the record in dynamic mode, but I'm not sure how else to get the possible line items. Simply opening a new record in dynamic mode clocks in at around SLOW seconds. Again, the submit is where most time is being eaten (often around OH DEAR SWEET MOTHER OF HORRIBLE seconds), although this one eats a decent amount of time as I mess with the line items, again presumably because I have opened the record in dynamic mode.

var customerPayment = nlapiCreateRecord('customerpayment',{recordmode: 'dynamic'});

customerPayment.setFieldValue('customer', parseInt(customerId));
customerPayment.setFieldValue('payment', paymentAmount);
customerPayment.setFieldValue('paymentmethod', paymentMethod);
customerPayment.setFieldValue('checknum', transactionId);
customerPayment.setFieldValue('currency', currency);
customerPayment.setFieldValue('account', account);

var applyCount = customerPayment.getLineItemCount('apply');

if(applyCount>0)
{
    for(var i=1;i<=applyCount;i++)
    {
        var thisInvoice = customerPayment.getLineItemValue('apply','refnum',i);

        if(thisInvoice == invoiceToPay)
        {
            customerPayment.selectLineItem('apply', i);
            customerPayment.setCurrentLineItemValue('apply','apply','T');
            customerPayment.setCurrentLineItemValue('apply', 'amount',paymentAmount);
            customerPayment.commitLineItem('apply');
        }
    }

}
nlapiSubmitRecord(customerPayment,false,true);

Answer 1

A few thoughts:

1. (just to get it off my chest) Integrating your accounting system with Netsuite just sounds odd. Netsuite is an accounting system and generally is the accounting system of record for the orgs that use it. If you are not using Netsuite for accounting you might want to consider what utility it has for the price and get off it.
2. When I integrate an external system with Netsuite I generally try to make it async. I do this by getting the raw info into a custom record and then kick off a scheduled script to process the queued update. This lets my api return quickly. When I process the queue I store errors in the queue record so that if anything comes up I can fix the data or code and re-submit it.
3. One apparently major source of slow transaction submit (aside from slow UE scripts) is the state of your books. I had a startup customer who did really well but they never closed their books and they were using IIRC Average Costing. Every time they saved a posting transaction Netsuite was recalculating the whole period (which at the point where things were grinding to a halt was about 2 years of records for a very busy site). When they started closing periods transaction save times went way down. When they converted to standard costing transaction save times went down again (I imagine LIFO would be faster than Avg and slower than standard but YMMV)

Also notes on your code.

The normal way to create an invoice is

nlapiTransformRecord('customer', customer.customerId, 'invoice'); or nlapiTransformRecord('customer', customer.customerId, 'invoice', {recordmode:'dynamic'});

I've never tested whether this has an effect on submit times but it might help since NS will start the save from a slightly better place (grasping at straws but with NS sometimes non-obvious changes have speed benefits)

Also not sure how changing the customform in dynamic mode works. When you know the form you can also add that to the init defaults:

{recordmode:'dynamic', customform:receivedInvoice.transactionType}

Answer 2

One of the reasons why you Submit is slow might be there are a lot of User Event scripts attached to the record. Since it is the Restlet that saves the record, it will trigger the User Event scripts.