More efficient: large array or many scalars

Question

Working in an embedded (PIC) environment, programming in c.

I have to keep track of 1000 values for a variable (history) and return the moving average of those values. I'm just wondering if it will be more efficient in terms of speed, ROM and RAM usage if I use an array or 1000 16bit variables. Is there a difinitive answer to that? Or would i have to just try both and see what works best?

Thanks.

EDIT: Hmm... I already ran into another problem. The compiler limits me to an array size maximum of 256.

EDIT2:

For clarification... the code fires about 1000 times a second. Each time, a value for history[n] (or history_n) is calculated and stored. Each time I need to calculate the average of the 1000 most recent history values (including current). So (history[1000] + history[999] + ... + history[1] + history[0]) / 1000; or something to that effect. Obviously each time I need to kick out the oldest and add the newest.

EDIT3:

I've re-worked the code such that now the 256 array size is not an issue. A sample size of around 100 is now suitable.

Answer 1

Assuming you need to keep the history, and given your 256 element array limit, here's a way to manage it:

int history1[256];
int history2[256];
int history3[256];
int history4[256];
int* arrays[] = {history1,history2,history3,history4}
int idx=0;
int sum = 0;
int n = 0;

int updateAverage(int newValue)
{
  int ai = (idx++)>>8;
  int* target = arrays[ai]+(idx&0xFF);

  sum -=*target;
  *target = newValue;
  sum += *target;
  n++;
  n=n<1000?n:1000;
  idx = (idx<1000)?idx:0;
  return sum/n;
}

Answer 2

I'm not sure if I completely understand your question. Are you asking for the difference between the code generated for 'short history[1000]', and 'short history1, history2, ..., history1000;'?

Both should use similar amounts of RAM: each entry is going to take be stored in a single file register (assuming you're using a 16-bit PIC). The code to calculate the average of the latter is going to be ugly though, and will likely take quite a bit of ROM, as it is going to need to reference each value separately (rather than just offsetting the base).

Edit: The reason for the 256 element limit is because of file register paging on the PIC. You can't address a larger array by just offsetting the base register, because you may need to request a page change.

Do you absolutely have to calculate a running average? Or can you do an overall average? If an overall average is okay, then use a variant of Alphaneo's answer: just keep the sum, and the number of values collected in two variables, and divide any time you need the average.

Answer 3

If you use an array the generated code will be much smaller. I'm not sure on this but I think an array access would use less memory since you don't have to keep track of multiple variables you just have a pointer to one Chunk. If your stack size is an issue an Array on the heap may be the best way to go since C variables are stored on the stack.

Answer 4

If you want to calculate an average by storing it in an array, will be definitely more expensive than calculating at run-time.

Reason-1: If you calculate it at run-time, you will justing keep adding for example look at the following flow

    init-0: _tempSum_ = 0
    step-1: Read current value to _currVal_
    step-2: Add current value to _tempSum_
    step-3: Check if we have required of values _num_ (eg. 1000), if not goto-1
    step-4: Calculate _avg_ = _tempSum_ / _num_ and return
    step-5: goto init-0

If you store in a temp array of 1000 values, actually things you will all the steps from init-0 to step-5, except that you will end up using a 1000 value array.

It might be slower, based on the array access timing ... so beware

Answer 5

First, you can change your linker file to allow a larger section. You will then have to put your history array in that section using pragmas.

Second, the array method is much better. To improve the performance you will also need a 32-bit integer to keep a running total of the history array.

For each firing of the history function you will subtract the oldest value from the HistoryRunningTotal and add in the new history value. You will also need a OldestHistoryIndex variable to keep track of where the newest value will go (and overwrite the old history).

Answer 6

If you have enough memory for the 1000 values, then why not statically allocate the memory, make a rotating buffer and move a pointer through the values to calculate the sum. (if the running avg isn't what you need). Just keep putting the new value over the oldest value, calculate the sum of the 1000 values and divide by 1000. So actually two pointers, one for inserting the new value and one to iterate over the whole buffer.

Answer 7

The compiler limits arrays to 256? That's pretty bad. That imposes a burden on the programmer that the compiler really should be taking care of. Are you sure there isn't a compiler option for eg a "large" memory model? If not, investigate a different micro and/or compiler.

Or, to keep things small, consider using a small IIR filter rather than a large FIR filter.

Anyway, to answer your original question: an array is the logical choice for such an algorithm, for the benefits of sensible (maintainable) code and small code size.

Answer 8

If the array is dynamically allocated, then using the static variables will be faster. If the array is statically allocated, then the compiler should be able to optimize it to be equivalently fast as static variables. Try both on some trivial code and profile it.