Ksh cannot allocate memory linux

Question

I have two big arrays of strings ( ). 字符串 ）。
I create them with set -A command. And I need to figure out which of strings in first array don't have equal string in second.
My code:

for i in {0..${#hard_drive_files[*]}}; do          

    has_reference=false
    for j in {0..${#files_in_db[*]}}; do    
        if [[ ${files_in_db[j]} == ${hard_drive_files[i]} ]]; then
            has_reference=true
            break
        fi  
    done 

    if [[ $has_reference == false ]]; then
        echo "${hard_drive_files[i]}"
    fi     
done

This part of code "eats" too much memory.
At the end of execution value of used memory is ~80000 MB
After this part of code I try to archive some files but get cannot fork [Cannot allocate memory]
Is there a solution for such problem?

PS kshVersion=Version AJM 93t+ 2010-02-02
To figure out how much ram memory is used I execute free -m

Answer 1

I assume there is specific reason to do it in Ksh ? Try to make some approximation of how much memory is needed for such table, and compare to amount of RAM and SWAP. I bet it's not a program specifically, but some per-process memory/swap usage limit in limits.conf or sysctl.conf.

You could also try to split data to some groups, by concept, like first letter of name or something, to decrease amount of needed memory. Your code probably is far from optimal, it will be better first to gather all information you need and then reuse it, instead of repeating whole procedure in nested loop like you're trying to do.