How to compare an array with an new created List java

Question

i have a String array with information like this:

name          street         streetnumber         City      house     flat
jetsons     jetstreet        12                  london       yes      no
jetsons     jetstreet        10                  washingston  n        y
jetsons     jetstreet        10                  washingston  n        y
jetsons     jetstreet        10                  washingston  yes      no 
ALF         alfStreet         3                  Shanghai      y        y

...and so on

now the exercise is to create an new list with unique data which is analyzed.

livingDataArray analyzedDataList

while(livingDataArray=reader.readLine() != null){
        street = livingDataArray[1];
        streetNumber = livinDataArray[2];
        city = livingDataArray[3;]

    if(analyzedDataList.isEmpty()) {
        createNewEntry in analyzedDataList(); // that line is fine. ;)
    } else {
        int analyzedDataSize = analyzedData.size();
        for (int i = 0; i <= analyzedDataSize; i++){

                  if(analyzedData.get(i)[1] == street && 
                    analyzedData.get(i)[2] == streetNumber &&
                    analyzedData.get(i)[3] == city ) {

                   categorize(); // this line is fine also
                   addToAnalyzedData();
                  break;
                  } else if (!(analyzedData.get(i)[1] == street && 
                    analyzedData.get(i)[2] == streetNumber &&
                    analyzedData.get(i)[3] == city) && (i+1 ==
                    livingData.size())) {

                             categorize();
                             addToAnalyzedData();
                             break;
                    }


             }
        }
}

My question is that efficient enough to use it for really big data? Like 100.000 rows and more? Because I'm not about the if else statements. Could anybody help me?

Answer 1

String comparison works via equals , not == ( How do I compare strings in Java? ). Next point: This looks like the implementation in java of a plain SELECT DISTINCT * FROM someWhere -statement in SQL. So why not simply outsource the code to a database? If that's not possible, a Set would be most likely the most efficient collection. Though i'd recommend SQL to improve performance a lot and save resources on your local PC. One final note: Modifying data in a loop over the same data like here:

int analyzedDataSize = analyzedData.size();
for (int i = 0; i <= analyzedDataSize; i++){
    ...
        addToAnalyzedData();

is extremely prone to bugs/exceptions. For eg you're retrieving and modifying a collection in the loop mentioned above, without updating the size of the collection. In this example, this behavior won't do any damage, but you should handle this rather carefully.