比较HashMap和Value中的键

Question

I have a HashMap as follows-

HashMap<String, Integer> BC = new HashMap<String, Integer>();

which stores as keys- "tokens/tages" and as values- "frequency of each tokens/tags".

Example-

"the/at" 153
"that/cs" 45
"Ann/np" 3

I now parse through each key and check whether for same token say "the" whether it's associated with more than one tag and then take the largest of the two.

Example-

"the/at" 153
"the/det" 80

Then I take the key- "the/at" with value - 153 .

The code that I have written to do so is as follows-

private HashMap<String, Integer> Unigram_Tagger = new HashMap<String, Integer>();

for(String curr_key: BC.keySet())
        {
            for(String next_key: BC.keySet())
            {
                if(curr_key.equals(next_key))
                    continue;
                else
                {
                    String[] split_key_curr_key = curr_key.split("/");
                    String[] split_key_next_key = next_key.split("/");

                    //out.println("CK- " + curr_key + ", NK- " + next_key);

                    if(split_key_curr_key[0].equals(split_key_next_key[0]))
                    {
                        int ck_v = 0, nk_v = 0;
                        ck_v = BC.get(curr_key);
                        nk_v = BC.get(next_key);

                        if(ck_v > nk_v)
                            Unigram_Tagger.put(curr_key, BC.get(curr_key));
                        else
                            Unigram_Tagger.put(next_key, BC.get(next_key));
                    }
                }
            }
        }

But this code is taking too long to compute since the original HashMap 'BC' has 68442 entries which comes approximately to its square = 4684307364 times (plus some more).

My question is this- can I accomplish the same output using a more efficient method?

Thanks!

Answer 1

Create a new

Map<String,Integer> highCount = new HashMap<>();

that will map tokens to their largest count.

Make a single pass through the keys.

Split each key into its component tokens.

For each token, look in highMap . If the key does not exist, add it with its count. If the entry already exists and the current count is greater than the previous maximum, replace the maximum in the map.

When you are done with the single pass the highCount will contain all the unique tokens along with the highest count seen for each token.

Note: This answer is intended to give you a starting point from which to develop a complete solution. The key concept is that you create and populate a new map from token to some "value" type (not necessarily just Integer ) that provides you with the functionality you need. Most likely the value type will be a new custom class that stores the tag and the count.

Answer 2

The slowest part of your current method is due to the pairwise comparison of keys. First, define a Tuple class:

public class Tuple<X, Y> { 
  public final X x; 
  public final Y y; 
  public Tuple(X x, Y y) { 
    this.x = x; 
    this.y = y; 
  } 
} 

Thus you can try an algorithm that does:

1. Initializes a new HashMap<String, Tuple<String, Integer>> result
2. Given input pair (key, value) from the old map, where key = "a/b" , check whether result.keySet().contains(a) and result.keySet().contains(b) .
3. If both a and b is not present, result.put(a, new Tuple<String, Integer>(b, value) and result.put(b, new Tuple<String, Integer>(a, value))
4. If a is present, compare value and v = result.get(a) . If value > v , remove a and b from result and do step 3. Do the same for b . Otherwise, get the next key-value pair.

After you have iterated through the old hash map and inserted everything, then you can easily reconstruct the output you want by transforming the key-values in result .

Answer 3

A basic thought on the algorithm:

1. You should get the entrySet() of the HashMap and convert it to a List:

    ArrayList<Map.Entry<String, Integer>> list = new ArrayList<>(map.entrySet()); 

2. Now you should sort the list by the keys in alphabetical order. We do that because the HashMap has no order, so you can expect that the corresponding keys might be far apart. But by sorting them, all related keys are directly next to each other.

    Collections.sort(list, Comparator.comparing(e -> e.getKey())); 

   The entries "the/at" and "the/det" will be next to each other, thanks to sorting alphabetically.

3. Now you can iterate over the entire list while remembering the best item, until you find a better one or you find the first item which has not the same prefix (eg "the").

    ArrayList<Map.Entry<String, Integer>> bestList = new ArrayList<>(); // The first entry of the list is considered the currently best item for it's group Map.Entry<String, Integer> currentBest = best.get(0); String key = currentBest.getKey(); String currentPrefix = key.substring(0, key.indexOf('/')); for (int i=1; i<list.size(); i++) { // The item we compare the current best with Map.Entry<String, Integer> next = list.get(i); String nkey = next.getKey(); String nextPrefix = nkey.substring(0, nkey.indexOf('/')); // If both items have the same prefix, then we want to keep the best one // as the current best item if (currentPrefix.equals(nextPrefix)) { if (currentBest.getValue() < next.getValue()) { currentBest = next; } // If the prefix is different we add the current best to the best list and // consider the current item the best one for the next group } else { bestList.add(currentBest); currentBest = next; currentPrefix = nextPrefix; } } // The last one must be added here, or we would forget it bestList.add(currentBest); 

4. Now you should have a list of Map.Entry objects representing the desired entries. The complexity should be n(log n) and is limited by the sorting algorithm, while grouping/collection the items has a complexity of n.

Answer 4

import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.TreeMap;
import java.util.stream.Collectors;

public class Point {

    public static void main(String[] args) {
        HashMap<String, Integer> BC = new HashMap<>();
        //some random values
        BC.put("the/at",5);
        BC.put("Ann/npe",6);
        BC.put("the/atx",7);
        BC.put("that/cs",8);
        BC.put("the/aty",9);
        BC.put("Ann/np",1);
        BC.put("Ann/npq",2);
        BC.put("the/atz",3);
        BC.put("Ann/npz",4);
        BC.put("the/atq",0);
        BC.put("the/atw",12);
        BC.put("that/cs",14);
        BC.put("that/cs1",16);
        BC.put("the/at1",18);
        BC.put("the/at2",100);
        BC.put("the/at3",123);
        BC.put("that/det",153);  
        BC.put("xyx",123);
        BC.put("xyx/w",2);  
        System.out.println("\nUnsorted Map......");
        printMap(BC); 

        System.out.println("\nSorted Map......By Key"); 
        //sort original map using TreeMap, it will sort the Map by keys automatically.
        Map<String, Integer> sortedBC = new TreeMap<>(BC);
        printMap(sortedBC);
        //  find all distinct prefixes by spliting the keys at "/"
        List<String> uniquePrefixes = sortedBC.keySet().stream().map(i->i.split("/")[0]).distinct().collect(Collectors.toList());
        System.out.println("\nuniquePrefixes: "+uniquePrefixes);        

        TreeMap<String,Integer> mapOfMaxValues = new TreeMap<>();
        // for each prefix from the list above filter the entries from the sorted map 
        // having keys starting with this prefix 
        //and sort them by value in descending order and get the first which will have the highst value
        uniquePrefixes.stream().forEach(i->{ 
                Entry <String,Integer> e = 
                sortedBC.entrySet().stream().filter(j->j.getKey().startsWith(i))
                .sorted(Map.Entry.comparingByValue(Comparator.reverseOrder())).findFirst().get();

                mapOfMaxValues.put(e.getKey(), e.getValue());
            });

        System.out.println("\nmapOfMaxValues...\n");
        printMap(mapOfMaxValues);  
    }
    //pretty print a map
    public static <K, V> void printMap(Map<K, V> map) {
        map.entrySet().stream().forEach((entry) -> {
            System.out.println("Key : " + entry.getKey()
                    + " Value : " + entry.getValue());
        });
    }
}

// note: only tested with random values provided in the code 
// behavior for large maps untested