DP: Longest Increasing Subsequence Thought Process & Solution

Question

For the Longest Increasing Subsequence problem I envisioned keeping a DP array that is always in order keeping the max value at the farthest end. Something that would look like this:

{1, 1, 2, 3, 3, 4, 5, 6, 6, 6}

The thought process I followed to produce my first incorrect solution was, we want to look at the entire array starting with only the first element, calculate the LIS, then incrementally add on a value to the end of our array. While doing this, we incrementally calculate the LIS in our DP array to the LIS of the old subarray plus the new element we added on. This means at index i of the dp array exists the value of the LCS of the subarray of length i .

More clearly put

array => {5, 6, 7, 1, 2, 3, 4}
dp    => {1, 2, 3, 3, 3, 3, 4}

This way the very last entry of the DP array will be the LIS of the current array. This would act as our invariant, so when we get the end, we can be assured that the last value is the only one we need. It then dawned on me that while we're traversing an array with a DP kinda feel, the next value does not depend on any of the previously tabulated values in the array, so this method is the same as maintaining a maxLIS variable, a pattern I've seen in many O(n) solutions. So my closest-to-correct solution is as follows:

1.) Save a copy of the input array/vector as old

2.) Sort the original input array

3.) Traverse the sorted array, incrementing a variable longest by one every time the next value (which should be larger than the current`) appears before the current in the original array.

4.) Return longest

The code would be ~this:

int lengthOfLIS(vector<int>& seq) {
  if (!seq.size()) return 0;
  vector<int> old = seq;

  sort(seq.begin(), seq.end());

  int longest = 1;

  for (int i = 1; i < seq.size(); ++i) {
    if (seq[i] > seq[i-1] && find(old.begin(), old.end(), seq[i]) - old.begin() > find(old.begin(), old.end(), seq[i-1]) - old.begin()) longest++;
  }
  return longest;
}

Where we have the find method (I'm assuming a linear operation) we could make a constant operation by just making a data structure to store the original index of the value along with the the value itself so we don't have to do any traversing to find the index of an element in the original array ( old ). I believe this would be an O(nlog(n)) solution however fails with this input array: [1,3,6,7,9,4,10,5,6] . CHECK HERE

Finally I did some research and I found that all solution guides I have read sneak in the fact that their solution keeps the values of their DP array not in order, but instead like this: A value in the DP array represents the length of an increasing subsequence with the last value of the subsequence being the value of originalArray[index] .

More clearly put,

array => {5, 6, 7, 1, 2, 3}
dp    => {1, 2, 3, 1, 2, 3}

Here, where 5 is the last value of an increasing subsequence, no values come before it so it must be of length 1. If 6 is the last value of an increasing subsequence, we must look at all values before it to determine how long a subsequence ending with 6 can be. Only 5 can come before it, thus making the longest increasing subsequence thus far 2. This continues, and you return the maximum value in the DP array. Time complexity for this solution is O(n^2) , standard naive solution.

Questions:

I'm curious as to how I can think about this problem correctly. I want to fine-tune my thought process so that I can come up with an optimal solution from scratch (that's the goal at least) so I'd like to know

1.) What property of this problem should've triggered to me to use a DP array differently than how I would've used it? In hindsight, my original way was simply equivalent to keeping a max variable but even then I struggle seeing a property of this problem that would trigger the thought `Hey, the value of an entry in my DP array at index i should be the length of the increasing subsequence ending with originalArray[i]. I'm struggling to see how I should've come up with that.

2.) Is it possible to get my proposed O(nlog(n)) solution to work? I know an O(nlog(n)) solution exists, but since I can't get mine working I think I need a nudge in the right direction.

Answer 1

I admit, it is an interesting question and i do not have exact answer to it but i guess i can give you a nudge in right direction. So here it goes:

While facing with such dilemma, I would usually turn to the basics. Like in your case go through definition of Dynamic Programming. It has two properties:

1. Overlapping Subproblems
2. Optimal Substructure.

You can easily find these property reflecting in standard solution but not yours. You can read about them in cormen or just google them in context of DP.

In my opinion your solution is not a DP, you just found some pattern and your are trying to solve based on this pattern. If you are not getting the solution, it means that either your pattern is wrong or your solution is overlooking something. In scenarios like this try to prove, mathematically, that the pattern you are observing is correct and prove that the solution should also work.

Give me some more time, while i work through your solution but mean while you can also try to develop a proof for your solution.