Using Pdf Sharp to merge a large amount of pdf's, running into system out of memory exception

Question

I've looked at questions similar and the answer is obviously to open and close the destination pdf so that the memory buffer doesn't run out. I am doing that, or at least i'm trying to, but it seems like even if i break out of the using statement that opens the pdf document. It does not close the memory buffer. Basically, the first time through I set the destination pdf to a new document, I copy the pdfs in chunks of 5000 ( each pdf is 222kb). Every chunk is then saved to the destination file path. Every following iteration, the destination pdf is set to the pdf that was saved the first time through. For some reason, even though I'm opening and closing the destination pdf, the memory buffer stays open.

I even added an obligatory close on the outPdf even though it's in a using statement out of desperation. How can I close the pdf and close the memory buffer so I can reopen the destination file and continue adding onto the pdf?

try
      {
        while (filesMerged < fileCount)
        {
          using (outPdf = firstTime ? new PdfDocument() : PdfReader.Open(path + "\\" + fileName, PdfDocumentOpenMode.Modify))
          {
            firstTime = false;
            for (int i = 0; i < 5000; i++)
            {
              if (filesMerged >= fileCount)
              {
                break;
              }
              pdfPath = filesInPath[index];

              using (var tempPdfFile = PdfReader.Open(pdfPath, PdfDocumentOpenMode.Import))
              {
                totalSize += tempPdfFile.FileSize;
                // helper method to copy pages
                CopyPages(tempPdfFile, outPdf);
              }
              filesMerged++;
              index++;
            }
            outPdf.Save(path + "\\" + fileName);
            totalOpens++;
          }
          outPdf.Close();
        }
      }