How to migrate documents in Firestore?

Question

I need to get an array of documents from one collection and migrate this documents into another one collection.

So i create an query to get all documents from first collection:

const orderQuery = db.collection("FIRST_COLLECTION").where("status", "==", "ACTIVE").get();

const orders: Order[] = mapToObject<Order>(orderQuery); //here I map query from firestore to objects

and iterate over this array and add to other one collection:

const secondCollectionQuery = db.collection("SECOND_COLLECTION").doc("ID").collection("THIRD_COLLECTION");

for await (const order from orders) {
    secondCollectionQuery.add(order);
}

is there a better way to migrate documents from one collection to another with firebase? thanks for any help!

PS. I also want to remove document from collection when he is migrated to another one collection, I do not want to have duplicate

Answer 1

You can push those add() in an array and then use Promise.all() to run them together (or use Batched writes and migrate them in batches of 500). Also the promise returned by first query doesn't seemed to be handled properly. Try running the following async function.

async function migrateDocs() {
  const oldColRef = db.collection("FIRST_COLLECTION")
  const newColRef = db.collection("SECOND_COLLECTION")

  const orderQuery = await oldColRef.where("status", "==", "ACTIVE").limit(500).get();

  const addBatch = db.batch();
  const delBatch = db.batch();

  orderQuery.docs.forEach(doc => {
    addBatch.set(newColRef.doc(doc.id), doc.data())
    delBatch.delete(oldColRef.doc(doc.id))
  })

  // add docs in new collection
  await addBatch.commit();

  // remove same docs from old collection
  await delbatch.commit();

  console.log("500 docs migrated")
}

Since a batched write can contain only 500 operation, we have that .limit(500) so we only fetch 500 docs. You can also read all docs then create batches of 500 (so you don't have to run the same again and again manually) as shown above or run all the promises individually (I'd prefer batched writes).

Answer 2

Is there a better way to migrate documents from one collection to another with Firestore? PS. I also want to remove document from collection

The only way to move a document from one collection to another collection is the way you have described in your question, ie mimicking a "copy & paste" with the following sequence:

Create a copy of the source document in the target collection, ie read it from the source collection, get the document fields and create a new document in the target collection with these fields.

If you want to do a " cut & paste" (also removing the documents from the source collection), you'll need to delete these documents from the source collection after having completed the copy/paste sequence.

---

Note that Firestore offers the possibility to export and import documents using the managed export and import service and Cloud Storage , but you cannot export docs from one collection and import them in another collection.