如何在几乎整个postgresql数据库中查询每个项目的属性？ （使用sequelize和Node.js）

Question

I'm trying to scan through an entire tree in my database, looking for two properties ('title' and 'id') for every item, and then I need to check if there's a linked database table below the current table, and if so, I need to perform the same actions on that table.

Once I get the whole tree, I need to insert those into a master variable that I can then import elsewhere throughout my web app.

(I'm at the point where I'm pretty sure I'm reinventing the wheel. I've searched the Sequelize documentation, but if there's a simple solution, I didn't see it.)

Here's the relevant portions of my current code (it's not fully working yet, but should give the idea).

( You can find the full project repository by clicking this link here. )

```

const buildTopicTreeFromCurrentDatabase = (callback) => {
  let topicTree = [];
  let isFinished = false;
  const isSearchFinished = new Emitter();
  console.log(`emitter created`);
  isSearchFinished.on('finished', () => {
    console.log(`the search is done`);
    if (isFinished = true) {
      callback(null, this.primaryTopicsShort)
    };
  });
  /* need to go back and refactor -- violates DRY */
  this.primaryTopicsShort = [];
  PrimaryTopic.all()
  .then((primaryTopics) => {
    if (primaryTopics.length === 0) {
      return callback('no Primary Topics defined');
    }
    for (let i = 0; i < primaryTopics.length; i++) {
      this.primaryTopicsShort[i] = {
        title: primaryTopics[i].title,
        id: primaryTopics[i].id,
        secondaryTopics: []
      };
      PrimaryTopic.findById(this.primaryTopicsShort[i].id, {
        include: [{
          model: SecondaryTopic,
          as: 'secondaryTopics'
        }]
      })
      .then((currentPrimaryTopic) => {
        if (currentPrimaryTopic.secondaryTopics.length !== 0) {
          for (let j = 0; j < currentPrimaryTopic.secondaryTopics.length; j++) {
            this.primaryTopicsShort[i].secondaryTopics[j] = {
              title: currentPrimaryTopic.secondaryTopics[j].title,
              id: currentPrimaryTopic.secondaryTopics[j].id,
              thirdTopics: []
            };
            SecondaryTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].id, {
              include: [{
                model: ThirdTopic,
                as: 'thirdTopics'
              }]
            })
            .then((currentSecondaryTopic) => {
              if (currentPrimaryTopic.secondaryTopics.length - 1 === j) {
                isSearchFinished.emit('finished');
              }
              if (currentSecondaryTopic.thirdTopics.length !== 0) {
                for (let k = 0; k < currentSecondaryTopic.thirdTopics.length; k++) {
                  this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k] = {
                    title: currentSecondaryTopic.thirdTopics[k].title,
                    id: currentSecondaryTopic.thirdTopics[k].id,
                    fourthTopics: []
                  };
                  ThirdTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].id, {
                    include: [{
                      model: FourthTopic,
                      as: 'fourthTopics'
                    }]
                  })
                  .then((currentThirdTopics) => {
                    if (currentThirdTopics.fourthTopics.length !== 0) {
                      for (let l = 0; l < currentThirdTopics.fourthTopics.length; l++) {
                        this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[k] = {
                          title: currentThirdTopics.fourthTopics[l].title,
                          id: currentThirdTopics.fourthTopics[l].id,
                          fifthTopics: []
                        }
                        FourthTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[l].id, {
                          include: [{
                            model: FifthTopic,
                            as: 'fifthTopics'
                          }]
                        })
                        .then((currentFourthTopics) => {
                          if (currentFourthTopics.fifthTopics.length !== 0) {
                            for (let m = 0; m < currentFourthTopics.fifthTopics.length; m++) {
                              this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[k].fifthTopics[m] = {
                                title: currentFourthTopics.fifthTopics[m].title,
                                id: currentFourthTopics.fifthTopics[m].id
                              }
                            }
                          }
                        })
                        .catch((err) => {
                          callback(err);
                        });
                      }
                    }
                  })
                  .catch((err) => {
                    callback(err)
                  });
                }
              }
            })
            .catch((err) => {
              callback(err);
            })
          }
        }
      })
      .catch((err) => {
        callback(err);
      })
    }
  })
  .catch((err) => {
    callback(err);
  });
};

There are a few problems with this code.

First, I need to be using a DRY and recursive solution, since the database structure may change in the future.

Second, I've played around a lot with the 'finished' emitter, but I haven't figured out yet how to place it so that the event is emitted at the end of searching the database, and also so that I don't cycle back through the database multiple times.

I've been working on the following recursive solution, but the hours keep stretching by and I don't feel like I'm getting anywhere.

const buildDatabaseNames = (DatabaseNameStr, callback) => {
  let camelSingular = DatabaseNameStr.slice(0,1);
  camelSingular = camelSingular.toLowerCase();
  camelSingular = camelSingular + DatabaseNameStr.slice(1, DatabaseNameStr.length);
  let camelPlural = DatabaseNameStr.slice(0,1);
  camelPlural = camelPlural.toLowerCase();
  camelPlural = camelPlural + DatabaseNameStr.slice(1, DatabaseNameStr.length) + 's';
  let databaseNameStr = `{ "capsSingular": ${"\"" + DatabaseNameStr + "\""}, "capsPlural": ${"\"" + DatabaseNameStr + 's' + "\""}, "camelSingular": ${"\"" + camelSingular + "\""}, "camelPlural": ${"\"" + camelPlural + "\""} }`;
  let names = JSON.parse(databaseNameStr);
  return callback(names);
};

const isAnotherDatabase = (DatabaseName, id, NextDatabaseName) => {
  DatabaseName.findById({
    where: {
      id: id
    }
  })
  .then((res) => {
    if (typeof res.NextDatabaseName === undefined) {
      return false;
    } else if (res.NextDatabaseName.length === 0) {
      return false;
    } else {
      return true;
    }
  })
  .catch((err) => {
    console.error(err);
    process.exit();
  });
};

const searchDatabase = (first, i, current, next, list, callback) => {
  if (typeof first === 'string') {
    first = buildDatabaseNames(first);
    current = buildDatabaseNames(current);
    next = buildDatabaseNames(next);
  }
  if (first === current) {
    this.first = current;
    let topicTree = [];
    const isSearchFinished = new Emitter();
    console.log(`emitter created`);
    isSearchFinished.on('finished', () => {
      console.log(`the search is done`);
      callback(null, this.primaryTopicsShort);
    });
  }
  current.CapsSingular.all()
  .then((res) => {
    current.camelPlural = res;
    if (if current.camelPlural.length !== 0) {
      for (let j = 0; j < currentParsed.camelPlural.length; j++) {
        if (first === current) {
          this.first[i].current[j].title = current.camelPlural[j].title,
          this.first[i].current[j].id = current.camelPlural[j].id
          next.camelSingular = []
        } else {
          this.first[i]..current[j].title = current.camelPlural[j].title,
          this.first[i].id = current.camelPlural[j].id,
          next.camelSingular = []
        }
        let isNext = isAnotherDatabase(current.)
        searchDatabase(null, j, next, list[i + 1], list, callback).bind(this);
      }
    } else {
      callback(null, this.first);
    }
  })
  .catch((err) => {
    callback(err);
  });
};

I decided to stop and ask for help, as I just realized that in order to make the properties ( this.first[i].current[j].title = current.camelPlural[j].title ) on each recursive iteration accurate, I'll have to do a JSON.stringify , alter the string for the next iteration, place all the required iterations of itinto a variable, pass it into the next recursion, and then do JSON.parse again afterwards. It seems like I'm making this ridiculously complicated?

Any help is appreciated, thank you.

Answer 1

While I wasn't able to make a recursive, generic solution, I did at least find some result.

const buildTopicTreeFromCurrentDatabase = (callback) => {
  let topicTree = [];
  const isSearchFinished = new Emitter();
  isSearchFinished.on('finished', () => {
    if (isFinished === 0) {
      callback(null, this.primaryTopicsShort);
    };
  });
  /* need to go back and refactor -- violates DRY */
  this.primaryTopicsShort = [];
  let isFinished = 0;
  isFinished = isFinished++;
  PrimaryTopic.all()
  .then((primaryTopics) => {
    isFinished = isFinished + primaryTopics.length;
    if (primaryTopics.length === 0) {
      return callback('no Primary Topics defined');
    }
    for (let i = 0; i < primaryTopics.length; i++) {
      if (i === 0) {
        var isLastPrimary = false;
      };
      this.primaryTopicsShort[i] = {
        title: primaryTopics[i].title,
        id: primaryTopics[i].id,
        secondaryTopics: []
      };
      PrimaryTopic.findById(this.primaryTopicsShort[i].id, {
        include: [{
          model: SecondaryTopic,
          as: 'secondaryTopics'
        }]
      })
      .then((currentPrimaryTopic) => {
        isFinished = isFinished - 1 + currentPrimaryTopic.secondaryTopics.length;
        if (i === primaryTopics.length - 1) {
          isLastPrimary = true;
        };
        if (currentPrimaryTopic.secondaryTopics.length === 0 && isLastPrimary) {
          isSearchFinished.emit('finished');
        } else if (currentPrimaryTopic.secondaryTopics.length !== 0) {
          for (let j = 0; j < currentPrimaryTopic.secondaryTopics.length; j++) {
            if (j === 0) {
              var isLastSecondary = false;
            }
            this.primaryTopicsShort[i].secondaryTopics[j] = {
              title: currentPrimaryTopic.secondaryTopics[j].title,
              id: currentPrimaryTopic.secondaryTopics[j].id,
              thirdTopics: []
            };
            SecondaryTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].id, {
              include: [{
                model: ThirdTopic,
                as: 'thirdTopics'
              }]
            })
            .then((currentSecondaryTopic) => {
              isFinished = isFinished - 1 + currentSecondaryTopic.thirdTopics.length;
              if (j === currentPrimaryTopic.secondaryTopics.length - 1) {
                isLastSecondary = true;
              }
              if (currentSecondaryTopic.thirdTopics.length === 0 && isLastPrimary && isLastSecondary) {
                isSearchFinished.emit('finished');
              } else if (currentSecondaryTopic.thirdTopics.length !== 0) {
                for (let k = 0; k < currentSecondaryTopic.thirdTopics.length; k++) {
                  if (k === 0) {
                    var isLastThird = false;
                  };
                  this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k] = {
                    title: currentSecondaryTopic.thirdTopics[k].title,
                    id: currentSecondaryTopic.thirdTopics[k].id,
                    fourthTopics: []
                  };
                  ThirdTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].id, {
                    include: [{
                      model: FourthTopic,
                      as: 'fourthTopics'
                    }]
                  })
                  .then((currentThirdTopic) => {
                    isFinished = isFinished - 1 + currentThirdTopic.fourthTopics.length;
                    if (k === currentSecondaryTopic.thirdTopics.length - 1) {
                      isLastThird = true;
                    };
                    if (currentThirdTopic.fourthTopics.length === 0 && isLastPrimary && isLastSecondary && isLastThird) {
                      isSearchFinished.emit('finished');
                    } else if (currentThirdTopic.fourthTopics.length !== 0) {
                      for (let l = 0; l < currentThirdTopic.fourthTopics.length; l++) {
                        if (l = 0) {
                          var isLastFourth = false;
                        }
                        this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[k] = {
                          title: currentThirdTopic.fourthTopics[l].title,
                          id: currentThirdTopic.fourthTopics[l].id,
                          fifthTopics: []
                        }
                        FourthTopic.findById(this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[l].id, {
                          include: [{
                            model: FifthTopic,
                            as: 'fifthTopics'
                          }]
                        })
                        .then((currentFourthTopics) => {
                          isFinished = isFinished - 1 + currentFourthTopics.fifthTopics.length;
                          if (l = currentThirdTopic.fourthTopics.length - 1) {
                            isLastFourth = true;
                          }
                          if (currentFourthTopic.fifthTopics.length === 0 && isLastPrimary && isLastSecondary && isLastThird && isLastFourth) {
                            isSearchFinished.emit('finished');
                          } else if (currentFourthTopic.fifthTopics.length !== 0) {
                            for (let m = 0; m < currentFourthTopic.fifthTopics.length; m++) {
                              if (m = 0) {
                                var isLastFifth = false;
                              }
                              if (m === currentFourthTopic.fifthTopics.length - 1) {
                                isLastFifth = true;
                              }
                              this.primaryTopicsShort[i].secondaryTopics[j].thirdTopics[k].fourthTopics[k].fifthTopics[m] = {
                                title: currentFourthTopic.fifthTopics[m].title,
                                id: currentFourthTopic.fifthTopics[m].id
                              };
                              if (isLastPrimary === true && isLastSecondary === true && isLastThird === true && isLastFourth === true && isLastFifth === true) {
                                isFinished = isFinished - 1;
                                isSearchFinished.emit('finished');
                              };
                            }
                          }
                        })
                        .catch((err) => {
                          callback(err);
                        });
                      }
                    }
                  })
                  .catch((err) => {
                    callback(err)
                  });
                }
              }
            })
            .catch((err) => {
              callback(err);
            })
          }
        }
      })
      .catch((err) => {
        callback(err);
      });
    }
  })
  .catch((err) => {
    callback(err);
  });
};

I eventually learned that the root of the problem I face was in the asynchronous nature of the database calls.

To prevent the final callback from getting fired before all the calls were complete, I used something called a semaphore. It's a thing used often in other languages, but not often used in JavaScript (so I hear).

You can see how it works by looking at the variable isFinished . The value starts at zero and goes up for every entry it retrieves from the database. When it finishes processing the data, the code subtracts 1 from the total value. The final callback event (in the emitter), can only go off once the isFinished value returns to zero.

This isn't the most elegant solution. As @Timshel said, it probably would have been wiser not to design this database portion not have so many different tables.

But this solution will do for now.

Thanks.