Nuxt3 Robots.txt - @nuxtjs/robots 不生成 robots.txt 文件

Question

I'm building a Nuxt 3 project. I need my build to generate a robots.txt file, just like this package states it does -> https://github.com/nuxt-community/robots-module

After running "nuxt build" and/or "nuxt generate", the robots.txt does not appear in the output or public folders as I'd expect.

I'm definitely missing something and likely being an idiot here.. Does anyone know what I'm missing? Here's my code:

package.json

  "dependencies": {
    ...
    "@nuxtjs/robots": "^2.5.0", 
    }

nuxt.config.ts

 target: "static",
  runtimeConfig: {
    NUXT_STORYBLOK_PRODUCTION_KEY: process.env.NUXT_STORYBLOK_PRODUCTION_KEY,
    public: {
      CDN: process.env.CDN,
      NUXT_STORYBLOK_PREVIEW_KEY: process.env.NUXT_STORYBLOK_PREVIEW_KEY,
      NUXT_DOMAIN_NAME: process.env.NUXT_DOMAIN_NAME,
    },
  },
  modules: [
    ...
    "@nuxtjs/robots",
  ],
  robots: {
    UserAgent: "*",
    Disallow: "",
  },
}

Answer 1

I just successfully installed this module on Nuxt 3, and I think there are a couple of things to note here.

The first is that I've never managed to get the module options for any module to work in Nuxt 3 the way you have shown (top-level options, according to the module docs):

  modules: [
    ...
    "@nuxtjs/robots",
  ],
  robots: {
    UserAgent: "*",
    Disallow: "",
  }

Have you tried the other options? You can also try using the Robots Config file, or pass the options when declaring the module (from the README.md for the repo):

export default {
  modules: [
    // Simple usage
    '@nuxtjs/robots',

    // With options
    ['@nuxtjs/robots', { /* module options */ }]
  ]
}

The other thing is that I also do not see a generated robots.txt file anywhere after running the build or dev, but if I go to '/robots.txt' in the dev or build preview, I can see the output working as intended. Have you tried visiting the path?

It looks like something the server generates when the route is visited rather than a static file generated on build by default. I think you can change that in the options, but it's not something I need so I haven't dug into it.

Hopefully, that helps somewhat!

Answer 2

You are using an incompatible version of nuxt robots for your project. Ensure you are using version ^3.0.0. Any versions below this are not compatible with nuxt 3.

npm install @nuxtjs/robots@3.0.0  

Then ensure your nuxt.config.ts looks similar to below. As John Overstreet mentioned above the simple method appears flawed.

I created a config folder in the root directory of my project and placed my robots.txt file within it:

export default {
  modules: [
    ['@nuxtjs/robots', { configPath: "~/config/robots.config" }]
  ]
}

You will also need to delete the robots.txt file from your public folder

Build/Generate your project and go to '/robots.txt'

The file is generated dynamically upon request of the above route.

With mine and John's help above I hope this helps resolve your issues:)