Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork138
Sitemap generator for next.js. Generate sitemap(s) and robots.txt for all static/pre-rendered/dynamic/server-side pages.
License
iamvishnusankar/next-sitemap
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
- Getting started
- Index sitemaps
- Splitting large sitemap into multiple files
- Configuration Options
- Custom transformation function
- Full configuration example
- Generating dynamic/server-side sitemaps
- Typescript JSDoc
yarn add next-sitemap
next-sitemap requires a basic config file (next-sitemap.config.js) under your project root
✅
next-sitemapwill load environment variables from.envfiles by default.
/**@type {import('next-sitemap').IConfig} */module.exports={siteUrl:process.env.SITE_URL||'https://example.com',generateRobotsTxt:true,// (optional)// ...other options}
Add next-sitemap as your postbuild script
{"build":"next build","postbuild":"next-sitemap"}You can also use a custom config file instead ofnext-sitemap.config.js. Just pass--config <your-config-file>.js to build command (Example:custom-config-file)
{"build":"next build","postbuild":"next-sitemap --config awesome.config.js"}When using pnpm you need to create a.npmrc file in the root of your project if you want to use a postbuild step:
//.npmrcenable-pre-post-scripts=true📣 Fromnext-sitemap v2.x onwards,sitemap.xml will beIndex Sitemap. It will contain urls of all other generated sitemap endpoints.
Index sitemap generation can be turned off by settinggenerateIndexSitemap: false in next-sitemap config file. (This is useful for small/hobby sites which does not require an index sitemap) (Example:no-index-sitemaps)
Define thesitemapSize property innext-sitemap.config.js to split large sitemap into multiple files.
/**@type {import('next-sitemap').IConfig} */module.exports={siteUrl:'https://example.com',generateRobotsTxt:true,sitemapSize:7000,}
Above is the minimal configuration to split a large sitemap. When the number of URLs in a sitemap is more than 7000,next-sitemap will create sitemap (e.g. sitemap-0.xml, sitemap-1.xml) and index (e.g. sitemap.xml) files.
| property | description | type |
|---|---|---|
| siteUrl | Base url of your website | string |
| output (optional) | Next.js output modes.Check documentation. | standalone,export |
| changefreq (optional) | Change frequency. Defaultdaily | string |
| priority (optional) | Priority. Default0.7 | number |
| sitemapBaseFileName (optional) | The name of the generated sitemap file before the file extension. Default"sitemap" | string |
| alternateRefs (optional) | Denote multi-language support by unique URL. Default[] | AlternateRef[] |
| sitemapSize(optional) | Split large sitemap into multiple files by specifying sitemap size. Default5000 | number |
| autoLastmod (optional) | Add<lastmod/> property. Defaulttrue | true |
| exclude (optional) | Array ofrelative paths (wildcard pattern supported) to exclude from listing onsitemap.xml orsitemap-*.xml. e.g.:['/page-0', '/page-*', '/private/*'].Apart from this option next-sitemap also offers a customtransform option which could be used to exclude urls that match specific patterns | string[] |
| sourceDir (optional) | next.js build directory. Default.next | string |
| outDir (optional) | All the generated files will be exported to this directory. Defaultpublic | string |
| transform (optional) | A transformation function, which runsfor eachrelative-path in the sitemap. Returningnull value from the transformation function will result in the exclusion of that specificpath from the generated sitemap list. | async function |
| additionalPaths (optional) | Async function that returns a list of additional paths to be added to the generated sitemap list. | async function |
| generateIndexSitemap | Generate index sitemaps. Defaulttrue | boolean |
| generateRobotsTxt (optional) | Generate arobots.txt file and list the generated sitemaps. Defaultfalse | boolean |
| robotsTxtOptions.transformRobotsTxt (optional) | Custom robots.txt transformer function. (Example:custom-robots-txt-transformer) Default: async(config, robotsTxt)=> robotsTxt | async function |
| robotsTxtOptions.policies (optional) | Policies for generatingrobots.txt.Default: [{ userAgent: '*', allow: '/' }] | IRobotPolicy[] |
| robotsTxtOptions.additionalSitemaps (optional) | Options to add additional sitemaps torobots.txt host entry | string[] |
| robotsTxtOptions.includeNonIndexSitemaps (optional) | From v2.4x onwards, generatedrobots.txt will only contain url ofindex sitemap and custom provided endpoints fromrobotsTxtOptions.additionalSitemaps.This is to prevent duplicate url submission (once through index-sitemap -> sitemap-url and once through robots.txt -> HOST) Set this option true to add all generated sitemap endpoints torobots.txtDefault false (Recommended) | boolean |
Custom transformation provides an extension method to add, remove or excludepath orproperties from a url-set. Transform function runsfor eachrelative path in the sitemap. And use thekey:value object to add properties in the XML.
Returningnull value from the transformation function will result in the exclusion of that specificrelative-path from the generated sitemap list.
/**@type {import('next-sitemap').IConfig} */module.exports={transform:async(config,path)=>{// custom function to ignore the pathif(customIgnoreFunction(path)){returnnull}// only create changefreq along with path// returning partial properties will result in generation of XML field with only returned values.if(customLimitedField(path)){// This returns `path` & `changefreq`. Hence it will result in the generation of XML field with `path` and `changefreq` properties only.return{loc:path,// => this will be exported as http(s)://<config.siteUrl>/<path>changefreq:'weekly',}}// Use default transformation for all other casesreturn{loc:path,// => this will be exported as http(s)://<config.siteUrl>/<path>changefreq:config.changefreq,priority:config.priority,lastmod:config.autoLastmod ?newDate().toISOString() :undefined,alternateRefs:config.alternateRefs??[],}},}
additionalPaths this function can be useful if you have a large list of pages, but you don't want to render them all and usefallback: true. Result of executing this function will be added to the general list of paths and processed withsitemapSize. You are free to add dynamic paths, but unlikeadditionalSitemap, you do not need to split the list of paths into different files in case there are a lot of paths for one file.
If your function returns a path that already exists, then it will simply be updated, duplication will not happen.
/**@type {import('next-sitemap').IConfig} */module.exports={additionalPaths:async(config)=>{constresult=[]// required value onlyresult.push({loc:'/additional-page-1'})// all possible valuesresult.push({loc:'/additional-page-2',changefreq:'yearly',priority:0.7,lastmod:newDate().toISOString(),// acts only on '/additional-page-2'alternateRefs:[{href:'https://es.example.com',hreflang:'es',},{href:'https://fr.example.com',hreflang:'fr',},],})// using transformation from the current configurationresult.push(awaitconfig.transform(config,'/additional-page-3'))returnresult},}
Url set can contain additional sitemaps defined by google. These areGoogle News sitemap,image sitemap orvideo sitemap.You can add the values for these sitemaps by updating entry intransform function or adding it withadditionalPaths. You have to return a sitemap entry in both cases, so it's the best place for updatingthe output. This example will add an image and news tag to each entry but IRL you would of course use it withsome condition or withinadditionalPaths result.
/**@type {import('next-sitemap').IConfig} */constconfig={transform:async(config,path)=>{return{loc:path,// => this will be exported as http(s)://<config.siteUrl>/<path>changefreq:config.changefreq,priority:config.priority,lastmod:config.autoLastmod ?newDate().toISOString() :undefined,images:[{loc:'https://example.com/image.jpg'}],news:{title:'Article 1',publicationName:'Google Scholar',publicationLanguage:'en',date:newDate(),},}},}exportdefaultconfig
Here's an examplenext-sitemap.config.js configuration with all options
/**@type {import('next-sitemap').IConfig} */module.exports={siteUrl:'https://example.com',changefreq:'daily',priority:0.7,sitemapSize:5000,generateRobotsTxt:true,exclude:['/protected-page','/awesome/secret-page'],alternateRefs:[{href:'https://es.example.com',hreflang:'es',},{href:'https://fr.example.com',hreflang:'fr',},],// Default transformation functiontransform:async(config,path)=>{return{loc:path,// => this will be exported as http(s)://<config.siteUrl>/<path>changefreq:config.changefreq,priority:config.priority,lastmod:config.autoLastmod ?newDate().toISOString() :undefined,alternateRefs:config.alternateRefs??[],}},additionalPaths:async(config)=>[awaitconfig.transform(config,'/additional-page'),],robotsTxtOptions:{policies:[{userAgent:'*',allow:'/',},{userAgent:'test-bot',allow:['/path','/path-2'],},{userAgent:'black-listed-bot',disallow:['/sub-path-1','/path-2'],},],additionalSitemaps:['https://example.com/my-custom-sitemap-1.xml','https://example.com/my-custom-sitemap-2.xml','https://example.com/my-custom-sitemap-3.xml',],},}
Above configuration will generate sitemaps based on your project and arobots.txt like this.
# *User-agent:*Allow: /# test-botUser-agent: test-botAllow: /pathAllow: /path-2# black-listed-botUser-agent: black-listed-botDisallow: /sub-path-1Disallow: /path-2# HostHost: https://example.com# SitemapsSitemap: https://example.com/sitemap.xml # Index sitemapSitemap: https://example.com/my-custom-sitemap-1.xmlSitemap: https://example.com/my-custom-sitemap-2.xmlSitemap: https://example.com/my-custom-sitemap-3.xml
next-sitemap now provides two APIs to generate server side sitemaps. This will help to dynamically generateindex-sitemap(s) andsitemap(s) by sourcing data from CMS or custom source.
getServerSideSitemapIndex: Generates index sitemaps based on urls provided and returnsapplication/xmlresponse. Supports next13+ route.{ts,js} file.- To continue using inside pages directory, import
getServerSideSitemapIndexLegacyinstead.
- To continue using inside pages directory, import
getServerSideSitemap: Generates sitemap based on field entires and returnsapplication/xmlresponse. Supports next13+ route.{ts,js} file.- To continue using inside pages directory, import
getServerSideSitemapLegacyinstead.
- To continue using inside pages directory, import
Here's a sample script to generate index-sitemap on server side.
1. Index sitemap (app directory)
Createapp/server-sitemap-index.xml/route.ts file.
// app/server-sitemap-index.xml/route.tsimport{getServerSideSitemapIndex}from'next-sitemap'exportasyncfunctionGET(request:Request){// Method to source urls from cms// const urls = await fetch('https//example.com/api')returngetServerSideSitemapIndex(['https://example.com/path-1.xml','https://example.com/path-2.xml',])}
2. Index sitemap (pages directory) (legacy)
Createpages/server-sitemap-index.xml/index.tsx file.
// pages/server-sitemap-index.xml/index.tsximport{getServerSideSitemapIndexLegacy}from'next-sitemap'import{GetServerSideProps}from'next'exportconstgetServerSideProps:GetServerSideProps=async(ctx)=>{// Method to source urls from cms// const urls = await fetch('https//example.com/api')returngetServerSideSitemapIndexLegacy(ctx,['https://example.com/path-1.xml','https://example.com/path-2.xml',])}// Default export to prevent next.js errorsexportdefaultfunctionSitemapIndex(){}
Now,next.js is serving the dynamic index-sitemap fromhttp://localhost:3000/server-sitemap-index.xml.
List the dynamic sitemap page inrobotsTxtOptions.additionalSitemaps and exclude this path from static sitemap list.
// next-sitemap.config.js/**@type {import('next-sitemap').IConfig} */module.exports={siteUrl:'https://example.com',generateRobotsTxt:true,exclude:['/server-sitemap-index.xml'],// <= exclude hererobotsTxtOptions:{additionalSitemaps:['https://example.com/server-sitemap-index.xml',// <==== Add here],},}
In this way,next-sitemap will manage the sitemaps for all your static pages and your dynamicindex-sitemap will be listed on robots.txt.
Here's a sample script to generate sitemaps on server side.
1. Sitemaps (app directory)
Createapp/server-sitemap.xml/route.ts file.
// app/server-sitemap.xml/route.tsimport{getServerSideSitemap}from'next-sitemap'exportasyncfunctionGET(request:Request){// Method to source urls from cms// const urls = await fetch('https//example.com/api')returngetServerSideSitemap([{loc:'https://example.com',lastmod:newDate().toISOString(),// changefreq// priority},{loc:'https://example.com/dynamic-path-2',lastmod:newDate().toISOString(),// changefreq// priority},])}
2. Sitemaps (pages directory) (legacy)
Createpages/server-sitemap.xml/index.tsx file.
// pages/server-sitemap.xml/index.tsximport{getServerSideSitemapLegacy}from'next-sitemap'import{GetServerSideProps}from'next'exportconstgetServerSideProps:GetServerSideProps=async(ctx)=>{// Method to source urls from cms// const urls = await fetch('https//example.com/api')constfields=[{loc:'https://example.com',// Absolute urllastmod:newDate().toISOString(),// changefreq// priority},{loc:'https://example.com/dynamic-path-2',// Absolute urllastmod:newDate().toISOString(),// changefreq// priority},]returngetServerSideSitemapLegacy(ctx,fields)}// Default export to prevent next.js errorsexportdefaultfunctionSitemap(){}
Now,next.js is serving the dynamic sitemap fromhttp://localhost:3000/server-sitemap.xml.
List the dynamic sitemap page inrobotsTxtOptions.additionalSitemaps and exclude this path from static sitemap list.
// next-sitemap.config.js/**@type {import('next-sitemap').IConfig} */module.exports={siteUrl:'https://example.com',generateRobotsTxt:true,exclude:['/server-sitemap.xml'],// <= exclude hererobotsTxtOptions:{additionalSitemaps:['https://example.com/server-sitemap.xml',// <==== Add here],},}
In this way,next-sitemap will manage the sitemaps for all your static pages and your dynamic sitemap will be listed on robots.txt.
Add the following line of code in yournext-sitemap.config.js for nice typescript autocomplete! 💖
/**@type {import('next-sitemap').IConfig} */module.exports={// YOUR CONFIG}
All PRs are welcome :)
About
Sitemap generator for next.js. Generate sitemap(s) and robots.txt for all static/pre-rendered/dynamic/server-side pages.
Topics
Resources
License
Code of conduct
Security policy
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Sponsor this project
Uh oh!
There was an error while loading.Please reload this page.
Packages0
Uh oh!
There was an error while loading.Please reload this page.
