Add Sitemaps
Learn how to add sitemaps as a source of data for AI-Bot training
Did you know that you can generate a Sitemap (a list of all the URLs in your website) and add them all at once as a source of data to train your Bot?
Uploading a sitemap is especially useful if you have an existing help center or a knowledge base site with tons of pages and articles. However, instead of copying every single URL from your website, Botsonic can generate a list of URLs and you can choose and select which ones to add for training your Bot.
Step 1 - View the Bot Settings
Click [Home] from the left hand side menu to access your [Dashboard]. Click to open a bot from the the "Your Bots" section. If you haven't created a Bot yet, read this Quick Start Guide
![](https://files.readme.io/621cb5a-Your_Bots.png)
Step 2 - Add a Sitemap
Botsonic can access and reference content from most websites, with a few exceptions:
- webpages that are not published publicly
- webpages that restrict crawling
- content that is behind paywalls or requires user authentication
- webpages that primarily consists of image, audio, or video content without any accompanying texts
- Click [Sources], then click [Links].
- You can either crawl webpages by providing a top-level domain or submit an XML sitemap.
- Crawl Webpages - provide a top-level domain, eg. https://writesonic.com
- Submit an XML sitemap - provide an XML sitemap URL, eg. https://writesonic.com/sitemap.xml
- Click [Continue] or [Load sitemap]
- Review the list of links generated, uncheck the ones you don't want to include, then hit [Start training].
![](https://files.readme.io/7804616-Sitemap_2.png)
Step 3 - View Link Status
Click [Sources], then click [Links] to view if the link status is waiting, processing, and trained
![](https://files.readme.io/746f69d-Links_-_status.png)
Step 4 - Configure the Link Settings
Pause File Usage
Click the [3-dot] icon, then click [View Details]. Toggle ON to temporarily pause file usage as a data Source for your Bot.
![](https://files.readme.io/e2c44b0-Links_-_paused_1.png)
Enable or Disable File Visibility as Reference
Toggle ON to set the file to be shown as a reference on your chatbot's responses.
![](https://files.readme.io/d92633b-Links_-_visibility.png)
![](https://files.readme.io/bcf50dd-image.png)
Edit the link
- Click the [3-dot] icon, then click [View details] to open the link settings
- Click [edit], then edit the link
- Click [Save]
![](https://files.readme.io/0b70daa-Links_-_rename.png)
Delete the uploaded link
Click [View details] to open the file settings, then click [Delete]
Note: Bulk deletion of links is not possible
![](https://files.readme.io/564dd0b-Links_-_delete.png)
View Extracted Data
To quickly view the extracted data to verify the contents of the link you've uploaded, click the [Extracted Data] tab from the file settings panel.
![](https://files.readme.io/8a0bb07-Links_-_extracted_data.png)
Autosync Data from Links (Webpages, Google Docs, Google Sheets)
Did you know that you can automatically synchronize the data from links (webpages, Google Docs, Google Sheets) that you've added as a training source for your Bot? In other words, once you've enabled AutoSync, Botsonic will use the latest data from the website pages that you've added in your Sources section. To learn more, read this step by step guide on how to Auto Sync Data from Webpages
Set Autosync or Sync Now
You can automatically synchronize the data from webpage links that you've added as a training source for your Bot. Once you've enabled AutoSync, Botsonic will use the latest data from the website pages that you've added in your Sources section. To learn more, read this step by step guide on how to Auto Sync Data from Webpages
![](https://files.readme.io/304c134-Links_-_Sync_Now.png)
Updated about 1 month ago