Add Sitemaps

Learn how to add sitemaps as a source of data for AI-Bot training

Did you know that you can generate a Sitemap (a list of all the URLs in your website) and add them all at once as a source of data to train your Bot?

Uploading a sitemap is especially useful if you have an existing help center or a knowledge base site with tons of pages and articles. However, instead of copying every single URL from your website, Botsonic can generate a list of URLs and you can choose and select which ones to add for training your Bot.

Step 1 - View the Bot Settings

Go to your "Your Bots" Page and click on a bot. If you haven't created a Bot yet, read this Quick Start Guide

Step 2 - Add a Sitemap


Botsonic can access and reference content from most websites, with a few exceptions:

  • webpages that are not published publicly
  • webpages that restrict crawling
  • content that is behind paywalls or requires user authentication
  • webpages that primarily consists of image, audio, or video content without any accompanying texts

Add a Sitemap with a Single URL

  1. Click the [Sources] tab, then click [Sitemaps]
  2. Enter a website URL, then click [Get Sitemap]

  1. Review the list of URLs generated on the pop-up, uncheck the ones you don't want to submit, then hit [Submit].

Bulk Upload a Sitemap with a CSV File

  1. Click the [Sources] tab, then click [Sitemaps]
  2. Click the [Bulk Upload] tab, then download the CSV template and follow the format to add all the URL's in the sheet (see snapshot of template below)

  1. Click to upload or drag the CSV file to the upload box.
  2. Click [Upload and Train]

Step 3 - View Sitemap Status

  1. On the Sitemaps table, click the website URL or file to pop-open the list of URLs you submitted
  2. Review ths status of each URL you submitted on the pop-up.

Step 4 - Configure the Sitemap Settings

  1. On the Sitemaps table, click the website URL or file to pop-open the list of URLs in the sitemap
  2. Click the [3-dot] icon, then click [View Details] to do the following actions:
    • Pause file usage
    • Set reference visibility

Pause File Usage

Toggle ON to temporarily pause file usage as a data Source for your Bot.

Enable or Disable File Visibility as Reference

Toggle ON to set the file to be shown as a reference on your chatbot's responses.

Alternatively, you can enable/disable this from sitemaps table

Edit the sitemap name

  1. Click the [3-dot] icon, then click [View details] to open the sitemap settings
  2. Click [edit], then edit the link
  3. Click [Save]

Delete the uploaded sitemap link or file

  • Click the [3-dot] icon, then click [Delete]
  • You can also click [View details] to open the sitemap settings, then click [Delete]

View Extracted Data

To quickly view the extracted data to verify the contents of the sitemap link or file you've uploaded, click the [Extracted Data] tab from the file settings panel.


Autosync Data from Webpages

Did you know that you can automatically synchronize the data from webpage links that you've added as a training source for your Bot? In other words, once you've enabled AutoSync, Botsonic will use the latest data from the website pages that you've added in your Sources section. To learn more, read this step by step guide on how to Auto Sync Data from Webpages