Mitigate AI Platform

Knowledge Base

Manage documents and document sources to build a knowledge base for your AI chatbot. Upload files, crawl websites, and integrate with Jira and Redmine.

Build your chatbot's knowledge base by uploading files or configuring automated document sources. Documents are processed into searchable chunks with vector embeddings, enabling the chatbot to provide accurate, context-aware responses.

Everything lives under Knowledge Base in the sidebar, which has two tabs: Document Sources — where content comes from — and Documents, a flat list of everything ingested. Every document belongs to a document source.

Documents

Documents are individual files or web pages that make up your knowledge base. Each document is split into chunks, enriched with metadata, and vectorized for semantic search.

Supported File Types

  • Documents: PDF, DOCX, XLSX, PPTX, TXT, RTF, ODT, ODS, ODP
  • Web Content: HTML, Markdown, XML
  • Data: CSV, JSON, JSONL
  • Code: Most common source file extensions
  • Media: MP3, MP4, WAV, WebM, and other audio/video formats (transcribed automatically)

Uploading Documents

Uploaded files live in a Local files document source. Create one and upload into it in a single step:

Add a Local Files Source

Go to Knowledge Base, click Add Document Source, and choose Files under Local.

Name It

Enter a Name, or keep the default Uploaded Files.

Choose Access

Select which Workspaces should have access, and optionally the Owners. Access is set on the source, not on the individual files.

Select Files

Drag files onto the drop zone, or click Select Files. Files are optional — you can create the source now and upload into it later.

Create

Click Create Local Files Source. Documents are automatically processed through the ingestion pipeline.

To add more files afterwards, open the source and click Upload Files.

Uploaded documents go through the following processing stages:

  1. Loading — File content is extracted
  2. Chunking — Content is split into searchable segments
  3. Enrichment — Title, description, and locale are generated using AI
  4. Contextual Processing — Each chunk receives surrounding context for better retrieval
  5. Vectorization — Embeddings are generated for semantic search

Managing Documents

The Documents tab lists every document in the knowledge base, whatever source it came from, with search and filtering:

  • Search by title or source URL
  • Filter by vectorization status (vectorized or pending)
  • Sort by name, date, or file size
  • Bulk delete multiple documents at once
  • Bulk rechunk — re-run chunking and embedding regeneration for multiple documents via background jobs

Each document displays its processing status, including the number of chunks created and how many have been vectorized.

Moving Documents

Documents in a Local files source can be moved to another Local files source. Only local sources are involved on either side: documents that were crawled or imported from a cloud source stay with the source that fetched them, and would be re-created there on the next crawl.

  • One document — open it and click Move in the header
  • Several at once — open the source's Documents tab, tick the documents, and click Move Selected

Either way a dialog asks for the destination document source. It lists the Local files sources you may edit, minus the one the documents are already in, each with the workspace count and owners it would hand the documents; if there are none, no Move action is offered. Moving does not re-process a document — its chunks and embeddings are kept as they are.

Because access is configured on the source, moving a document hands it the destination source's Workspaces, Shared with organization and Owners settings. Moving a file into a source with wider access widens who can retrieve it.

Workspace Access

A document's workspace access always comes from its document source. To change who can see a document, open its source from the document detail page and edit the source's Workspaces or Shared with organization setting. There is no per-document workspace assignment. To give one document different access from the rest of its source, move it to another Local files source.

Document Sources

A document source is where content comes from. Local files sources hold files you upload; the other types crawl or import from an external system automatically.

Every source can be given a Name, which is what the knowledge base shows. Leave it blank on a crawled source and it is titled by its host (for example docs.example.com) or, for connector-backed types, by the connector's name.

Add one from Knowledge BaseAdd Document Source, which groups the types into Local and Cloud.

The Source Page

Opening a source shows its name, URL and description, along with its type, schedule, workspaces and owners. Below that, three tabs hold the rest:

  • Documents — the documents that came from this source, with the same search, filtering and bulk actions as the knowledge base Documents tab, plus Move Selected on Local files sources (see Moving Documents)
  • Crawler Runs — the crawl history for this source (not shown for Local files sources, which are never crawled)
  • Settings — every configured setting of the source, read-only; use Edit to change them

Start Crawl, Start Forced Crawl, Upload Files, Edit and Delete are available from every tab, whichever ones apply to the source.

Local Files

A Local files source holds documents you upload yourself, rather than content fetched from somewhere else. Create one from Knowledge BaseAdd Document SourceFiles (see Uploading Documents), and add more files to it later with Upload Files on its page.

Unlike the other types it has no URL, no crawl schedule and no crawler runs, so it is never processed or re-crawled. What it does carry is the access configuration for the files inside it: its Workspaces, Shared with organization and Owners settings govern every document it holds. Rename it, change its access, or turn on Translate from its edit page. Documents can also be moved between local sources, which is how a file changes the access it inherits.

Deleting a source also deletes the documents in it. They are soft-deleted first and permanently removed after 7 days.

Translation is disabled by default on local sources, so uploaded documents are not translated unless you enable it.

Web Source

Crawl websites to import their content as documents.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose Web under Cloud.

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial crawl.

Web Source Settings

SettingDescription
URLStarting URL for the crawler (must be HTTP or HTTPS)
LimitMaximum number of documents to fetch (1–100,000, default: 100)
Max DepthHow deep the crawler follows links (0–10, default: 10)
Include URL GlobsURL patterns to include, semicolon-separated (e.g., https://example.com/docs/*)
Exclude URL GlobsURL patterns to exclude, semicolon-separated. Exclude rules take precedence over include rules.
JavaScript EnabledEnable headless Chrome rendering for JavaScript-heavy websites. Disabled by default for faster crawling.
Ignore SelectorsCSS selectors for elements to remove from pages (e.g., nav;footer;.sidebar)
HeaderCustom HTTP headers, semicolon-separated (see Crawling Authenticated Websites)
Proxy URLOptional proxy URL to route crawler requests through (e.g. http://user:pass@proxy.example.com:8080). Useful for sites that geo-restrict or block the platform's egress IPs. Supports both JavaScript-disabled and JavaScript-enabled crawls, including authenticated proxies.
LocaleExpected language of documents (ISO 639-1 code, e.g., en, lv)
Use Canonical URLWhen enabled, stores the page's <link rel="canonical"> URL as the document's source URL so citations point to the canonical address instead of the crawled URL. Falls back to the visited URL when no canonical link is present.
TimeoutMaximum seconds to wait for a server response per page (1–300, default: 30). Increase for slower sites.
Crawl ScheduleCron expression controlling how often the source is automatically re-crawled (e.g. 0 0 * * * for daily at midnight). Leave empty to crawl manually only.
Delete Old PagesWhen enabled, pages that no longer exist on the site are removed after a successful re-crawl.
TranslateAutomatically translate documents to the base language
DescriptionOptional description of the source

Crawling Authenticated Websites

If the website you want to crawl is behind a login or requires authentication, you can use the Header setting to pass custom HTTP headers with each request. Headers are semicolon-separated.

Option 1: Basic Authentication

If the website uses HTTP Basic Authentication, add an Authorization header with a Base64-encoded username:password value:

Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

To generate the Base64 value, encode username:password (e.g., echo -n 'username:password' | base64).

Option 2: Re-using a Session Cookie

If you can log into the website manually, you can copy the session cookie from your browser and pass it as a header:

Cookie: session_id=abc123def456

Session cookies typically expire after some time. You will need to update the header when the cookie expires.

Option 3: API Key or Bearer Token

If the website supports token-based access, use the appropriate header:

Authorization: Bearer your_access_token

or

X-Api-Key: your_api_key

This option typically requires involvement from the website developers to configure robot/service account access that does not expire.

Jira Source

Import Jira issues as documents. Each issue is converted to a structured document containing its summary, description, comments, status, and other metadata.

Note: Jira integration requires the Jira feature to be enabled.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose Jira under Cloud.

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial import.

Jira Source Settings

SettingDescription
URLJira instance URL
Chunk SizeText chunk size for splitting issue content (1–100,000)
LocaleExpected language of issues (ISO 639-1 code)
Crawl ScheduleCron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only issues updated since the last run are fetched.
TranslateAutomatically translate issues to the base language
DescriptionOptional description of the source

Google Drive Source

Import files from Google Drive into your knowledge base. Supports Google Docs, Sheets, and Slides (automatically converted to DOCX, XLSX, and PPTX), along with all other supported file types.

Note: Google Drive integration requires a Google Drive connector with OAuth authentication.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose Google Drive under Cloud.

Select a Connector

Select the Google Drive connector to use for authentication.

Pick Files

Click Browse Google Drive to open the file picker. Select individual files or entire folders. When a folder is selected, all files within it (including subfolders) are ingested.

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial import.

Google Drive Source Settings

SettingDescription
ConnectorGoogle Drive connector for OAuth authentication
Selected FilesFiles and folders to import (selected via the file picker)
Chunk SizeText chunk size for splitting document content (1–100,000)
LocaleExpected language of documents (ISO 639-1 code)
Crawl ScheduleCron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed.
TranslateAutomatically translate documents to the base language
DescriptionOptional description of the source

Google native formats (Docs, Sheets, Slides) are automatically exported to Office formats for processing. Files that have not changed since the last crawl are skipped.

SharePoint Source

Import files from Microsoft SharePoint into your knowledge base. Browse SharePoint sites, drives, and folders to select files for ingestion.

Note: SharePoint integration requires a Microsoft SharePoint connector with OAuth authentication.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose SharePoint under Cloud.

Select a Connector

Select the SharePoint connector to use for authentication.

Pick Files

Select a SharePoint site, then click Browse to open the file picker. Select individual files or entire folders. Folders are traversed recursively during ingestion.

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial import.

SharePoint Source Settings

SettingDescription
ConnectorSharePoint connector for OAuth authentication
Selected FilesFiles and folders to import (selected via the file picker)
Chunk SizeText chunk size for splitting document content (1–100,000)
LocaleExpected language of documents (ISO 639-1 code)
Crawl ScheduleCron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed.
TranslateAutomatically translate documents to the base language
DescriptionOptional description of the source

Dropbox Source

Import files from Dropbox into your knowledge base. Browse folders to select files and folders for ingestion.

Note: Dropbox integration requires a Dropbox connector with OAuth authentication.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose Dropbox under Cloud.

Select a Connector

Select the Dropbox connector to use for authentication.

Pick Files

Click Browse to open the file picker. Navigate your Dropbox folders and select individual files or entire folders. Folders are traversed recursively during ingestion.

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial import.

Dropbox Source Settings

SettingDescription
ConnectorDropbox connector for OAuth authentication
Selected FilesFiles and folders to import (selected via the file picker)
Chunk SizeText chunk size for splitting document content (1–100,000)
LocaleExpected language of documents (ISO 639-1 code)
Crawl ScheduleCron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed.
TranslateAutomatically translate documents to the base language
DescriptionOptional description of the source

Only the connected user's own Dropbox files are imported. Files without downloadable content, such as Dropbox Paper documents, are listed as unsupported rather than imported.

Redmine Source

Import Redmine issues into your knowledge base. Each issue becomes a structured document containing its subject, description, metadata, journal notes, subtasks, and related issues; its attachments are imported as separate documents.

Note: Redmine integration requires a Redmine connector with OAuth authentication.

Add the Source

Go to Knowledge Base, click Add Document Source, and choose Redmine under Cloud.

Select a Connector

Select the Redmine connector to use for authentication.

Paste a Filtered Issues URL

In Redmine, open the Issues page and apply the filters you want, then copy the page URL into the Issues URL field (see below).

Configure Settings

Configure the source settings (see table below).

Save and Process

Click Save, then click Process to start the initial import.

Redmine Source Settings

SettingDescription
ConnectorRedmine connector for OAuth authentication
Issues URLA filtered Redmine Issues page URL that defines which issues are imported (see below)
Chunk SizeText chunk size for splitting issue content (1–100,000)
LocaleExpected language of issues (ISO 639-1 code)
Crawl ScheduleCron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only issues changed since the last run are fetched.
TranslateAutomatically translate issues to the base language
DescriptionOptional description of the source

Choosing which issues to import. In Redmine, open the Issues page and apply the filters you want (status, project, tracker, assignee, …), then paste that page's URL into Issues URL. The same filters are applied on every import. Without a status filter, only open issues are imported (Redmine's default). Filters relative to the current user (e.g. "assigned to me") resolve to the source's owner, whose connection authenticates the crawl.

Processing Sources

After creating a source, click Start Crawl to start the crawl. You can monitor progress from the source's Crawler Runs tab, which shows:

  • Status — Pending, running, completed, or failed
  • Progress — Percentage of completion
  • Duration — How long the crawl took
  • Error Message — Details if the crawl failed

Scheduled Crawling

Set a Crawl Schedule (a cron expression) on a source to have the system re-crawl it automatically on that cadence — for example, 0 0 * * * for daily at midnight or 0 */6 * * * every six hours. Leave the schedule empty to crawl the source manually only. For Jira, Redmine, Google Drive, and SharePoint sources, only items changed since the last run are fetched, making subsequent crawls faster. Web sources can additionally enable Delete Old Pages to prune documents whose pages no longer exist on the site after a successful re-crawl.

Workspace Access

Document sources can be assigned to specific workspaces. Every document belonging to a source is available in the same workspaces.

Translation

When the translation feature is enabled, documents can be automatically translated to a configured base language. This is useful when your knowledge base contains documents in multiple languages but you want consistent retrieval.

  • Enable Translate on the document source (off by default for Local files folders)
  • Set the base translation language in AdminOrganization Settings
  • Translated content is used alongside original content for search
  • Documents already in the target language are left unchanged

On this page