Knowledge Base
Manage documents and document sources to build a knowledge base for your AI chatbot. Upload files, crawl websites, and integrate with Jira and Redmine.
Build your chatbot's knowledge base by uploading files or configuring automated document sources. Documents are processed into searchable chunks with vector embeddings, enabling the chatbot to provide accurate, context-aware responses.
Everything lives under Knowledge Base in the sidebar, which has two tabs: Document Sources — where content comes from — and Documents, a flat list of everything ingested. Every document belongs to a document source.
Documents
Documents are individual files or web pages that make up your knowledge base. Each document is split into chunks, enriched with metadata, and vectorized for semantic search.
Supported File Types
- Documents: PDF, DOCX, XLSX, PPTX, TXT, RTF, ODT, ODS, ODP
- Web Content: HTML, Markdown, XML
- Data: CSV, JSON, JSONL
- Code: Most common source file extensions
- Media: MP3, MP4, WAV, WebM, and other audio/video formats (transcribed automatically)
Uploading Documents
Uploaded files live in a Local files document source. Create one and upload into it in a single step:
Add a Local Files Source
Go to Knowledge Base, click Add Document Source, and choose Files under Local.
Name It
Enter a Name, or keep the default Uploaded Files.
Choose Access
Select which Workspaces should have access, and optionally the Owners. Access is set on the source, not on the individual files.
Select Files
Drag files onto the drop zone, or click Select Files. Files are optional — you can create the source now and upload into it later.
Create
Click Create Local Files Source. Documents are automatically processed through the ingestion pipeline.
To add more files afterwards, open the source and click Upload Files.
Uploaded documents go through the following processing stages:
- Loading — File content is extracted
- Chunking — Content is split into searchable segments
- Enrichment — Title, description, and locale are generated using AI
- Contextual Processing — Each chunk receives surrounding context for better retrieval
- Vectorization — Embeddings are generated for semantic search
Managing Documents
The Documents tab lists every document in the knowledge base, whatever source it came from, with search and filtering:
- Search by title or source URL
- Filter by vectorization status (vectorized or pending)
- Sort by name, date, or file size
- Bulk delete multiple documents at once
- Bulk rechunk — re-run chunking and embedding regeneration for multiple documents via background jobs
Each document displays its processing status, including the number of chunks created and how many have been vectorized.
Moving Documents
Documents in a Local files source can be moved to another Local files source. Only local sources are involved on either side: documents that were crawled or imported from a cloud source stay with the source that fetched them, and would be re-created there on the next crawl.
- One document — open it and click Move in the header
- Several at once — open the source's Documents tab, tick the documents, and click Move Selected
Either way a dialog asks for the destination document source. It lists the Local files sources you may edit, minus the one the documents are already in, each with the workspace count and owners it would hand the documents; if there are none, no Move action is offered. Moving does not re-process a document — its chunks and embeddings are kept as they are.
Because access is configured on the source, moving a document hands it the destination source's Workspaces, Shared with organization and Owners settings. Moving a file into a source with wider access widens who can retrieve it.
Workspace Access
A document's workspace access always comes from its document source. To change who can see a document, open its source from the document detail page and edit the source's Workspaces or Shared with organization setting. There is no per-document workspace assignment. To give one document different access from the rest of its source, move it to another Local files source.
Document Sources
A document source is where content comes from. Local files sources hold files you upload; the other types crawl or import from an external system automatically.
Every source can be given a Name, which is what the knowledge base shows. Leave it blank on a crawled source and it is titled by its host (for example docs.example.com) or, for connector-backed types, by the connector's name.
Add one from Knowledge Base → Add Document Source, which groups the types into Local and Cloud.
The Source Page
Opening a source shows its name, URL and description, along with its type, schedule, workspaces and owners. Below that, three tabs hold the rest:
- Documents — the documents that came from this source, with the same search, filtering and bulk actions as the knowledge base Documents tab, plus Move Selected on Local files sources (see Moving Documents)
- Crawler Runs — the crawl history for this source (not shown for Local files sources, which are never crawled)
- Settings — every configured setting of the source, read-only; use Edit to change them
Start Crawl, Start Forced Crawl, Upload Files, Edit and Delete are available from every tab, whichever ones apply to the source.
Local Files
A Local files source holds documents you upload yourself, rather than content fetched from somewhere else. Create one from Knowledge Base → Add Document Source → Files (see Uploading Documents), and add more files to it later with Upload Files on its page.
Unlike the other types it has no URL, no crawl schedule and no crawler runs, so it is never processed or re-crawled. What it does carry is the access configuration for the files inside it: its Workspaces, Shared with organization and Owners settings govern every document it holds. Rename it, change its access, or turn on Translate from its edit page. Documents can also be moved between local sources, which is how a file changes the access it inherits.
Deleting a source also deletes the documents in it. They are soft-deleted first and permanently removed after 7 days.
Translation is disabled by default on local sources, so uploaded documents are not translated unless you enable it.
Web Source
Crawl websites to import their content as documents.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose Web under Cloud.
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial crawl.
Web Source Settings
| Setting | Description |
|---|---|
| URL | Starting URL for the crawler (must be HTTP or HTTPS) |
| Limit | Maximum number of documents to fetch (1–100,000, default: 100) |
| Max Depth | How deep the crawler follows links (0–10, default: 10) |
| Include URL Globs | URL patterns to include, semicolon-separated (e.g., https://example.com/docs/*) |
| Exclude URL Globs | URL patterns to exclude, semicolon-separated. Exclude rules take precedence over include rules. |
| JavaScript Enabled | Enable headless Chrome rendering for JavaScript-heavy websites. Disabled by default for faster crawling. |
| Ignore Selectors | CSS selectors for elements to remove from pages (e.g., nav;footer;.sidebar) |
| Header | Custom HTTP headers, semicolon-separated (see Crawling Authenticated Websites) |
| Proxy URL | Optional proxy URL to route crawler requests through (e.g. http://user:pass@proxy.example.com:8080). Useful for sites that geo-restrict or block the platform's egress IPs. Supports both JavaScript-disabled and JavaScript-enabled crawls, including authenticated proxies. |
| Locale | Expected language of documents (ISO 639-1 code, e.g., en, lv) |
| Use Canonical URL | When enabled, stores the page's <link rel="canonical"> URL as the document's source URL so citations point to the canonical address instead of the crawled URL. Falls back to the visited URL when no canonical link is present. |
| Timeout | Maximum seconds to wait for a server response per page (1–300, default: 30). Increase for slower sites. |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-crawled (e.g. 0 0 * * * for daily at midnight). Leave empty to crawl manually only. |
| Delete Old Pages | When enabled, pages that no longer exist on the site are removed after a successful re-crawl. |
| Translate | Automatically translate documents to the base language |
| Description | Optional description of the source |
Crawling Authenticated Websites
If the website you want to crawl is behind a login or requires authentication, you can use the Header setting to pass custom HTTP headers with each request. Headers are semicolon-separated.
Option 1: Basic Authentication
If the website uses HTTP Basic Authentication, add an Authorization header with a Base64-encoded username:password value:
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=To generate the Base64 value, encode username:password (e.g., echo -n 'username:password' | base64).
Option 2: Re-using a Session Cookie
If you can log into the website manually, you can copy the session cookie from your browser and pass it as a header:
Cookie: session_id=abc123def456Session cookies typically expire after some time. You will need to update the header when the cookie expires.
Option 3: API Key or Bearer Token
If the website supports token-based access, use the appropriate header:
Authorization: Bearer your_access_tokenor
X-Api-Key: your_api_keyThis option typically requires involvement from the website developers to configure robot/service account access that does not expire.
Jira Source
Import Jira issues as documents. Each issue is converted to a structured document containing its summary, description, comments, status, and other metadata.
Note: Jira integration requires the Jira feature to be enabled.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose Jira under Cloud.
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial import.
Jira Source Settings
| Setting | Description |
|---|---|
| URL | Jira instance URL |
| Chunk Size | Text chunk size for splitting issue content (1–100,000) |
| Locale | Expected language of issues (ISO 639-1 code) |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only issues updated since the last run are fetched. |
| Translate | Automatically translate issues to the base language |
| Description | Optional description of the source |
Google Drive Source
Import files from Google Drive into your knowledge base. Supports Google Docs, Sheets, and Slides (automatically converted to DOCX, XLSX, and PPTX), along with all other supported file types.
Note: Google Drive integration requires a Google Drive connector with OAuth authentication.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose Google Drive under Cloud.
Select a Connector
Select the Google Drive connector to use for authentication.
Pick Files
Click Browse Google Drive to open the file picker. Select individual files or entire folders. When a folder is selected, all files within it (including subfolders) are ingested.
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial import.
Google Drive Source Settings
| Setting | Description |
|---|---|
| Connector | Google Drive connector for OAuth authentication |
| Selected Files | Files and folders to import (selected via the file picker) |
| Chunk Size | Text chunk size for splitting document content (1–100,000) |
| Locale | Expected language of documents (ISO 639-1 code) |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed. |
| Translate | Automatically translate documents to the base language |
| Description | Optional description of the source |
Google native formats (Docs, Sheets, Slides) are automatically exported to Office formats for processing. Files that have not changed since the last crawl are skipped.
SharePoint Source
Import files from Microsoft SharePoint into your knowledge base. Browse SharePoint sites, drives, and folders to select files for ingestion.
Note: SharePoint integration requires a Microsoft SharePoint connector with OAuth authentication.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose SharePoint under Cloud.
Select a Connector
Select the SharePoint connector to use for authentication.
Pick Files
Select a SharePoint site, then click Browse to open the file picker. Select individual files or entire folders. Folders are traversed recursively during ingestion.
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial import.
SharePoint Source Settings
| Setting | Description |
|---|---|
| Connector | SharePoint connector for OAuth authentication |
| Selected Files | Files and folders to import (selected via the file picker) |
| Chunk Size | Text chunk size for splitting document content (1–100,000) |
| Locale | Expected language of documents (ISO 639-1 code) |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed. |
| Translate | Automatically translate documents to the base language |
| Description | Optional description of the source |
Dropbox Source
Import files from Dropbox into your knowledge base. Browse folders to select files and folders for ingestion.
Note: Dropbox integration requires a Dropbox connector with OAuth authentication.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose Dropbox under Cloud.
Select a Connector
Select the Dropbox connector to use for authentication.
Pick Files
Click Browse to open the file picker. Navigate your Dropbox folders and select individual files or entire folders. Folders are traversed recursively during ingestion.
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial import.
Dropbox Source Settings
| Setting | Description |
|---|---|
| Connector | Dropbox connector for OAuth authentication |
| Selected Files | Files and folders to import (selected via the file picker) |
| Chunk Size | Text chunk size for splitting document content (1–100,000) |
| Locale | Expected language of documents (ISO 639-1 code) |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only changed files are reprocessed. |
| Translate | Automatically translate documents to the base language |
| Description | Optional description of the source |
Only the connected user's own Dropbox files are imported. Files without downloadable content, such as Dropbox Paper documents, are listed as unsupported rather than imported.
Redmine Source
Import Redmine issues into your knowledge base. Each issue becomes a structured document containing its subject, description, metadata, journal notes, subtasks, and related issues; its attachments are imported as separate documents.
Note: Redmine integration requires a Redmine connector with OAuth authentication.
Add the Source
Go to Knowledge Base, click Add Document Source, and choose Redmine under Cloud.
Select a Connector
Select the Redmine connector to use for authentication.
Paste a Filtered Issues URL
In Redmine, open the Issues page and apply the filters you want, then copy the page URL into the Issues URL field (see below).
Configure Settings
Configure the source settings (see table below).
Save and Process
Click Save, then click Process to start the initial import.
Redmine Source Settings
| Setting | Description |
|---|---|
| Connector | Redmine connector for OAuth authentication |
| Issues URL | A filtered Redmine Issues page URL that defines which issues are imported (see below) |
| Chunk Size | Text chunk size for splitting issue content (1–100,000) |
| Locale | Expected language of issues (ISO 639-1 code) |
| Crawl Schedule | Cron expression controlling how often the source is automatically re-imported (e.g. 0 0 * * * for daily at midnight). Leave empty to import manually only. Only issues changed since the last run are fetched. |
| Translate | Automatically translate issues to the base language |
| Description | Optional description of the source |
Choosing which issues to import. In Redmine, open the Issues page and apply the filters you want (status, project, tracker, assignee, …), then paste that page's URL into Issues URL. The same filters are applied on every import. Without a status filter, only open issues are imported (Redmine's default). Filters relative to the current user (e.g. "assigned to me") resolve to the source's owner, whose connection authenticates the crawl.
Processing Sources
After creating a source, click Start Crawl to start the crawl. You can monitor progress from the source's Crawler Runs tab, which shows:
- Status — Pending, running, completed, or failed
- Progress — Percentage of completion
- Duration — How long the crawl took
- Error Message — Details if the crawl failed
Scheduled Crawling
Set a Crawl Schedule (a cron expression) on a source to have the system re-crawl it automatically on that cadence — for example, 0 0 * * * for daily at midnight or 0 */6 * * * every six hours. Leave the schedule empty to crawl the source manually only. For Jira, Redmine, Google Drive, and SharePoint sources, only items changed since the last run are fetched, making subsequent crawls faster. Web sources can additionally enable Delete Old Pages to prune documents whose pages no longer exist on the site after a successful re-crawl.
Workspace Access
Document sources can be assigned to specific workspaces. Every document belonging to a source is available in the same workspaces.
Translation
When the translation feature is enabled, documents can be automatically translated to a configured base language. This is useful when your knowledge base contains documents in multiple languages but you want consistent retrieval.
- Enable Translate on the document source (off by default for Local files folders)
- Set the base translation language in Admin → Organization Settings
- Translated content is used alongside original content for search
- Documents already in the target language are left unchanged