Search & Answers
How search works in data pools, release levels, citations, and the 3-stage model
How MeinGPT searches a data pool, what shows up in an answer, and how deep the AI goes into an individual file when needed. Setup and configuration of a data source: Data Sources & RAG.
How search works
Customer knowledge bases are often large. Hundreds or thousands of gigabytes of Word, PDF, and other files are not unusual. Sending every search query through every file would be far too slow. That is why a search index is built up front. It works similar to Google, just for your internal documents.
The initial indexing can take hours to days, depending on the data volume. Word, PDF, and similar formats are binary, so the text has to be extracted first. This one-time effort pays off in fast search results afterwards.
You do not have to wait for all of it. The index is built in two stages: keyword search first, semantic search second. A file is therefore searchable as soon as its keyword index is ready — semantic search follows in the background, file by file. You can see how far each stage has got under Indexing progress.
Search characteristics:
- Keyword search: Finds exactly the terms you type — case numbers, article numbers, proper names. It is built first and is always available.
- Semantic: Documents are converted into mathematical representations (called embeddings) that capture meaning, not just individual words. Admins cannot turn it off. By default keyword and meaning search run together; the assistant may narrow a single search to one of the two. Meaning search pauses only while a cloud data source is in sleep mode.
- Sorted by relevance: Hits are ranked by content fit, not by frequency of a search term. A document that thematically matches the question can rank higher than one with the exact keyword.
- Number of results: One search returns eight passages by default, across all data sources searched and at most three from the same file. A search within exactly one file, such as one pinned in the chat, is exempt: all passages can come from that file. The assistant can ask for fewer and run further searches when needed. For cloud data sources the number cannot be set; for Outpost data sources see Search settings.
- Filename search: Besides content, you can also search specifically by filename, for example "Show me file XY".
Attention
No filtering by custom metadata fields. Data sources search file contents (semantically) and file names. You can additionally filter by built-in file properties: path, extension, MIME type, and created/modified dates. Custom metadata fields from source systems are not carried over as separate, filterable fields — at best they exist as plain text inside the document content, so they can only be found via semantic search, not filtered on reliably.
Filter in a chat: file type and date
In a chat, pick the source via + → Search a data source. Below the sources you find Filter by file type or date: two filters then appear in the message field — File type (PDF, documents, spreadsheets, presentations, text & web, images) and Date (last 7 days, 30 days, 12 months). Once you have picked a source, they appear next to it on their own.
The filters apply to every search the AI runs in that chat until you reset them. The AI can narrow further within your choice but never widen it: ask for spreadsheets while only PDFs are selected and it tells you the filter would have to change. Outpost data sources do not support these filters and are left out while one is set.
Search settings
A Cloud data source has no search settings: keyword and meaning search always both run, and how it is searched is tuned by MeinGPT. The one setting for data sources applies to the whole workspace: Permissions from the source.
For Outpost data sources you set Passages per answer under Settings at the bottom of the data source page (default 10, range 1 to 100); in the cloud the assistant chooses the number per search. To check what a particular question finds in the source, use Try it on the same page.
More, with recommendations per document type: Data pool search settings.
Indexing progress
Under the data source's name, its status appears in one word — the same one the list of data sources shows — with the reason or the time of the last update.
While a source is being indexed, the Overview shows every file as a dot that changes colour: found → keyword-searchable → meaning-searchable. Above it you see how many files are already searchable, below it which file is being read right now. As soon as the first files are keyword-searchable, the data source is usable; meaning search follows in the background. For very large sources one dot stands for several files. When nothing is moving, the page folds the same numbers into one line. What the three progress steps count and what "Stuck" means: Progress and "Stuck".
Files that are not searchable are counted separately: could not be read (the latest attempt failed) and skipped (left out on purpose, such as formats without text or system files). A few such files do not make a source "unhealthy". The Files tab groups them by reason, with one next step per group, and shows under Check how far each file got. Files whose extension is not on the list of file formats are never picked up by the sync.
If the platform cannot read the state right now, the page says so instead of showing zeros — for a cloud data source because the Cloud Vault did not answer, for an Outpost data source because the Outpost is offline or unreachable. A sync that is already running is not affected.
Citations in answers
Answers from a data source include numbered citations as clickable badges next to the text. Clicking one opens a preview with the title, location, and an excerpt of the original document - plus an option to open or download the original file. For an Outpost data source that option appears only when the folder releases the original (release level).
Attention
Citation isn't clickable? That is not expected behavior. Contact support@meingpt.com with a link to the affected chat so we can investigate.
What is available behind a citation
For a cloud data source, everyone who may use the source gets everything: the full text of the documents, the original file to open and download, and images from the documents in the chat. The assistant may load an original file into the chat or the Code Sandbox when needed. There is no level that narrows this for cloud data sources — who sees what is controlled only by access control. If an admin restricted a cloud data source to Matching passages only or Text only earlier, that restriction stays in force until an admin lifts it with Lift restriction under Settings on the data source page; it cannot be set again.
For an Outpost data source, the folder in the Outpost app decides what the platform receives — see Release level.
Back to the source system
For citations from SharePoint, OneDrive, Google Drive, Confluence and the web crawler, the preview also shows Open source. It opens the document in the source system in a new tab; the source system's permissions apply there, so you only see the document if you may open it there yourself. If you have connected your Microsoft or Google account to MeinGPT, MeinGPT shows the provider's own preview inside MeinGPT instead for SharePoint, OneDrive and Google Drive.
For these sources Open source replaces the preview of the copy stored in MeinGPT; only PDF citations with a page still open the copy at the cited page. Download original remains available wherever the source releases the original file.
All other sources (such as WebDAV, Amazon S3 or an Outpost) have no link into the source system. If the source releases the text or the original file, the citation opens the copy stored in MeinGPT; otherwise it shows only the matching passage.
Knowledge Map
Every cloud data source has its own Knowledge map tab (open the data source → Knowledge map). It shows how the documents in this source are actually used - not manually curated, but computed from real searches over the last 30 days:
- Folder map: Each area is a top-level folder; its size is the number of files, its colour how often searches find something there.
- Most used documents: The files with the most hits.
- Lots in it, never searched: Large folders where no search found anything in the period - a hint to review the scope.
The map shows frequencies only, no search text and no people.
Note
The tab needs prior search activity. Until this pool has been searched, the tab shows a message that knowledge is being indexed - this is not an error. The first snapshot usually appears within a few minutes of a search happening against this pool.
How MeinGPT works with files (3-stage model)
Not every request needs the same processing depth. MeinGPT decides per request how deep it has to go. There are three stages:
| Stage | What happens | Sufficient for |
|---|---|---|
| 1. Search | The platform searches all configured sources and returns snippets and filenames | Simple questions like "Is there a document about topic X?" |
| 2. Full-text retrieval | The model loads the complete content of individual files that look relevant after stage 1 | Content questions about individual, not overly large documents |
| 3. Code Sandbox | The original file is opened in an isolated environment (the sandbox) and processed with Python | Calculations, analyses, charts from large or structured files (e.g. Excel with many rows) |
You do not have to configure anything manually. More about the sandbox: Code Sandbox.
Attention
Full-text retrieval can use a lot of context. Stage 2 loads the complete content of a file or Confluence page into the chat at once – not just the relevant excerpt. When several large sources are connected to the same assistant or chat (for example a data source and a Confluence space), a single question can already consume a significant share of the model's context window. Keep an eye on the context window indicator in the chat input, and prefer narrow, targeted questions for very large or heavily nested pages (e.g. Confluence pages with many embedded macros). More on this: Chat too large for the model.
On-premise data sources carry one extra restriction at stage 3:
Note
Note for on-premise setups: Stages 2 and 3 (loading a file into the chat, sandbox) do not exist for Outpost data sources: the assistant cannot load a file from an Outpost into the chat or the sandbox, not even at the Text and original files level. An original file leaves the Outpost only when someone opens or downloads it from a citation.
Next up
Formats & Access
Supported file formats, emails and attachments, access control.
Sync Status & Troubleshooting
Check connection status, spot and fix sync failures.
Data Sources & RAG
Back to the overview: setup and configuration.
PDF citation pages
When a PDF citation includes a page and the source permits access to the original file, its preview opens that page. Pages are counted from the start of the PDF, including cover sheets and blank pages, so the position can differ from a printed page number.
Previously indexed PDFs need to be reprocessed before new answers can include page targets. Existing chat citations and sources without page information continue to open the whole file.
Release level: Outpost data sources only
For a data source on an Outpost, you set per folder in the Outpost app what may leave your premises. The platform only displays the reported value under Settings → Data sources and cannot change it.
| Level | What leaves the folder |
|---|---|
| Text only (preset) | The matching passages with the text directly before and after them, and the file name |
| Text and original files | Plus the file itself |
Only Text and original files unlocks opening and downloading the original file from a citation. The assistant, however, never loads files from an Outpost into the chat or the Code Sandbox. Who may use the data source at all still applies on top (Access control).
Attention
The release level applies equally to everyone with access to this source. There is currently no way to differentiate it by role or user group. Anyone with access to the data source gets the same level as everyone else with access.
Cloud data sources have no release level: whoever may use them gets the text, the original file and images.