Document properties from companion files

Many documents do not say what they belong to: a scanned delivery note carries its order number only in the document management system, not in its text. If you put a small companion file with its properties beside each document, the Outpost reads it too. Search then finds the document by its order number, and to a question like "Which documents belong to order 30600?" MeinGPT names the matching files.

How the companion file is named

The name is the document's full file name plus .meta.json or .meta.txt:

Orders/
  P00214556.pdf
  P00214556.pdf.meta.json
  P00214557.docx
  P00214557.docx.meta.txt

The document's extension is part of the name. P00214556.json is not enough: P00214556.pdf and P00214556.docx could sit side by side, and an ordinary JSON file would otherwise be read as properties. On Windows, upper and lower case do not matter. If a document has both variants, .meta.json wins.

Content

{
  "Auftragsnummer": "30600",
  "Bestellnummer": 4500120017,
  "Dokumentenart": "Lieferschein",
  "Schlagworte": ["Flansch", "DN100"]
}

Character encoding: UTF-8 (with or without BOM) or UTF-16 with BOM. A .meta.txt may also be Windows-1252, as older export tools write it.

Limits

Properties beyond these limits are left out; the Outpost uses the rest. It skips a companion file that is too large or unreadable entirely.

What the Outpost does with it

• The properties belong to the document. They are stored as an extra text section of the document. A search hit points to the document, not to the companion file, and the companion file does not appear as a document of its own.
• Changes apply without reading the document again. When a companion file is added, changed or deleted, the Outpost updates only the properties. It does not read the document again, and no text recognition runs.
• Without a document it is an ordinary file. A companion file with no document beside it is read like any other text file.
• Errors do not hold up the document. If a companion file is invalid, the document is indexed without properties. The Outpost notes the reason in its technical log, which is part of the diagnostics package.
• Documents without text stay out. If a document yields no text, such as a scan text recognition could not read or a file in a format the Outpost does not read, its properties do not make it searchable either.
• Links leading outside are not read. If a companion file is a symbolic link to a file outside the shared folder, the Outpost skips it, as it does for documents.

Companion files are always read once a folder is shared. There is no separate setting for them.

Who sees the properties

The properties are visible to exactly the people who may find the document. With Windows user permissions (beta), the Outpost only takes them over when the companion file has the same NTFS permissions as the document: the same owner, the same group and the same access rules. Otherwise it skips the companion file, so nobody reads properties through the document that are meant to stay hidden from them. If only the companion file's permissions change, the Outpost checks again once the companion file or the document changes.

Asking about an order

A question about a number finds the documents that carry this number in their properties and puts them ahead of similar documents without it. This also works for questions that combine content and number, such as "What test pressure does the test report for order 30600 state?".