cell | cell | cell,
which pollutes semantic search and still cannot answer “what is the total COGM for
B000”.
Pinter therefore treats tabular files (csv, xlsx, xls) as data, not text:
This is deliberate: retrieval over spreadsheet rows returns plausible-looking noise, while a
GROUP BY returns the exact number. Tabular documents are excluded from semantic search.What happens on upload
1
Validate and store the raw file
The extension must be
csv, xlsx, or xls, and the content must match the
extension. The original file is stored unchanged at
knowledge/<asset_id>/v1_<filename>.2
Skip the document parser
No parser job is submitted and no chunks or embeddings are created. The version
is marked
ready with chunk_count = 0, so the document appears as processed
right away.3
Normalize on first access
The first time the document is opened — in the UI detail view or through an agent
tool or route — the backend downloads the raw file and builds one SQL table per
sheet. The result is cached in GCS as
knowledge/<asset_id>/tabular/<checksum>-v1.sqlite.gz, so later requests on any
replica skip the work.Normalization rules
Each worksheet becomes its own table, named after the sheet:
A workbook with a single sheet, and every CSV file, uses the table name
data.
Column names are always quoted identifiers because real headers contain spaces and
parentheses ("Stok Admin (Ton)", "Beban Administrasi dan Umum").
The normalizer then applies these steps, recording every decision in the artifact
metadata:
Read tables as an agent
Agents never write SQL. They send a structured query — table, columns, filters, aggregations — and the backend compiles it with validated identifiers and bound parameters. That removes SQL injection entirely and keeps the model from inventing functions or joins that do not exist. Endpoints live under the knowledge document path and require the caller to have use permission on the document.GET /tables returns the schema the agent needs before querying:
values holds up to 20 distinct values for up to five low-cardinality TEXT
columns — the agent filters on real values ("ANPER": "B000") instead of guessing
the spelling. sample is columnar so column names are not repeated per row.
Query parts
Supported
op values: eq, neq, gt, gte, lt, lte, in, not_in,
contains, starts_with, between, is_null, is_not_null.
Supported fn values: count, count_distinct, sum, avg, min, max.
sum and avg are rejected on TEXT columns instead of returning a wrong number.
truncated: true means more rows matched than limit allowed — raise limit or
paginate with offset instead of reporting the partial count as the total.
In chat, agents use the built-in tools
get_table_schema(document_id, table?) and
query_table(document_id, table?, columns?, filters?, group_by?, aggregations?, order_by?, limit?). Same builder, same validation.Compare raw and parsed data in the UI
The document detail view shows both representations, so you can confirm what ingestion changed:- Parsed — the normalized tables (header detected, subtotals removed, typed columns). When a workbook has several sheets, a selector switches between them.
- Raw file — the untouched grid, columns labelled
A,B,C, including the title rows. Useful when a header was detected in the wrong place.
Limits and errors
Known limitations
- Sheets that stack several unrelated tables vertically are read as one table; only the first header row is used.
- Duplicate column labels in the source get
_2,_3suffixes when there is no group row above them to disambiguate. - Dates are read as text; cast them in SQL when you need date arithmetic.
- Cells are read from the cached formula results stored in the workbook. A workbook that was never recalculated may show stale or empty values.