Skip to content
DaneshyarDocs
Sign in
  • Introduction
  • Quickstart
  • What is Daneshyar?
  • The Apps page
  • Inside an app
  • Workspace: files and chat
  • Sharing and members
  • Notes and dictation
  • Language practice
  • Appointment booking
  • Automations
  • The file reader
  • Your profile
  • Wallet
  • Customer chats
  • API keys
  • Authentication
  • Apps, workspaces, resources
  • Ingestion
  • Chat and sessions
  • Citations
  • Skills
  • Conventions
  • Workspaces
  • Resources
  • Chat
  • Skills
  • Errors
  • Customer support assistant
  • Internal knowledge base
  • Documentation chatbot
  • Academy tutor
  • Embedded widget
  • Widget configuration
  • Framework guides
  • Security
  • Troubleshooting
  • FAQ
  1. Docs
  2. Concepts
  3. Ingestion

Ingestion

What happens between uploading a file and being able to ask about it.

Uploading a file returns immediately. Everything that makes it answerable happens afterwards, on a queue.

The four statuses

A resource is always in exactly one of four states:

  • In queueAccepted and waiting for a worker. Nothing has been read yet.
  • VectorizingBeing extracted, split and embedded. This is where the time goes.
  • ReadyIndexed and answerable. This is the only state retrieval uses.
  • PausedThe AI provider rate-limited us partway through, and it was still limited after a few automatic waits and retries. Big files are sent in small batches to make this rare. Nothing is wrong with the file and no work is lost, but it will not resume on its own — call the retry endpoint.
  • FailedSomething went wrong. error_message says what, and the resource will not become ready on its own.

What the pipeline does

Between queued and ready, three things happen:

  1. Extraction. A PDF is read page by page and images and tables inside it are described, not skipped. Audio and video are transcribed with timestamps. An image is described by the model. A text file (.txt, UTF-8) is read as it is. A Word file (.docx) gives its paragraphs and tables, one table row per line; pictures inside it are not read yet, and a .docx citation has no page number. An Excel file (.xlsx) or CSV gives every sheet, one row per line, and each row repeats the column names from the first row ("Prices, row 3: Service: Checkup | Price: 120000"); dates stay dates, formulas give their last saved value, and charts and pictures are not read.
  2. Segmentation. The text is merged into readable passages of roughly 500 to 1500 characters, broken at sentence ends so a passage doesn't stop mid-thought.
  3. Embedding. Each passage is turned into a vector and indexed, which is what makes retrieval by meaning rather than keyword possible.
An image resource produces no readable passages at all — it is described rather than transcribed. That's expected, not a failure.

Waiting for ready

There is no webhook. Poll the resource endpoint until status leaves the queued and vectorizing pair. A short PDF is usually ready in seconds; a long recording can take several minutes.

Poll every few seconds with a ceiling, and treat failed as terminal — retrying the read will never change it. If you upload in bulk, poll the list endpoint once rather than each resource separately.

cURL
JavaScript
Python
curl "https://api.daneshyar.info/api/v1/workspaces/8f14e45f-ceea-467a-9f4c-1a2b3c4d5e6f/resources/3c9a1b77-0d24-4e8b-b0f1-7a5e9c2d4b81/" \
  -H "X-API-Key: sk_9f3aC2xQ7mB1vT8sE4nK6pR0jY5wZ2hL" \
  | grep -o '"status": *"[^"]*"'

When ingestion fails

Ingestion fails for ordinary reasons: a scanned PDF with no recoverable text, a corrupt file, an audio track with no speech. error_message carries the reason.

Failures are per resource, not per workspace — one bad file does not stop the others from becoming answerable. Delete the failed resource and upload a better copy; there is no retry endpoint.

PreviousApps, workspaces, resourcesNextChat and sessions

On this page

  • The four statuses
  • What the pipeline does
  • Waiting for ready
  • When ingestion fails

Every example on this site runs against the live API. If one doesn't, tell us — that's a bug in the docs.

ServicesAboutTalk to us