---
title: "Assist Bucket Ingestion"
description: "Point Edge Assist at an Edge Storage bucket: supported document formats, the ETag diff cycle, the 10,000-document cap, and private access with per-site API keys and visitor tokens."
url: https://edge.network/docs/assist/bucket-ingestion/
---

# Assist Bucket Ingestion

Assist

# Bucket Ingestion

The knowledge-base tier: point Assist at an [Edge Storage](https://edge.network/storage) bucket
and drop documents in. No index file, no build step — the bucket is the source of truth,
and Assist keeps itself in sync with it. Available on pay-as-you-go.

## How It Works

- Create a bucket in **Storage** and upload your documents.
- In **Assist → Ingestion**, switch the content source to **Storage bucket**, pick the bucket (and an optional key prefix like `docs/`), and save.
- Assist lists the bucket hourly and diffs it by ETag: new and changed objects are parsed and indexed, deleted objects drop out of the index. Only what changed is re-processed.

There is no push API by design — uploading, replacing, or deleting objects in the bucket
*is* the ingestion interface. Any tool that can write to Edge Storage (the S3-compatible
API, the Control panel, a CI job) can maintain the knowledge base.

Any content change clears the site's [answer cache](https://edge.network/docs/assist/caching),
exactly as with index-file ingestion — and "Re-ingest now" in Control forces a diff immediately.

## Supported Formats

| Extension | Notes |
| `.md / .markdown` | Front matter is stripped; the first heading becomes the document title; code fences are skipped. |
| `.txt` | Ingested as-is. The filename becomes the title. |
| `.html / .htm` | Tags, scripts, and styles are stripped; the becomes the document title. |
| `.pdf` | Text extraction only — scanned or image-only PDFs have no extractable text and are recorded as parse errors. |

Everything else in the bucket is skipped. Documents are chunked into ~1,400-character
passages on paragraph and sentence boundaries — the same retrieval unit as the index-file
tier — and cited in answers by document name. A document that fails to parse never blocks
the rest of the bucket; it's listed with its error on the Ingestion tab and retried
automatically when its content changes (or on a forced re-ingest).

## Limits

| Limit | Value |
| Documents per site | 10,000 |
| Size per document | 20 MB |
| Chunks per site | 50,000 |

Knowledge bases beyond 10,000 documents are handled by
[Expert Services](https://edge.network/solutions/expert-services) —
custom retrieval, access control per prefix, and dedicated capacity.

## Private Access

Bucket-sourced sites are **private by default** — an
internal knowledge base shouldn't be one lifted sitekey away from public. A private site
rejects anonymous search, ask, and config calls; access comes from per-site
**API keys**, managed in **Assist → Settings → Access**.
Keys are shown once at creation and stored hashed. (You can flip a site back to public
there too — public sites are protected by the Origin check instead.)

### The widget on a private site

Browsers never see the API key. Your server exchanges it for a short-lived visitor token
(15-minute expiry, site-scoped) and renders the token into the embed:

```
# Server-side: exchange your API key for a short-lived visitor token
curl -X POST https://assist.edge.network/api/token \
  -H "Authorization: Bearer eak_your_site_api_key" \
  -H "Content-Type: application/json" \
  -d '{"sitekey": "ea_your_sitekey"}'

# → { "token": "eat_…", "expiresIn": 900 }
```

```
<!-- Render the token into the embed — the API key never reaches the browser -->
<script src="https://assist.edge.network/widget.js"
  data-sitekey="ea_your_sitekey"
  data-token="eat_…" defer></script>
```

Mint the token in the request that serves the page (behind your own login), and only
authenticated users can ever reach the assistant. The hosted view works the same way:
link to `/w/ ?token=eat_…`.

### Headless use

Server-side callers can skip tokens entirely and present the API key directly:

```
# Headless: the API key works directly on search and ask
curl "https://assist.edge.network/api/search?sitekey=ea_your_sitekey&q=refund+policy" \
  -H "Authorization: Bearer eak_your_site_api_key"
```

## Next Steps

[Content Ingestion — the index-file tier for website content](https://edge.network/docs/assist/ingestion) [API Reference — search, ask, and token endpoints](https://edge.network/docs/assist/api) [Edge Storage — buckets, uploads, and the S3-compatible API](https://edge.network/docs/storage)
[Back to Docs](https://edge.network/docs) [Need help?](https://edge.network/support)