Source connector

Amazon S3 connector

Amazon S3 integration for databases, warehouses and BI tools

Object storage for files, exports, and data lakes. Datrise captures Amazon S3 entities such as buckets, objects, prefixes, object metadata, versions and storage classes, then lands normalized and typed models for warehouse reporting, attribution, and AI workflows.

The Amazon S3 REST API (ListObjectsV2) + S3 Inventory + Event Notifications authenticates with API key, service account, incremental runs key on LastModified, and Amazon S3 caps calls at 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per partitioned key prefix, scaling roughly linearly by adding more prefixes.

Of the 6 objects, 6 sync as upserts keyed on the primary key and 0 as append-only streams, so each incremental run moves only what changed since the last watermark. Every object lands in its own table (for example `s3_buckets`) with a schema derived from the Amazon S3 API rather than a flattened CSV, in any of the 43 supported destinations.

Datrise is in early access: pick a destination below to open the Amazon S3 pipeline page, and join the waitlist to get the Amazon S3 connector with guided onboarding.

Incremental syncSchema-aware mapping43 destinations available

What Amazon S3 data Datrise extracts

Each object lands in its own typed table. Sync mode is derived from the object: history-style objects append, everything else upserts.

Object	Landing table	Sync mode
buckets	s3_buckets	Upsert by primary key, incremental on updated-at
objects	s3_objects	Upsert by primary key, incremental on updated-at
prefixes	s3_prefixes	Upsert by primary key, incremental on updated-at
object metadata	s3_object_metadata	Upsert by primary key, incremental on updated-at
versions	s3_versions	Upsert by primary key, incremental on updated-at
storage classes	s3_storage_classes	Upsert by primary key, incremental on updated-at

Amazon S3 API: setup facts for the connector

Verified against the vendor documentation on 2026-09-02. Unknown values are left blank rather than guessed.

API	Amazon S3 REST API (ListObjectsV2) + S3 Inventory + Event Notifications
Authentication	API key, service account — Requests are signed with AWS Signature Version 4 using either long-lived IAM user access keys or short-lived credentials from an IAM role/instance profile.
Pagination	token — `ListObjectsV2` returns up to 1,000 keys per call (`MaxKeys`) with `IsTruncated`/`NextContinuationToken`; pass the token back as `continuation-token` to get the next page.
Rate limit	3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per partitioned key prefix, scaling roughly linearly by adding more prefixes
Incremental cursor	LastModified — There is no server-side 'modified since' filter on `ListObjectsV2`; incremental pulls filter the per-object `LastModified` timestamp client-side after listing, or consume S3 Event Notifications for near-real-time capture instead of relisting.
Webhooks	Yes — S3 Event Notifications (ObjectCreated/ObjectRemoved to SNS, SQS, Lambda, or EventBridge)
Bulk export	Yes — S3 Inventory (bucket-level object listing exported as CSV, ORC, or Parquet, daily or weekly)
Deletes detectable	Yes — requires S3 Versioning; a DELETE writes a delete marker (`x-amz-delete-marker: true`) visible via ListObjectVersions/Inventory, or an `ObjectRemoved:*` event fires in near real time

Official API reference

What bites when replicating Amazon S3

`ListObjectsV2` caps every response at 1,000 keys (`MaxKeys`/`KeyCount`); a bucket with millions of objects needs many paginated calls via `NextContinuationToken` or S3 Inventory instead of live listing.
Request-rate limits apply per partitioned key prefix (3,500 write / 5,500 read requests per second), so a connector hammering one prefix throttles well before any account-wide ceiling is reached.
Detecting deletes needs S3 Versioning turned on first: without it, a deleted object simply vanishes and no delete marker is ever written for `ListObjectVersions` or Inventory to report.
S3 Event Notifications are delivered at-least-once and can arrive out of order, so a connector consuming `ObjectCreated`/`ObjectRemoved` events must deduplicate (e.g. on ETag plus event time) instead of trusting delivery order or count.

Compare this against every other documented API

Popular destinations for Amazon S3

Choose a destination below to open the ready pipeline page.

A
Airtable
Relational spreadsheet destination for ops and go-to-market teams.
View pipeline
A
Amazon Athena
Serverless SQL over S3 data lake tables.
View pipeline
A
Amazon DynamoDB
Serverless key-value and document store on AWS.
View pipeline
A
Amazon QuickSight
AWS serverless BI with SPICE and embedded analytics.
View pipeline
A
Amazon Redshift
AWS petabyte-scale warehouse with Spectrum.
View pipeline
A
Amazon S3 Data Lake
Object storage landing zone for parquet and snapshots.
View pipeline
A
Apache Superset
Open-source BI for SQL exploration, charts, and dashboard publishing.
View pipeline
A
Azure Data Lake Storage
ADLS Gen2 object storage for analytics workloads.
View pipeline
A
Azure Synapse
Microsoft analytics workspace with SQL pools.
View pipeline

Amazon S3 connector FAQ

What Amazon S3 data does the connector extract?: 6 objects: buckets, objects, prefixes, object metadata, versions and storage classes. Each lands in its own table with types preserved; history-style objects (0) load as append-only streams.
Is the Amazon S3 integration incremental?: Yes. After the initial full load, every run reads only records changed since the last watermark (updated-at or event timestamp) and applies an upsert or an append depending on the object.
Where can I load Amazon S3 data?: Into 43 destinations: relational databases, cloud warehouses, data lakes, spreadsheets and BI tools. Each Amazon S3 → destination pair has its own page with the object-to-table mapping.

Related sources

Other sources in the same category, each with its own connector page.

Early access

Launch your Amazon S3 pipeline with guided onboarding

Join the waitlist to get priority access for Amazon S3 integrations with guided setup, schema-aware mapping, and production-grade incremental sync.