au3 bucket & au3 bucket-location — Command Reference

CLI command groups: au3 bucket, au3 bucket-location

Commands for managing buckets (local, remote), querying and transferring bucket data, bucket object management, and bucket locations.

---

Sections

Section	Commands
Bucket (top-level)	10
Bucket management (sub-group)	3
bucket-location (separate group)	7

---

au3 bucket

au3 bucket list

List buckets

Option	Type	Required	Default	Notes
--fields	string			Comma-separate list of fields to display
--filter	string			Filter in JSON/YAML format or path to file containing filter
--sort	string			Ordered list of fields to sort by in JSON/YAML format or path to file containing sorting
--search	string			Search in JSON/YAML format or path to file containing search
--limit			100	Number of results to fetch
--skip				Number of results to skip
--all	boolean		false	Whether to fetch results from all pages
--stream	boolean		false	Whether to print results as they are received without buffering (as JSON)
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket view

View bucket

Option	Type	Required	Default	Notes
--id	string	✅		bucket id
--fields	string			comma-separate list of fields to display
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket create

Create bucket

Option	Type	Required	Default	Notes
--type	enum	✅		Choices: "local", "remote"
--name	string	✅		Object name with 2-64 characters
--description	string			Object description with 0 or 2-256 characters
--connection	string			Sync agent connection id
--discovered_id	string			Unique id for bucket within connection
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket update

Update bucket

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for update
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: UpdateBucketArgumentDto

This section documents the --spec payload accepted by au3 bucket update. Run bucket update --spec-schema to fetch this schema live from the CLI at any time.

Tip: additionalProperties: false — the schema rejects any field not listed below.

Top-Level Fields

Field	Type	Required	Notes
id	string		Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
name	string		Object name with 2-64 characters — 2–64 characters
description	string		Object description with 0 or 2-256 characters — Pattern: (^$)|(^.{2,256}$) — 0–256 characters
tags	array		List of tags

Example Spec

name: <string>
description: <string>

---

au3 bucket transfer-to-warehouse

Transfer data from bucket to warehouse

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for run
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: BucketTransferToWarehouseArgumentDto

This section documents the --spec payload accepted by au3 bucket transfer-to-warehouse. Run bucket transfer-to-warehouse --spec-schema to fetch this schema live from the CLI at any time.

Top-Level Fields

Field	Type	Required	Notes
source	object	✅
destination	object	✅

source

The source object is a discriminated union. One of the following 3 shapes applies.

1. remote_bucket_ingest

Field	Type	Required	Notes
type	const	✅	remote_bucket_ingest
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		Remote file path — e.g. /file/path with spaces
format	enum		Choices: parquet, ldjson, json, csv, psv, tsv, xlsx, nacha, origence:cudl-vehicle-loan-application, origence:cudl-vehicle-loan-application:address, origence:cudl-vehicle-loan-application:applicant, origence:cudl-vehicle-loan-application:application, origence:cudl-vehicle-loan-application:application-amount, origence:cudl-vehicle-loan-application:calculation, origence:cudl-vehicle-loan-application:collateral, origence:cudl-vehicle-loan-application:credit-score, origence:cudl-vehicle-loan-application:decision, origence:cudl-vehicle-loan-application:decision-reason, origence:cudl-vehicle-loan-application:employer, origence:cudl-vehicle-loan-application:funding, origence:cudl-vehicle-loan-application:seller, origence:cudl-vehicle-loan-application:stage-status-history
depth	integer		Depth to discover — Default: 1
walk	enum		Walk type — Choices: BFS, DFS — Default: BFS
include_source_meta	boolean		Adds the au_source_filename column with the full path and name of the source file. Only applies when destination.type is local_warehouse_ingest — Default: False
include_task_meta	boolean		Adds the au_task_created_at column with the ingestion time in UTC. Only applies when destination.type is local_warehouse_ingest — Default: False
schema_arg	string		Schema describing the data to be parsed
include_patterns	array		List of patterns to include when searching for part files. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when searching for part files. Only applicable if location points to a directory.
schema_file	string		Path to a file which contains the parsing schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces
header_rows	integer		Number of rows to skip at the beginning of each part file — Min 0 — Max 1 — Default: 1
footer_rows	integer		Number of rows to skip at the end of each part file — Min 0 — Default: 0
parse_as_string	boolean		Whether to treat all fields as strings. Only applicable to csv, psv, tsv, and xlsx format.
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10
transform	object
query	object
pagination	object

transform

Field	Type	Required	Notes
file	array	✅
field	object	✅

file (array items)

field

Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.

query

The query object is a discriminated union. One of the following 2 shapes applies.

Variant 1

Field	Type	Required	Notes
format	enum	✅	Choices: sql, json_sql, plugin_dsl
query	string	✅	Stringified query in the specified format

Variant 2

Field	Type	Required	Notes
format	enum	✅	Choices: entity
entity	string	✅	Unique id for a queryable entity within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*(-cells)?$
discovered_id	string		Unique id from a remote source

pagination

Field	Type	Required	Notes
limit	integer		Number of items to return per page. Each object type will have predefined defaults and maximums. — Min 0
skip	integer		Number of items to skip from previous pages — Min 0
sort	array		List of fields and directions to sort on. Order determines precedence.
fields	array		List of fields to include in result. Omit for all.
filter	object		Filter to narrow the returned results in filter JSON format
collation	object
total	boolean		Whether to compute the total count across all pages, which can sometimes be an expensive calculation — Default: False

sort (array items)

List of fields and directions to sort on. Order determines precedence.

Field	Type	Required	Notes
field	string	✅	Field key
direction	enum	✅	Direction to sort — Choices: ASC, DESC

collation

Field	Type	Required	Notes
locale	string	✅	Locale to use for string comparison
strength	integer	✅	Strength — Min 1 — Max 4

2. local_bucket_ingest

Field	Type	Required	Notes
type	const	✅	local_bucket_ingest
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		System file path — Pattern: ^[\/\a-zA-Z0-9_.\-: ]+$ — 0–256 characters — e.g. /file/path
format	enum		Choices: parquet, ldjson, json, csv, psv, tsv, xlsx, nacha, origence:cudl-vehicle-loan-application, origence:cudl-vehicle-loan-application:address, origence:cudl-vehicle-loan-application:applicant, origence:cudl-vehicle-loan-application:application, origence:cudl-vehicle-loan-application:application-amount, origence:cudl-vehicle-loan-application:calculation, origence:cudl-vehicle-loan-application:collateral, origence:cudl-vehicle-loan-application:credit-score, origence:cudl-vehicle-loan-application:decision, origence:cudl-vehicle-loan-application:decision-reason, origence:cudl-vehicle-loan-application:employer, origence:cudl-vehicle-loan-application:funding, origence:cudl-vehicle-loan-application:seller, origence:cudl-vehicle-loan-application:stage-status-history
schema_arg	string		Schema describing the data to be parsed
include_patterns	array		List of patterns to include when searching for part files. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when searching for part files. Only applicable if location points to a directory.
schema_file	string		Path to a file which contains the parsing schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces
header_rows	integer		Number of rows to skip at the beginning of each part file — Min 0 — Max 1 — Default: 1
footer_rows	integer		Number of rows to skip at the end of each part file — Min 0 — Default: 0
parse_as_string	boolean		Whether to treat all fields as strings. Only applicable to csv, psv, tsv, and xlsx format.
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10
transform	object
query	object
pagination	object

transform

Field	Type	Required	Notes
file	array	✅
field	object	✅

file (array items)

field

Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.

query

The query object is a discriminated union. One of the following 2 shapes applies.

Variant 1

Field	Type	Required	Notes
format	enum	✅	Choices: sql, json_sql, plugin_dsl
query	string	✅	Stringified query in the specified format

Variant 2

Field	Type	Required	Notes
format	enum	✅	Choices: entity
entity	string	✅	Unique id for a queryable entity within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*(-cells)?$
discovered_id	string		Unique id from a remote source

pagination

Field	Type	Required	Notes
limit	integer		Number of items to return per page. Each object type will have predefined defaults and maximums. — Min 0
skip	integer		Number of items to skip from previous pages — Min 0
sort	array		List of fields and directions to sort on. Order determines precedence.
fields	array		List of fields to include in result. Omit for all.
filter	object		Filter to narrow the returned results in filter JSON format
collation	object
total	boolean		Whether to compute the total count across all pages, which can sometimes be an expensive calculation — Default: False

sort (array items)

List of fields and directions to sort on. Order determines precedence.

Field	Type	Required	Notes
field	string	✅	Field key
direction	enum	✅	Direction to sort — Choices: ASC, DESC

collation

Field	Type	Required	Notes
locale	string	✅	Locale to use for string comparison
strength	integer	✅	Strength — Min 1 — Max 4

3. user_bucket_ingest

Field	Type	Required	Notes
type	const	✅	user_bucket_ingest
format	enum		Choices: parquet, ldjson, json, csv, psv, tsv, xlsx, nacha, origence:cudl-vehicle-loan-application, origence:cudl-vehicle-loan-application:address, origence:cudl-vehicle-loan-application:applicant, origence:cudl-vehicle-loan-application:application, origence:cudl-vehicle-loan-application:application-amount, origence:cudl-vehicle-loan-application:calculation, origence:cudl-vehicle-loan-application:collateral, origence:cudl-vehicle-loan-application:credit-score, origence:cudl-vehicle-loan-application:decision, origence:cudl-vehicle-loan-application:decision-reason, origence:cudl-vehicle-loan-application:employer, origence:cudl-vehicle-loan-application:funding, origence:cudl-vehicle-loan-application:seller, origence:cudl-vehicle-loan-application:stage-status-history
schema_arg	string		Schema describing the data to be parsed
include_patterns	array		List of patterns to include when searching for part files. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when searching for part files. Only applicable if location points to a directory.
schema_file	string		Path to a file which contains the parsing schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces
header_rows	integer		Number of rows to skip at the beginning of each part file — Min 0 — Max 1 — Default: 1
footer_rows	integer		Number of rows to skip at the end of each part file — Min 0 — Default: 0
parse_as_string	boolean		Whether to treat all fields as strings. Only applicable to csv, psv, tsv, and xlsx format.
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10
transform	object
query	object
pagination	object

transform

Field	Type	Required	Notes
file	array	✅
field	object	✅

file (array items)

field

Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.

query

The query object is a discriminated union. One of the following 2 shapes applies.

Variant 1

Field	Type	Required	Notes
format	enum	✅	Choices: sql, json_sql, plugin_dsl
query	string	✅	Stringified query in the specified format

Variant 2

Field	Type	Required	Notes
format	enum	✅	Choices: entity
entity	string	✅	Unique id for a queryable entity within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*(-cells)?$
discovered_id	string		Unique id from a remote source

pagination

Field	Type	Required	Notes
limit	integer		Number of items to return per page. Each object type will have predefined defaults and maximums. — Min 0
skip	integer		Number of items to skip from previous pages — Min 0
sort	array		List of fields and directions to sort on. Order determines precedence.
fields	array		List of fields to include in result. Omit for all.
filter	object		Filter to narrow the returned results in filter JSON format
collation	object
total	boolean		Whether to compute the total count across all pages, which can sometimes be an expensive calculation — Default: False

sort (array items)

List of fields and directions to sort on. Order determines precedence.

Field	Type	Required	Notes
field	string	✅	Field key
direction	enum	✅	Direction to sort — Choices: ASC, DESC

collation

Field	Type	Required	Notes
locale	string	✅	Locale to use for string comparison
strength	integer	✅	Strength — Min 1 — Max 4

destination

The destination object is a discriminated union. One of the following 4 shapes applies.

1. remote_warehouse_ingest

Field	Type	Required	Notes
type	const	✅	remote_warehouse_ingest
warehouse	string	✅	Warehouse id — Pattern: ^whs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. whs-123e4567-e89b-41d2-a345-678901234567
table	string	✅	Unique id for a table within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ — 2–64 characters
mode	enum	✅	Choices: overwrite, upsert
auto_migrate	boolean		Whether to automatically migrate the warehouse table to match the incoming schema

2. local_warehouse_ingest

Field	Type	Required	Notes
type	const	✅	local_warehouse_ingest
warehouse	string	✅	Warehouse id — Pattern: ^whs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. whs-123e4567-e89b-41d2-a345-678901234567
table	string	✅	Unique id for a table within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ — 2–64 characters
mode	enum	✅	Choices: overwrite, upsert, overwrite_cells, upsert_cells
auto_migrate	boolean		Whether to automatically migrate the warehouse table to match the incoming schema — Default: False
normalize_properties	boolean		Whether to automatically normalize the incoming property ids to au compatible ids — Default: False

3. accelerated_warehouse_ingest

Field	Type	Required	Notes
type	const	✅	accelerated_warehouse_ingest
warehouse	string	✅	Warehouse id — Pattern: ^whs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. whs-123e4567-e89b-41d2-a345-678901234567
table	string	✅	Unique id for a table within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ — 2–64 characters
mode	enum	✅	Choices: overwrite, upsert
auto_migrate	boolean		Whether to automatically migrate the warehouse table to match the incoming schema

4. user_warehouse_export

Field	Type	Required	Notes
type	const	✅	user_warehouse_export
format	enum	✅	Format for query results. Max limit is applied if "json". — Choices: parquet, ldjson, json, csv, psv, tsv — Default: json
header	enum		Whether to include a header row. For DSV, "short" omits type indicator. — Choices: none, full, short — Default: full
normalize_properties	boolean		Whether to automatically normalize the incoming property ids to au compatible ids — Default: False

Example Spec

source:
  type: remote_bucket_ingest
  bucket: bkt-123e4567-e89b-41d2-a345-678901234567
destination:
  type: local_warehouse_ingest
  warehouse: whs-123e4567-e89b-41d2-a345-678901234567
  table: <string>
  mode: overwrite

Ingestion metadata columns

These options apply only to transfers from a remote bucket (remote_bucket_ingest) to a local warehouse (local_warehouse_ingest).

You can add ingestion metadata to every record at ingestion time. This removes the need to add it later in a Dataflow.

Option	Column added	Value
include_source_meta: true	au_source_filename	Full path and name of the source file
include_task_meta: true	au_task_created_at	Time the file was ingested, in UTC

• All records from the same ingestion get the same values.
• The timestamp is the ingestion time, not the Dataflow run time. This avoids duplicates when a Dataflow runs more than once a day.
• Both options are off by default. Existing configurations are not affected.

Note: These options are available through the CLI spec today. UI support is coming in a future release.

Example: ingestion metadata

source:
  type: remote_bucket_ingest
  bucket: bkt-019740b0-8ddb-7554-a10d-f666bc6e4988
  # path: '/home/ubuntu/tmp/au3-bkt/many-big'
  path: '/home/ubuntu/tmp/au3-bkt/many-cars'
  format: csv
  # format: json
  depth: 1
  # walk: BFS
  header_rows: 1
  # footer_rows: 1096
  parse_as_string: true
  include_patterns:
    - '.*cars3\.csv$'
  # include_patterns:
  #   - '.*\.json$'
  # exclude_patterns:
  #   - '.*\.psv$'
  # concurrency: 1
  include_source_meta: true
  include_task_meta: true
  # trigger_window:
  #   ttl: 900000
  #   check_interval_ms: 30000
  #   ready_file_root_path: '/home/ubuntu/tmp/au3-bkt/many-cars'
  #   ready_file_pattern: 'ready'
  #   ready_file_modified_offset_duration: 'PT1M' # 1 minute
  # trigger_only: true
destination:
  type: local_warehouse_ingest
  warehouse: whs-01920b54-0d4b-7eed-9691-c1176301babc
  table: tbl_0197ad14_1e56_7cce_91ac_b4ab28921c70 # CustomBucket
  mode: overwrite
  auto_migrate: true
  normalize_properties: true

---

au3 bucket list-history

List bucket history

Option	Type	Required	Default	Notes
--id	string	✅		bucket id
--fields	string			comma-separate list of fields to display
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket delete

Delete a bucket

Option	Type	Required	Default	Notes
--id	string	✅		Bucket id
--trash	boolean
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket query

Query Bucket

Option	Type	Required	Default	Notes
--bucket	string	✅		Bucket to run query against
--path	string	✅		Path within the bucket to query
--source_format	enum	✅		Format of the data to query Choices: "parquet", "ldjson", "json", "csv", "psv", "tsv", "xlsx", "nacha", "origence:cudl-vehicle-loan-application", "origence:cudl-vehicle-loan-application:address", "origence:cudl-vehicle-loan-application:applicant", "origence:cudl-vehicle-loan-application:application", "origence:cudl-vehicle-loan-application:application-amount", "origence:cudl-vehicle-loan-application:calculation", "origence:cudl-vehicle-loan-application:collateral", "origence:cudl-vehicle-loan-application:credit-score", "origence:cudl-vehicle-loan-application:decision", "origence:cudl-vehicle-loan-application:decision-reason", "origence:cudl-vehicle-loan-application:employer", "origence:cudl-vehicle-loan-application:funding", "origence:cudl-vehicle-loan-application:seller", "origence:cudl-vehicle-loan-application:stage-status-history"
--depth				Depth to look for files within the path
--walk	enum			Method to walk the bucket path Choices: "BFS", "DFS"
--header_rows				Number of header rows to skip when reading the data
--footer_rows				Number of footer rows to skip when reading the data
--include_patterns	string			Comma-separated list of regex patterns to include
--exclude_patterns	string			Comma-separated list of regex patterns to exclude
--concurrency				Number of concurrent file reads/uploads
--parse_as_string	boolean			Whether to parse all fields as string
--normalize_properties	boolean			Whether to automatically normalize the incoming property ids to au compatible ids
--limit				Number of rows to limit result to
--skip				Number of rows to skip
--total	boolean			Whether to calculate total number of rows across all pages
--sort	string			Ordered list of fields to sort by in JSON/YAML format or path to file containing sorting
--filter	string			Filter in JSON/YAML format or path to file containing filter
--fields	string			Comma-separate list of fields to display
--format	enum		json	Choices: "parquet", "ldjson", "json", "csv", "psv", "tsv"
--header	enum		full	Header format to include. Not applicable for "json" or "parquet" format. Choices: "none", "full", "short"
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket transfer-to-bucket

Transfer to Bucket

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for run
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: BucketTransferToBucketArgumentDto

This section documents the --spec payload accepted by au3 bucket transfer-to-bucket. Run bucket transfer-to-bucket --spec-schema to fetch this schema live from the CLI at any time.

Top-Level Fields

Field	Type	Required	Notes
source	object	✅
destination	object	✅

source

The source object is a discriminated union. One of the following 3 shapes applies.

1. remote_bucket_sync

Field	Type	Required	Notes
type	const	✅	remote_bucket_sync
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		Remote file path — e.g. /file/path with spaces
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.
depth	integer		Depth to discover — Default: 1
walk	enum		Walk type — Choices: BFS, DFS — Default: BFS

2. local_bucket_sync

Field	Type	Required	Notes
type	const	✅	local_bucket_sync
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		System file path — Pattern: ^[\/\a-zA-Z0-9_.\-: ]+$ — 0–256 characters — e.g. /file/path
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.
depth	integer		Depth to discover — Default: 1

3. user_bucket_sync

Field	Type	Required	Notes
type	const	✅	user_bucket_sync
path	string	✅	Remote file path — e.g. /file/path with spaces
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.
depth	integer		Depth to discover — Default: 1
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10

destination

The destination object is a discriminated union. One of the following 3 shapes applies.

1. remote_bucket_sync

Field	Type	Required	Notes
type	const	✅	remote_bucket_sync
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		Remote file path — e.g. /file/path with spaces

2. local_bucket_sync

Field	Type	Required	Notes
type	const	✅	local_bucket_sync
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
path	string		System file path — Pattern: ^[\/\a-zA-Z0-9_.\-: ]+$ — 0–256 characters — e.g. /file/path
write_settings	object		Settings that govern how to handle writes during bucket sync operations

write_settings

Settings that govern how to handle writes during bucket sync operations

Field	Type	Required	Notes
assert_src_state_pre	enum		Assert Source State Pre-Sync — Choices: empty, not_empty
assert_dst_state_pre	enum		Assert Destination State Pre-Sync — Choices: empty, not_empty
assert_src_state_post	enum		Assert Source State Post-Sync — Choices: empty, not_empty
assert_dst_state_post	enum		Assert Destination State Post-Sync — Choices: empty, not_empty
handle_merge_src_only	enum	✅	How to handle objects which only appear in source — Choices: copy, skip, fail — Default: copy
handle_merge_dst_only	enum	✅	How to handle objects which only appear in destination. — Choices: keep, remove, fail — Default: keep
handle_merge_conflict	enum	✅	How to handle objects which appear in both source and destination which have different content (per size & checksum) — Choices: src, dst, newer, older, rename, fail — Default: src
handle_merge_no_conflict	enum	✅	How to handle objects which appear in both source and destination which have the same content (per size and checksum) — Choices: newer, touch, skip, fail — Default: skip
handle_merge_type_conflict	enum	✅	How to handle objects which are a (full-segment) of 1+ other objects (i.e. file at folder path) — Choices: src, dst, obj, prefix, newer, fail — Default: fail

3. user_bucket_sync

Field	Type	Required	Notes
type	const	✅	user_bucket_sync
path	string	✅	Remote file path — e.g. /file/path with spaces
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10
write_settings	object		Settings that govern how to handle writes during bucket sync operations

write_settings

Settings that govern how to handle writes during bucket sync operations

Field	Type	Required	Notes
assert_src_state_pre	enum		Assert Source State Pre-Sync — Choices: empty, not_empty
assert_dst_state_pre	enum		Assert Destination State Pre-Sync — Choices: empty, not_empty
assert_src_state_post	enum		Assert Source State Post-Sync — Choices: empty, not_empty
assert_dst_state_post	enum		Assert Destination State Post-Sync — Choices: empty, not_empty
handle_merge_src_only	enum	✅	How to handle objects which only appear in source — Choices: copy, skip, fail — Default: copy
handle_merge_dst_only	enum	✅	How to handle objects which only appear in destination. — Choices: keep, remove, fail — Default: keep
handle_merge_conflict	enum	✅	How to handle objects which appear in both source and destination which have different content (per size & checksum) — Choices: src, dst, newer, older, rename, fail — Default: src
handle_merge_no_conflict	enum	✅	How to handle objects which appear in both source and destination which have the same content (per size and checksum) — Choices: newer, touch, skip, fail — Default: skip
handle_merge_type_conflict	enum	✅	How to handle objects which are a (full-segment) of 1+ other objects (i.e. file at folder path) — Choices: src, dst, obj, prefix, newer, fail — Default: fail

Example Spec

source:
  type: remote_bucket_sync
  bucket: bkt-123e4567-e89b-41d2-a345-678901234567
destination:
  type: local_bucket_sync
  bucket: bkt-123e4567-e89b-41d2-a345-678901234567

---

au3 bucket management

au3 bucket management list-objects

List objects in a bucket

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for run
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: ListBucketObjectsArgumentDto

This section documents the --spec payload accepted by au3 bucket management list-objects. Run bucket management list-objects --spec-schema to fetch this schema live from the CLI at any time.

Tip: additionalProperties: false — the schema rejects any field not listed below.

Top-Level Fields

Field	Type	Required	Notes
source	object	✅

source

The source object is a discriminated union. One of the following 2 shapes applies.

1. remote_bucket_mgmt

Field	Type	Required	Notes
type	const	✅	remote_bucket_mgmt
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
paths	array	✅
depth	integer		Depth to discover — Default: 1
walk	enum		Walk type — Choices: BFS, DFS — Default: BFS
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.

2. local_bucket_mgmt

Field	Type	Required	Notes
type	const	✅	local_bucket_mgmt
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
paths	array	✅
depth	integer		Depth to discover — Default: 1
walk	enum		Walk type — Choices: BFS, DFS — Default: BFS
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.

Example Spec

source:
  type: remote_bucket_mgmt
  bucket: bkt-123e4567-e89b-41d2-a345-678901234567
  paths:
  - /file/path with spaces

---

au3 bucket management remove-objects

Remove objects from a bucket

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for run
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: RemoveBucketObjectsArgumentDto

This section documents the --spec payload accepted by au3 bucket management remove-objects. Run bucket management remove-objects --spec-schema to fetch this schema live from the CLI at any time.

Tip: additionalProperties: false — the schema rejects any field not listed below.

Top-Level Fields

Field	Type	Required	Notes
source	object	✅

source

The source object is a discriminated union. One of the following 1 shapes applies.

1. local_bucket_mgmt

Field	Type	Required	Notes
type	const	✅	local_bucket_mgmt
bucket	string	✅	Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567
bucket_location	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
paths	array	✅
depth	integer		Depth to discover — Default: 1
walk	enum		Walk type — Choices: BFS, DFS — Default: BFS
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.

Example Spec

source:
  type: local_bucket_mgmt
  bucket: bkt-123e4567-e89b-41d2-a345-678901234567
  paths:
  - /file/path

---

au3 bucket-location

au3 bucket-location list

List bucket locations

Option	Type	Required	Default	Notes
--fields	string			Comma-separate list of fields to display
--filter	string			Filter in JSON/YAML format or path to file containing filter
--sort	string			Ordered list of fields to sort by in JSON/YAML format or path to file containing sorting
--search	string			Search in JSON/YAML format or path to file containing search
--limit			100	Number of results to fetch
--skip				Number of results to skip
--all	boolean		false	Whether to fetch results from all pages
--stream	boolean		false	Whether to print results as they are received without buffering (as JSON)
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket-location view

View bucket location

Option	Type	Required	Default	Notes
--id	string	✅		bucket location id
--fields	string			comma-separate list of fields to display
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket-location create

Create bucket location

Option	Type	Required	Default	Notes
--bucket	string	✅		Bucket id
--name	string	✅		Object name with 2-64 characters
--description	string			Object description with 0 or 2-256 characters
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket-location update

Update bucket location

Option	Type	Required	Default	Notes
--spec-schema	boolean			view the schema for update
--spec	string			spec in JSON/YAML format or path to file containing spec
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

--spec payload reference

Schema ID: UpdateBucketLocationArgumentDto

This section documents the --spec payload accepted by au3 bucket-location update. Run bucket-location update --spec-schema to fetch this schema live from the CLI at any time.

Tip: additionalProperties: false — the schema rejects any field not listed below.

Top-Level Fields

Field	Type	Required	Notes
id	string		Bucket Location id — Pattern: ^bktl-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bktl-123e4567-e89b-41d2-a345-678901234567
name	string		Object name with 2-64 characters — 2–64 characters
description	string		Object description with 0 or 2-256 characters — Pattern: (^$)|(^.{2,256}$) — 0–256 characters
tags	array		List of tags
path	string		Remote file path — e.g. /file/path with spaces
directory	boolean		Whether the location is expected to be a directory
ingest	object		Configuration if location is intended for ingestion
sync	object		Configuration if the location is intended for sync
export	object

ingest

Configuration if location is intended for ingestion

Field	Type	Required	Notes
format	enum	✅	Choices: parquet, ldjson, json, csv, psv, tsv, xlsx, nacha, origence:cudl-vehicle-loan-application, origence:cudl-vehicle-loan-application:address, origence:cudl-vehicle-loan-application:applicant, origence:cudl-vehicle-loan-application:application, origence:cudl-vehicle-loan-application:application-amount, origence:cudl-vehicle-loan-application:calculation, origence:cudl-vehicle-loan-application:collateral, origence:cudl-vehicle-loan-application:credit-score, origence:cudl-vehicle-loan-application:decision, origence:cudl-vehicle-loan-application:decision-reason, origence:cudl-vehicle-loan-application:employer, origence:cudl-vehicle-loan-application:funding, origence:cudl-vehicle-loan-application:seller, origence:cudl-vehicle-loan-application:stage-status-history
depth	integer	✅	Depth to discover — Default: 1
walk	enum	✅	Walk type — Choices: BFS, DFS — Default: BFS
schema_arg	string		Schema describing the data to be parsed
include_patterns	array		List of patterns to include when searching for part files. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when searching for part files. Only applicable if location points to a directory.
schema_file	string		Path to a file which contains the parsing schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces
header_rows	integer		Number of rows to skip at the beginning of each part file — Min 0 — Max 1 — Default: 1
footer_rows	integer		Number of rows to skip at the end of each part file — Min 0 — Default: 0
skip_rows	integer		Number of rows to skip at the start of each part file before the header — Min 0 — Default: 0
parse_as_string	boolean		Whether to treat all fields as strings. Only applicable to csv, psv, tsv, and xlsx format.
filter_invalid_characters	boolean		Whether to filter out invalid characters from the input prior to ingestion
transform	object
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10

transform

Field	Type	Required	Notes
file	array	✅
field	object	✅

file (array items)

field

Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.

sync

Configuration if the location is intended for sync

Field	Type	Required	Notes
depth	integer	✅	Depth to discover — Default: 1
walk	enum	✅	Walk type — Choices: BFS, DFS — Default: BFS
include_patterns	array		List of patterns to include when syncing. Only applicable if location points to a directory.
exclude_patterns	array		List of patterns to exclude when syncing. Only applicable if location points to a directory.
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10
write_settings	object		Settings that govern how to handle writes during bucket sync operations

write_settings

Settings that govern how to handle writes during bucket sync operations

Field	Type	Required	Notes
assert_src_state_pre	enum		Assert Source State Pre-Sync — Choices: empty, not_empty
assert_dst_state_pre	enum		Assert Destination State Pre-Sync — Choices: empty, not_empty
assert_src_state_post	enum		Assert Source State Post-Sync — Choices: empty, not_empty
assert_dst_state_post	enum		Assert Destination State Post-Sync — Choices: empty, not_empty
handle_merge_src_only	enum		How to handle objects which only appear in source — Choices: copy, skip, fail — Default: copy
handle_merge_dst_only	enum		How to handle objects which only appear in destination. — Choices: keep, remove, fail — Default: keep
handle_merge_conflict	enum		How to handle objects which appear in both source and destination which have different content (per size & checksum) — Choices: src, dst, newer, older, rename, fail — Default: src
handle_merge_no_conflict	enum		How to handle objects which appear in both source and destination which have the same content (per size and checksum) — Choices: newer, touch, skip, fail — Default: skip
handle_merge_type_conflict	enum		How to handle objects which are a (full-segment) of 1+ other objects (i.e. file at folder path) — Choices: src, dst, obj, prefix, newer, fail — Default: fail

export

Field	Type	Required	Notes
format	enum	✅	Format for query results. Max limit is applied if "json". — Choices: parquet, ldjson, json, csv, psv, tsv — Default: json
max_parts	integer		Maximum number of part files to export to. Mutually exclusive with max_part_size. — Min 1
max_part_size	string		Maximum part file size to export to. Mutually exclusive with max_parts.
schema_file	string		Path to a file to export latest schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces
header	enum		Whether to include a header row. For DSV, "short" omits type indicator. — Choices: none, full, short — Default: full
concurrency	integer		Concurrency of processing remote files — Min 1 — Default: 10

Example Spec

name: <string>
description: <string>

---

au3 bucket-location list-history

List bucket location history

Option	Type	Required	Default	Notes
--id	string	✅		bucket location id
--fields	string			comma-separate list of fields to display
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

au3 bucket-location delete

Delete a bucket location

Option	Type	Required	Default	Notes
--id	string	✅		Bucket Location id
--trash	boolean
--fmt	enum		human	Output format Choices: "human", "json", "yaml"
--debug	boolean		false	Enable more verbose output on error

---

Generated from au3 bucket --help, au3 bucket-location --help, and subcommand help output.