au3 solution installation phase acquisition — Command Reference¶
CLI command group: au3 solution installation phase acquisition
Commands for the Acquisition phase: managing transfers that pull data from the Remote Source Model into the Local Source Model.
au3 solution installation phase acquisition transfer¶
au3 solution installation phase acquisition transfer list¶
List transfers
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--fields |
string | Comma-separate list of fields to display | ||
--filter |
string | Filter in JSON/YAML format or path to file containing filter | ||
--sort |
string | Ordered list of fields to sort by in JSON/YAML format or path to file containing sorting | ||
--search |
string | Search in JSON/YAML format or path to file containing search | ||
--limit |
100 | Number of results to fetch | ||
--skip |
Number of results to skip | |||
--all |
boolean | false | Whether to fetch results from all pages | |
--stream |
boolean | false | Whether to print results as they are received without buffering (as JSON) | |
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
au3 solution installation phase acquisition transfer view¶
View transfer
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--id |
string | ✅ | transfer id | |
--fields |
string | comma-separate list of fields to display | ||
--update-spec |
boolean | print in the form which is usable for the update command | ||
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
au3 solution installation phase acquisition transfer create¶
Create transfer
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--name |
string | Object name with 2-64 characters | ||
--description |
string | Object description with 0 or 2-256 characters | ||
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
au3 solution installation phase acquisition transfer update¶
Update transfer
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--spec-schema |
boolean | view the schema for update | ||
--spec |
string | spec in JSON/YAML format or path to file containing spec | ||
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
--spec payload reference¶
Schema ID: UpdateSolutionInstallationDraftAcquisitionTransferArgumentDto
This section documents the --spec payload accepted by au3 solution installation phase acquisition transfer update. Run solution installation phase acquisition transfer update --spec-schema to fetch this schema live from the CLI at any time.
Tip:
additionalProperties: false— the schema rejects any field not listed below.
Top-Level Fields¶
| Field | Type | Required | Notes |
|---|---|---|---|
id |
string | ✅ | Id of acquisition phase transfer — Pattern: ^siacqtrf-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ |
name |
string | Object name with 2-64 characters — 2–64 characters | |
description |
string | Object description with 0 or 2-256 characters — Pattern: (^$)|(^.{2,256}$) — 0–256 characters |
|
source |
object | ✅ | |
destination |
object | ✅ | |
dependencies |
array |
Ids of other acquisition transfers this transfer depends on | |
error_handling |
object | Options for error handling |
source¶
The source object is a discriminated union. One of the following 3 shapes applies.
1. remote_bucket_sync¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | remote_bucket_sync |
path |
string | Remote file path — e.g. /file/path with spaces |
|
include_patterns |
array |
List of patterns to include when syncing. Only applicable if location points to a directory. | |
exclude_patterns |
array |
List of patterns to exclude when syncing. Only applicable if location points to a directory. | |
bucket |
string | ✅ | Id of logical bucket which maps to primary or secondary bucket depending on the connection's mode — Pattern: ^sirsmbkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ |
2. remote_bucket_ingest¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | remote_bucket_ingest |
path |
string | Remote file path — e.g. /file/path with spaces |
|
format |
enum | Choices: parquet, ldjson, json, csv, psv, tsv, xlsx, nacha, origence:cudl-vehicle-loan-application, origence:cudl-vehicle-loan-application:address, origence:cudl-vehicle-loan-application:applicant, origence:cudl-vehicle-loan-application:application, origence:cudl-vehicle-loan-application:application-amount, origence:cudl-vehicle-loan-application:calculation, origence:cudl-vehicle-loan-application:collateral, origence:cudl-vehicle-loan-application:credit-score, origence:cudl-vehicle-loan-application:decision, origence:cudl-vehicle-loan-application:decision-reason, origence:cudl-vehicle-loan-application:employer, origence:cudl-vehicle-loan-application:funding, origence:cudl-vehicle-loan-application:seller, origence:cudl-vehicle-loan-application:stage-status-history |
|
depth |
integer | Depth to discover — Default: 1 |
|
walk |
enum | Walk type — Choices: BFS, DFS — Default: BFS |
|
schema_arg |
string | Schema describing the data to be parsed | |
include_patterns |
array |
List of patterns to include when searching for part files. Only applicable if location points to a directory. | |
exclude_patterns |
array |
List of patterns to exclude when searching for part files. Only applicable if location points to a directory. | |
schema_file |
string | Path to a file which contains the parsing schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces |
|
header_rows |
integer | Number of rows to skip at the beginning of each part file — Min 0 — Max 1 — Default: 1 |
|
footer_rows |
integer | Number of rows to skip at the end of each part file — Min 0 — Default: 0 |
|
parse_as_string |
boolean | Whether to treat all fields as strings. Only applicable to csv, psv, tsv, and xlsx format. | |
transform |
object | ||
concurrency |
integer | Concurrency of processing remote files — Min 1 — Default: 10 |
|
trigger_window |
object | ||
bucket |
string | ✅ | Id of logical bucket which maps to primary or secondary bucket depending on the connection's mode — Pattern: ^sirsmbkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ |
transform
| Field | Type | Required | Notes |
|---|---|---|---|
file |
array | ✅ | |
field |
object | ✅ |
file (array items)
field
Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.
trigger_window
| Field | Type | Required | Notes |
|---|---|---|---|
check_interval_ms |
integer | ✅ | Interval in milliseconds to check for ready state. — Min 10000 |
ttl |
integer | ✅ | Time to live in milliseconds. Used to compute an expiry. — Min 0 — Max 21600000 |
ready_file_root_path |
string | ✅ | Remote file path — e.g. /file/path with spaces |
ready_file_pattern |
string | ✅ | Regex pattern for matching file paths to include/exclude — Format: regex |
ready_file_modified_offset_duration |
string | ✅ | ISO 8601 duration string to determine if ready file is modified more recently than the task start time minus this duration. — Pattern: ^P(?!$)([0-9]+(?:\.[0-9]+)?Y)?([0-9]+(?:\.[0-9]+)?M)?([0-9]+(?:\.[0-9]+)?W)?([0-9]+(?:\.[0-9]+)?D)?(T(?=[0-9])([0-9]+(?:\.[0-9]+)?H)?([0-9]+(?:\.[0-9]+)?M)?([0-9]+(?:\.[0-9]+)?S)?)?$ — Default: P1D |
3. remote_warehouse_query¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | remote_warehouse_query |
query |
object | ||
entity_aliases |
object | Aliases used within query for substitution | |
pagination |
object | ||
trigger_window |
object | ||
warehouse |
string | ✅ | Id of logical warehouse which maps to primary or secondary warehouse depending on the connection's mode — Pattern: ^sirsmwhs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ |
query
The query object is a discriminated union. One of the following 2 shapes applies.
Variant 1
| Field | Type | Required | Notes |
|---|---|---|---|
format |
enum | ✅ | Choices: sql, json_sql, plugin_dsl |
query |
string | ✅ | Stringified query in the specified format |
Variant 2
| Field | Type | Required | Notes |
|---|---|---|---|
format |
enum | ✅ | Choices: entity |
entity |
string | ✅ | Unique id for a queryable entity within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*(-cells)?$ |
discovered_id |
string | Unique id from a remote source |
entity_aliases
Aliases used within query for substitution
Map keyed by field name matching pattern ^[a-zA-Z][a-zA-Z0-9_]*$.
| Field | Type | Required | Notes |
|---|---|---|---|
warehouse |
string | Warehouse id — Pattern: ^whs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. whs-123e4567-e89b-41d2-a345-678901234567 |
|
entity |
string | ✅ | Unique id for a table/view within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ — 2–64 characters |
pagination
| Field | Type | Required | Notes |
|---|---|---|---|
limit |
integer | Number of items to return per page. Each object type will have predefined defaults and maximums. — Min 0 | |
skip |
integer | Number of items to skip from previous pages — Min 0 | |
sort |
array | List of fields and directions to sort on. Order determines precedence. | |
fields |
array |
List of fields to include in result. Omit for all. | |
filter |
object | Filter to narrow the returned results in filter JSON format | |
collation |
object | ||
total |
boolean | Whether to compute the total count across all pages, which can sometimes be an expensive calculation — Default: False |
sort (array items)
List of fields and directions to sort on. Order determines precedence.
| Field | Type | Required | Notes |
|---|---|---|---|
field |
string | ✅ | Field key |
direction |
enum | ✅ | Direction to sort — Choices: ASC, DESC |
collation
| Field | Type | Required | Notes |
|---|---|---|---|
locale |
string | ✅ | Locale to use for string comparison |
strength |
integer | ✅ | Strength — Min 1 — Max 4 |
trigger_window
| Field | Type | Required | Notes |
|---|---|---|---|
check_interval_ms |
integer | ✅ | Interval in milliseconds to check for ready state. — Min 10000 |
ttl |
integer | ✅ | Time to live in milliseconds. Used to compute an expiry. — Min 0 — Max 21600000 |
ready_field |
string | ✅ | Field to check for ready state. Accepts boolean and truthy values to indicate whether the source is ready for processing. |
query |
object | ✅ |
query
| Field | Type | Required | Notes |
|---|---|---|---|
format |
enum | ✅ | Choices: sql, json_sql, plugin_dsl |
query |
string | ✅ | Stringified query in the specified format |
destination¶
The destination object is a discriminated union. One of the following 3 shapes applies.
1. local_bucket_sync¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | local_bucket_sync |
bucket |
string | ✅ | Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567 |
path |
string | System file path — Pattern: ^[\/\a-zA-Z0-9_.\-: ]+$ — 0–256 characters — e.g. /file/path |
2. local_bucket_export¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | local_bucket_export |
bucket |
string | ✅ | Bucket id — Pattern: ^bkt-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. bkt-123e4567-e89b-41d2-a345-678901234567 |
path |
string | System file path — Pattern: ^[\/\a-zA-Z0-9_.\-: ]+$ — 0–256 characters — e.g. /file/path |
|
format |
enum | Format for query results. Max limit is applied if "json". — Choices: parquet, ldjson, json, csv, psv, tsv — Default: json |
|
max_parts |
integer | Maximum number of part files to export to. Mutually exclusive with max_part_size. — Min 1 | |
max_part_size |
string | Maximum part file size to export to. Mutually exclusive with max_parts. | |
schema_file |
string | Path to a file to export latest schema. Relative to location path. Only applicable if location points to directory. — e.g. /file/path with spaces |
|
header |
enum | Whether to include a header row. For DSV, "short" omits type indicator. — Choices: none, full, short — Default: full |
3. local_warehouse_ingest¶
| Field | Type | Required | Notes |
|---|---|---|---|
type |
const | ✅ | local_warehouse_ingest |
warehouse |
string | ✅ | Warehouse id — Pattern: ^whs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ — e.g. whs-123e4567-e89b-41d2-a345-678901234567 |
table |
string | ✅ | Unique id for a table within an ERM — Pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ — 2–64 characters |
mode |
enum | ✅ | Choices: overwrite, upsert, overwrite_cells, upsert_cells |
normalize_properties |
boolean | Whether to automatically normalize the incoming property ids to au compatible ids — Default: False |
error_handling¶
Options for error handling
| Field | Type | Required | Notes |
|---|---|---|---|
retry |
boolean | Whether to retry the task in the case of a failure — Default: False |
|
retries |
integer | The maximum amount of times to retry the operation — Min 0 — Default: 2 |
|
factor |
number | The exponential factor to use — Min 0 — Default: 2 |
|
minTimeout |
integer | The number of milliseconds before starting the first retry — Min 0 — Default: 10000 |
|
maxTimeout |
integer | The maximum number of milliseconds between two retries — Min 0 — Default: 180000 |
|
randomize |
boolean | Randomizes the timeouts by multiplying with a factor between 1 to 2 — Default: False |
Example Spec¶
id: siacqtrf-123e4567-e89b-41d2-a345-678901234567
source:
type: remote_bucket_ingest
bucket: sirsmbkt-123e4567-e89b-41d2-a345-678901234567
destination:
type: local_bucket_export
bucket: bkt-123e4567-e89b-41d2-a345-678901234567
name: <string>
description: <string>
au3 solution installation phase acquisition transfer run¶
Run transfer
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--id |
string | ✅ | Id of transfer to run | |
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
au3 solution installation phase acquisition transfer delete¶
Delete transfer
| Option | Type | Required | Default | Notes |
|---|---|---|---|---|
--id |
string | ✅ | transfer id | |
--fmt |
enum | human | Output format Choices: "human", "json", "yaml" | |
--debug |
boolean | false | Enable more verbose output on error |
Generated from au3 solution installation phase acquisition --help and subcommand help output.