transfer_configs

Creates, updates, deletes, gets or lists a transfer_configs resource.

Overview

Name	`transfer_configs`
Type	Resource
Id	`google.bigquerydatatransfer.transfer_configs`

Fields

Name	Datatype	Description
`name`	`string`	Identifier. The resource name of the transfer config. Transfer config names have the form either `projects/{project_id}/locations/{region}/transferConfigs/{config_id}` or `projects/{project_id}/transferConfigs/{config_id}`, where `config_id` is usually a UUID, even though it is not guaranteed or required. The name is ignored when creating a transfer config.
`dataRefreshWindowDays`	`integer`	The number of days to look back to automatically refresh the data. For example, if `data_refresh_window_days = 10`, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.
`dataSourceId`	`string`	Data source ID. This cannot be changed once data transfer is created. The full list of available data source IDs can be returned through an API call: https://cloud.google.com/bigquery-transfer/docs/reference/datatransfer/rest/v1/projects.locations.dataSources/list
`datasetRegion`	`string`	Output only. Region in which BigQuery dataset is located.
`destinationDatasetId`	`string`	The BigQuery target dataset id.
`disabled`	`boolean`	Is this config disabled. When set to true, no runs will be scheduled for this transfer config.
`displayName`	`string`	User specified display name for the data transfer.
`emailPreferences`	`object`	Represents preferences for sending email notifications for transfer run events.
`encryptionConfiguration`	`object`	Represents the encryption configuration for a transfer.
`nextRunTime`	`string`	Output only. Next time when data transfer will run.
`notificationPubsubTopic`	`string`	Pub/Sub topic where notifications will be sent after transfer runs associated with this transfer config finish. The format for specifying a pubsub topic is: `projects/{project_id}/topics/{topic_id}`
`ownerInfo`	`object`	Information about a user.
`params`	`object`	Parameters specific to each data source. For more information see the bq tab in the 'Setting up a data transfer' section for each data source. For example the parameters for Cloud Storage transfers are listed here: https://cloud.google.com/bigquery-transfer/docs/cloud-storage-transfer#bq
`schedule`	`string`	Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: `1st,3rd monday of month 15:30`, `every wed,fri of jan,jun 13:15`, and `first sunday of quarter 00:00`. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: The minimum interval time between recurring transfers depends on the data source; refer to the documentation for your data source.
`scheduleOptions`	`object`	Options customizing the data transfer schedule.
`state`	`string`	Output only. State of the most recently updated transfer run.
`updateTime`	`string`	Output only. Data transfer modification time. Ignored by server on input.
`userId`	`string`	Deprecated. Unique ID of the user on whose behalf transfer is done.

Methods

Name	Accessible by	Required Params	Description
`projects_locations_transfer_configs_get`	`SELECT`	`locationsId, projectsId, transferConfigsId`	Returns information about a data transfer config.
`projects_locations_transfer_configs_list`	`SELECT`	`locationsId, projectsId`	Returns information about all transfer configs owned by a project in the specified location.
`projects_transfer_configs_get`	`SELECT`	`projectsId, transferConfigsId`	Returns information about a data transfer config.
`projects_transfer_configs_list`	`SELECT`	`projectsId`	Returns information about all transfer configs owned by a project in the specified location.
`projects_locations_transfer_configs_create`	`INSERT`	`locationsId, projectsId`	Creates a new data transfer configuration.
`projects_transfer_configs_create`	`INSERT`	`projectsId`	Creates a new data transfer configuration.
`projects_locations_transfer_configs_delete`	`DELETE`	`locationsId, projectsId, transferConfigsId`	Deletes a data transfer configuration, including any associated transfer runs and logs.
`projects_transfer_configs_delete`	`DELETE`	`projectsId, transferConfigsId`	Deletes a data transfer configuration, including any associated transfer runs and logs.
`projects_locations_transfer_configs_patch`	`UPDATE`	`locationsId, projectsId, transferConfigsId`	Updates a data transfer configuration. All fields must be set, even if they are not updated.
`projects_transfer_configs_patch`	`UPDATE`	`projectsId, transferConfigsId`	Updates a data transfer configuration. All fields must be set, even if they are not updated.
`projects_locations_transfer_configs_schedule_runs`	`EXEC`	`locationsId, projectsId, transferConfigsId`	Creates transfer runs for a time range [start_time, end_time]. For each date - or whatever granularity the data source supports - in the range, one transfer run is created. Note that runs are created per UTC time in the time range. DEPRECATED: use StartManualTransferRuns instead.
`projects_locations_transfer_configs_start_manual_runs`	`EXEC`	`locationsId, projectsId, transferConfigsId`	Start manual transfer runs to be executed now with schedule_time equal to current time. The transfer runs can be created for a time range where the run_time is between start_time (inclusive) and end_time (exclusive), or for a specific run_time.
`projects_transfer_configs_schedule_runs`	`EXEC`	`projectsId, transferConfigsId`	Creates transfer runs for a time range [start_time, end_time]. For each date - or whatever granularity the data source supports - in the range, one transfer run is created. Note that runs are created per UTC time in the time range. DEPRECATED: use StartManualTransferRuns instead.
`projects_transfer_configs_start_manual_runs`	`EXEC`	`projectsId, transferConfigsId`	Start manual transfer runs to be executed now with schedule_time equal to current time. The transfer runs can be created for a time range where the run_time is between start_time (inclusive) and end_time (exclusive), or for a specific run_time.

`SELECT` examples

Returns information about all transfer configs owned by a project in the specified location.

SELECT
name,
dataRefreshWindowDays,
dataSourceId,
datasetRegion,
destinationDatasetId,
disabled,
displayName,
emailPreferences,
encryptionConfiguration,
nextRunTime,
notificationPubsubTopic,
ownerInfo,
params,
schedule,
scheduleOptions,
state,
updateTime,
userId
FROM google.bigquerydatatransfer.transfer_configs
WHERE projectsId = '{{ projectsId }}';

`INSERT` example

Use the following StackQL query and manifest file to create a new transfer_configs resource.

All Properties
Manifest

/*+ create */
INSERT INTO google.bigquerydatatransfer.transfer_configs (
projectsId,
name,
destinationDatasetId,
displayName,
dataSourceId,
params,
schedule,
scheduleOptions,
dataRefreshWindowDays,
disabled,
userId,
notificationPubsubTopic,
emailPreferences,
encryptionConfiguration
)
SELECT 
'{{ projectsId }}',
'{{ name }}',
'{{ destinationDatasetId }}',
'{{ displayName }}',
'{{ dataSourceId }}',
'{{ params }}',
'{{ schedule }}',
'{{ scheduleOptions }}',
'{{ dataRefreshWindowDays }}',
{{ disabled }},
'{{ userId }}',
'{{ notificationPubsubTopic }}',
'{{ emailPreferences }}',
'{{ encryptionConfiguration }}'
;

- name: your_resource_model_name
  props:
    - name: name
      value: string
    - name: destinationDatasetId
      value: string
    - name: displayName
      value: string
    - name: dataSourceId
      value: string
    - name: params
      value: object
    - name: schedule
      value: string
    - name: scheduleOptions
      value:
        - name: disableAutoScheduling
          value: boolean
        - name: startTime
          value: string
        - name: endTime
          value: string
    - name: dataRefreshWindowDays
      value: integer
    - name: disabled
      value: boolean
    - name: updateTime
      value: string
    - name: nextRunTime
      value: string
    - name: state
      value: string
    - name: userId
      value: string
    - name: datasetRegion
      value: string
    - name: notificationPubsubTopic
      value: string
    - name: emailPreferences
      value:
        - name: enableFailureEmail
          value: boolean
    - name: ownerInfo
      value:
        - name: email
          value: string
    - name: encryptionConfiguration
      value:
        - name: kmsKeyName
          value: string

`UPDATE` example

Updates a transfer_configs resource.

/*+ update */
UPDATE google.bigquerydatatransfer.transfer_configs
SET 
name = '{{ name }}',
destinationDatasetId = '{{ destinationDatasetId }}',
displayName = '{{ displayName }}',
dataSourceId = '{{ dataSourceId }}',
params = '{{ params }}',
schedule = '{{ schedule }}',
scheduleOptions = '{{ scheduleOptions }}',
dataRefreshWindowDays = '{{ dataRefreshWindowDays }}',
disabled = true|false,
userId = '{{ userId }}',
notificationPubsubTopic = '{{ notificationPubsubTopic }}',
emailPreferences = '{{ emailPreferences }}',
encryptionConfiguration = '{{ encryptionConfiguration }}'
WHERE 
projectsId = '{{ projectsId }}'
AND transferConfigsId = '{{ transferConfigsId }}';

`DELETE` example

Deletes the specified transfer_configs resource.

/*+ delete */
DELETE FROM google.bigquerydatatransfer.transfer_configs
WHERE projectsId = '{{ projectsId }}'
AND transferConfigsId = '{{ transferConfigsId }}';

Overview​

Fields​

Methods​

SELECT examples​

INSERT example​

UPDATE example​

DELETE example​