data_scans

Creates, updates, deletes, gets or lists a data_scans resource.

Overview

Name	data_scans
Type	Resource
Id	google.dataplex.data_scans

Fields

Name	Datatype	Description
name	string	Output only. The relative resource name of the scan, of the form: projects/{project}/locations/{location_id}/dataScans/{datascan_id}, where project refers to a project_id or project_number and location_id refers to a GCP region.
description	string	Optional. Description of the scan. Must be between 1-1024 characters.
createTime	string	Output only. The time when the scan was created.
data	object	The data source for DataScan.
dataProfileResult	object	DataProfileResult defines the output of DataProfileScan. Each field of the table will have field type specific profile result.
dataProfileSpec	object	DataProfileScan related setting.
dataQualityResult	object	The output of a DataQualityScan.
dataQualitySpec	object	DataQualityScan related setting.
displayName	string	Optional. User friendly display name. Must be between 1-256 characters.
executionSpec	object	DataScan execution settings.
executionStatus	object	Status of the data scan execution.
labels	object	Optional. User-defined labels for the scan.
state	string	Output only. Current state of the DataScan.
type	string	Output only. The type of DataScan.
uid	string	Output only. System generated globally unique ID for the scan. This ID will be different if the scan is deleted and re-created with the same name.
updateTime	string	Output only. The time when the scan was last updated.

Methods

Name	Accessible by	Required Params	Description
projects_locations_data_scans_get	SELECT	dataScansId, locationsId, projectsId	Gets a DataScan resource.
projects_locations_data_scans_list	SELECT	locationsId, projectsId	Lists DataScans.
projects_locations_data_scans_create	INSERT	locationsId, projectsId	Creates a DataScan resource.
projects_locations_data_scans_delete	DELETE	dataScansId, locationsId, projectsId	Deletes a DataScan resource.
projects_locations_data_scans_patch	UPDATE	dataScansId, locationsId, projectsId	Updates a DataScan resource.
projects_locations_data_scans_generate_data_quality_rules	EXEC	dataScansId, locationsId, projectsId	Generates recommended data quality rules based on the results of a data profiling scan.Use the recommendations to build rules for a data quality scan.
projects_locations_data_scans_run	EXEC	dataScansId, locationsId, projectsId	Runs an on-demand execution of a DataScan

SELECT examples

Lists DataScans.

SELECT
name,
description,
createTime,
data,
dataProfileResult,
dataProfileSpec,
dataQualityResult,
dataQualitySpec,
displayName,
executionSpec,
executionStatus,
labels,
state,
type,
uid,
updateTime
FROM google.dataplex.data_scans
WHERE locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';

INSERT example

Use the following StackQL query and manifest file to create a new data_scans resource.

All Properties

/*+ create */
INSERT INTO google.dataplex.data_scans (
locationsId,
projectsId,
description,
displayName,
labels,
data,
executionSpec,
dataQualitySpec,
dataProfileSpec
)
SELECT 
'{{ locationsId }}',
'{{ projectsId }}',
'{{ description }}',
'{{ displayName }}',
'{{ labels }}',
'{{ data }}',
'{{ executionSpec }}',
'{{ dataQualitySpec }}',
'{{ dataProfileSpec }}'
;

Manifest

- name: your_resource_model_name
  props:
    - name: name
      value: string
    - name: uid
      value: string
    - name: description
      value: string
    - name: displayName
      value: string
    - name: labels
      value: object
    - name: state
      value: string
    - name: createTime
      value: string
    - name: updateTime
      value: string
    - name: data
      value:
        - name: entity
          value: string
        - name: resource
          value: string
    - name: executionSpec
      value:
        - name: trigger
          value:
            - name: onDemand
              value: []
            - name: schedule
              value:
                - name: cron
                  value: string
        - name: field
          value: string
    - name: executionStatus
      value:
        - name: latestJobStartTime
          value: string
        - name: latestJobEndTime
          value: string
        - name: latestJobCreateTime
          value: string
    - name: type
      value: string
    - name: dataQualitySpec
      value:
        - name: rules
          value:
            - - name: rangeExpectation
                value:
                  - name: minValue
                    value: string
                  - name: maxValue
                    value: string
                  - name: strictMinEnabled
                    value: boolean
                  - name: strictMaxEnabled
                    value: boolean
              - name: nonNullExpectation
                value: []
              - name: setExpectation
                value:
                  - name: values
                    value:
                      - string
              - name: regexExpectation
                value:
                  - name: regex
                    value: string
              - name: uniquenessExpectation
                value: []
              - name: statisticRangeExpectation
                value:
                  - name: statistic
                    value: string
                  - name: minValue
                    value: string
                  - name: maxValue
                    value: string
                  - name: strictMinEnabled
                    value: boolean
                  - name: strictMaxEnabled
                    value: boolean
              - name: rowConditionExpectation
                value:
                  - name: sqlExpression
                    value: string
              - name: tableConditionExpectation
                value:
                  - name: sqlExpression
                    value: string
              - name: sqlAssertion
                value:
                  - name: sqlStatement
                    value: string
              - name: column
                value: string
              - name: ignoreNull
                value: boolean
              - name: dimension
                value: string
              - name: threshold
                value: number
              - name: name
                value: string
              - name: description
                value: string
              - name: suspended
                value: boolean
        - name: samplingPercent
          value: number
        - name: rowFilter
          value: string
        - name: postScanActions
          value:
            - name: bigqueryExport
              value:
                - name: resultsTable
                  value: string
            - name: notificationReport
              value:
                - name: recipients
                  value:
                    - name: emails
                      value:
                        - string
                - name: scoreThresholdTrigger
                  value:
                    - name: scoreThreshold
                      value: number
                - name: jobFailureTrigger
                  value: []
                - name: jobEndTrigger
                  value: []
    - name: dataProfileSpec
      value:
        - name: samplingPercent
          value: number
        - name: rowFilter
          value: string
        - name: postScanActions
          value:
            - name: bigqueryExport
              value:
                - name: resultsTable
                  value: string
        - name: includeFields
          value:
            - name: fieldNames
              value:
                - string
    - name: dataQualityResult
      value:
        - name: passed
          value: boolean
        - name: score
          value: number
        - name: dimensions
          value:
            - - name: dimension
                value:
                  - name: name
                    value: string
              - name: passed
                value: boolean
              - name: score
                value: number
        - name: columns
          value:
            - - name: column
                value: string
              - name: score
                value: number
        - name: rules
          value:
            - - name: rule
                value:
                  - name: column
                    value: string
                  - name: ignoreNull
                    value: boolean
                  - name: dimension
                    value: string
                  - name: threshold
                    value: number
                  - name: name
                    value: string
                  - name: description
                    value: string
                  - name: suspended
                    value: boolean
              - name: passed
                value: boolean
              - name: evaluatedCount
                value: string
              - name: passedCount
                value: string
              - name: nullCount
                value: string
              - name: passRatio
                value: number
              - name: failingRowsQuery
                value: string
              - name: assertionRowCount
                value: string
        - name: rowCount
          value: string
        - name: scannedData
          value:
            - name: incrementalField
              value:
                - name: field
                  value: string
                - name: start
                  value: string
                - name: end
                  value: string
        - name: postScanActionsResult
          value:
            - name: bigqueryExportResult
              value:
                - name: state
                  value: string
                - name: message
                  value: string
    - name: dataProfileResult
      value:
        - name: rowCount
          value: string
        - name: profile
          value:
            - name: fields
              value:
                - - name: name
                    value: string
                  - name: type
                    value: string
                  - name: mode
                    value: string
                  - name: profile
                    value:
                      - name: nullRatio
                        value: number
                      - name: distinctRatio
                        value: number
                      - name: topNValues
                        value:
                          - - name: value
                              value: string
                            - name: count
                              value: string
                            - name: ratio
                              value: number
                      - name: stringProfile
                        value:
                          - name: minLength
                            value: string
                          - name: maxLength
                            value: string
                          - name: averageLength
                            value: number
                      - name: integerProfile
                        value:
                          - name: average
                            value: number
                          - name: standardDeviation
                            value: number
                          - name: min
                            value: string
                          - name: quartiles
                            value:
                              - string
                          - name: max
                            value: string
                      - name: doubleProfile
                        value:
                          - name: average
                            value: number
                          - name: standardDeviation
                            value: number
                          - name: min
                            value: number
                          - name: quartiles
                            value:
                              - number
                          - name: max
                            value: number
        - name: postScanActionsResult
          value:
            - name: bigqueryExportResult
              value:
                - name: state
                  value: string
                - name: message
                  value: string

UPDATE example

Updates a data_scans resource.

/*+ update */
UPDATE google.dataplex.data_scans
SET 
description = '{{ description }}',
displayName = '{{ displayName }}',
labels = '{{ labels }}',
data = '{{ data }}',
executionSpec = '{{ executionSpec }}',
dataQualitySpec = '{{ dataQualitySpec }}',
dataProfileSpec = '{{ dataProfileSpec }}'
WHERE 
dataScansId = '{{ dataScansId }}'
AND locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';

DELETE example

Deletes the specified data_scans resource.

/*+ delete */
DELETE FROM google.dataplex.data_scans
WHERE dataScansId = '{{ dataScansId }}'
AND locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';