datasets

Creates, updates, deletes, gets or lists a datasets resource.

Overview

Name	datasets
Type	Resource
Id	google.aiplatform.datasets

Fields

Name	Datatype	Description
name	string	Output only. Identifier. The resource name of the Dataset.
description	string	The description of the Dataset.
createTime	string	Output only. Timestamp when this Dataset was created.
dataItemCount	string	Output only. The number of DataItems in this Dataset. Only apply for non-structured Dataset.
displayName	string	Required. The user-defined name of the Dataset. The name can be up to 128 characters long and can consist of any UTF-8 characters.
encryptionSpec	object	Represents a customer-managed encryption key spec that can be applied to a top-level resource.
etag	string	Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
labels	object	The labels with user-defined metadata to organize your Datasets. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Dataset (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with "aiplatform.googleapis.com/" and are immutable. Following system labels exist for each Dataset: * "aiplatform.googleapis.com/dataset_metadata_schema": output only, its value is the metadata_schema's title.
metadata	any	Required. Additional information about the Dataset.
metadataArtifact	string	Output only. The resource name of the Artifact that was created in MetadataStore when creating the Dataset. The Artifact resource name pattern is projects/{project}/locations/{location}/metadataStores/{metadata_store}/artifacts/{artifact}.
metadataSchemaUri	string	Required. Points to a YAML file stored on Google Cloud Storage describing additional information about the Dataset. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/metadata/.
modelReference	string	Optional. Reference to the public base model last used by the dataset. Only set for prompt datasets.
satisfiesPzi	boolean	Output only. Reserved for future use.
satisfiesPzs	boolean	Output only. Reserved for future use.
savedQueries	array	All SavedQueries belong to the Dataset will be returned in List/Get Dataset response. The annotation_specs field will not be populated except for UI cases which will only use annotation_spec_count. In CreateDataset request, a SavedQuery is created together if this field is set, up to one SavedQuery can be set in CreateDatasetRequest. The SavedQuery should not contain any AnnotationSpec.
updateTime	string	Output only. Timestamp when this Dataset was last updated.

Methods

Name	Accessible by	Required Params	Description
get	SELECT	datasetsId, locationsId, projectsId	Gets a Dataset.
list	SELECT	locationsId, projectsId	Lists Datasets in a Location.
create	INSERT	locationsId, projectsId	Creates a Dataset.
delete	DELETE	datasetsId, locationsId, projectsId	Deletes a Dataset.
patch	UPDATE	datasetsId, locationsId, projectsId	Updates a Dataset.
export	EXEC	datasetsId, locationsId, projectsId	Exports data from a Dataset.
import	EXEC	datasetsId, locationsId, projectsId	Imports data into a Dataset.
search_data_items	EXEC	datasetsId, locationsId, projectsId	Searches DataItems in a Dataset.

SELECT examples

Lists Datasets in a Location.

SELECT
name,
description,
createTime,
dataItemCount,
displayName,
encryptionSpec,
etag,
labels,
metadata,
metadataArtifact,
metadataSchemaUri,
modelReference,
satisfiesPzi,
satisfiesPzs,
savedQueries,
updateTime
FROM google.aiplatform.datasets
WHERE locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';

INSERT example

Use the following StackQL query and manifest file to create a new datasets resource.

All Properties

/*+ create */
INSERT INTO google.aiplatform.datasets (
locationsId,
projectsId,
displayName,
metadataSchemaUri,
encryptionSpec,
description,
metadata,
modelReference,
savedQueries,
labels,
etag
)
SELECT 
'{{ locationsId }}',
'{{ projectsId }}',
'{{ displayName }}',
'{{ metadataSchemaUri }}',
'{{ encryptionSpec }}',
'{{ description }}',
'{{ metadata }}',
'{{ modelReference }}',
'{{ savedQueries }}',
'{{ labels }}',
'{{ etag }}'
;

Manifest

- name: your_resource_model_name
  props:
    - name: dataItemCount
      value: string
    - name: displayName
      value: string
    - name: updateTime
      value: string
    - name: metadataSchemaUri
      value: string
    - name: encryptionSpec
      value:
        - name: kmsKeyName
          value: string
    - name: description
      value: string
    - name: name
      value: string
    - name: metadata
      value: any
    - name: modelReference
      value: string
    - name: savedQueries
      value:
        - - name: createTime
            value: string
          - name: etag
            value: string
          - name: metadata
            value: any
          - name: problemType
            value: string
          - name: displayName
            value: string
          - name: name
            value: string
          - name: supportAutomlTraining
            value: boolean
          - name: annotationFilter
            value: string
          - name: annotationSpecCount
            value: integer
          - name: updateTime
            value: string
    - name: labels
      value: object
    - name: createTime
      value: string
    - name: satisfiesPzi
      value: boolean
    - name: etag
      value: string
    - name: metadataArtifact
      value: string
    - name: satisfiesPzs
      value: boolean

UPDATE example

Updates a datasets resource.

/*+ update */
UPDATE google.aiplatform.datasets
SET 
displayName = '{{ displayName }}',
metadataSchemaUri = '{{ metadataSchemaUri }}',
encryptionSpec = '{{ encryptionSpec }}',
description = '{{ description }}',
metadata = '{{ metadata }}',
modelReference = '{{ modelReference }}',
savedQueries = '{{ savedQueries }}',
labels = '{{ labels }}',
etag = '{{ etag }}'
WHERE 
datasetsId = '{{ datasetsId }}'
AND locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';

DELETE example

Deletes the specified datasets resource.

/*+ delete */
DELETE FROM google.aiplatform.datasets
WHERE datasetsId = '{{ datasetsId }}'
AND locationsId = '{{ locationsId }}'
AND projectsId = '{{ projectsId }}';