Google Cloud Storage Adapter
The Google Cloud Storage Adapter provides access to Google Cloud Storage buckets and objects in the Google Cloud Platform.
The adapter supports the following features:
- The Google Cloud Storage adapter provides access to Google Cloud Storage buckets and objects in the Google Cloud Platform.
- Discovering available buckets and objects with the option to filter objects based on the provided delimiter and prefix values.
- Authenticating the user using the specified service account credentials file, or automatically by the environment.
- Reading the specified object in a single chunk or in multiple chunks of the specified size.
- Keeping or deleting the specified object after reading and processing it successfully, as part of committing the map transaction.
- Reading the list of objects in the specified bucket, with optional filtering, for further processing the list in a map.
- Writing an object, with the option to fail or overwrite the target object if it exists already.
- Writing an object in a single chunk or in multiple chunks by configuring the map inputs to run in burst mode.
- Writing data to an intermediate (temporary) object and moving it to the target object as part of committing the map transaction.