Back up to Amazon Simple Storage Service using ON-Bar and the PSM

You can use ON-Bar and the Primary Storage Manager to back up and restore data to or from the Amazon Simple Storage Service (S3). You are responsible for terms and any charges associated with your use of the Amazon Simple Storage Service.

Before you begin

Prerequisites:

  • You must have an Amazon account to perform cloud storage backups. See the Amazon website for instructions about setting up an account.

About this task

The following steps show how to back up data to the Amazon Simple Storage Service (S3) System and restore from it by using ON-Bar and the PSM. In this context, cloud storage refers to an online storage service over the Internet. If you choose to back up to cloud storage, you do not need to provide local devices. Instead, you back up the data to a virtual device, most likely located on the Internet.

Procedure

  1. Create a group to access S3
    1. Using a web browser, navigate to the Amazon S3 website and log on.
    2. Select Groups tab and click Create New Group.
    3. Specify the Group name and click Next Step.
    4. In the Attach Policy page, attach a policy to the group. In this case we will select AmazonS3FullAccess . As the name implies, this policy will allow any member of this group to do anything in all the containers in S3. Click Next Step.
      Note:

      If you want to use S3 for other purposes or for multiple instances, you can change this by going in to the Policies tab before creating the group and creating a customized policy that suits your needs. Ensure to be careful before granting full access to the Amazon S3, as it will provide the user to access all the S3 data in the account .

    5. In the Review page, review your entries and click Create Group.
    The new group is created.
  2. Create a User to Access S3
    1. Select Users tab and click Create New Users.
    2. Specify the User name and select Generate and access key for each user.
    3. Click Create.

      Access Key and the Secret Access Key are generated. These keys are equivalent to username and password that can be used to store and retrieve data from S3 using APIs.

      Note: Only authorized user can access these keys. You need to download these credentials into a text file and store in a safe location. If you lose them, you need to create a new user again.
    4. Click Close.

      User page with newly created user will be displayed.

  3. Assign the user to the group.
    1. Click the check box next to the user and select User Actions.
    2. Click Add User to Groups.
  4. Create a bucket.
    1. Select Services tab and click Amazon S3.
    2. Click Create Bucket.
    3. Specify a bucket name and select the appropriate region.
  5. Use the onkstore utility to create a keystore of type "aws-bar" with your AWS credentials
  6. Configure Primary Storage Manager to use the Bucket and Credentials you just created.
    Create a device in PSM of type CLOUD using S3 as provider.
    onpsm -D add https://ifmx-s3-dev.s3.amazonaws.com \
-g DBSPOOL \
-p HIGHEST \
-t CLOUD \
--creds /home/onedb/keystores/aws-bar.p12  \
-max_part_size 25600
    In this command line:
    1. The URL must be provided in Virtual-Hosted style and not in Path style. This allows us also to use prefixes in the URL, Therefore you can use the same container for several purposes and you can choose to store your onedb data in the "onedb" prefix. For example "https://ifmx-s3-dev.s3.amazonaws.com/onedb".
    2. ‘-t CLOUD’, is the device type that enable PSM to store/retrieve the data to/from a CLOUD infrastructure.
    3. --creds is the full path of the keystore containing the credentials to access the S3 object storage.
    4. —-max_part_size 25600 will fragment your objects in 25MB pieces, in the case of S3, size between 25 and 100 MB is recommended.
  7. Check the created device: 
    $ onpsm -D list                                                                                                                                                      

OneDB Primary Storage Manager Devices List

Type    Prio      Pool     Provider Name                        Keystore
CLOUD   HIGH      DBSPOOL  https://ifmx-s3-dev.s3.amazonaws.com /home/onedb/keystores/aws-bar.p12
CLOUD   HIGH      LOGPOOL  https://ifmx-s3-dev.s3.amazonaws.com /home/onedb/keystores/aws-bar.p12
  8. Take a Level Zero Backup: 
    $ onbar -b -L 0 -w
$ echo $?
0
  9. Check Backup Data in your Bucket using S3 management console.