The S3 connector allows you to create a Shaped model directly from a Parquet file within an S3 bucket. The Parquet files schema must match the schema of the user, item and interactions mapped within the Create Model call.

Shaped fetches the given Parquet file periodically each time the model is trained. To ensure it’s trained on the most recent data, make sure you push the latest data to S3 periodically.

Cross account access

Shaped needs access to the S3 bucket that contains your Parquet files. This can be done by making it public or by granting explicit read access to the Shaped AWS IAM. To grant access:

  1. Create an IAM role that is granted permission to read from the S3 bucket
  2. In this role’s trust policy, grant the Shaped AWS IAM permissions to assume the role:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "<shaped_aws_account_read_iam>"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

This can be created from the AWS UI or using the AWS CLI with the create-role command.

The details of shaped_aws_account_read_iam are available on request.

Connector Config Definition

Below are the fields required for the File connection_config

"connector_configs": [{
    "type": "File",
    "id": "file",
}]
FieldExampleDescription
type“File”Specifies the connector type, in this case “File”.
id“file”Specifies the connector id, in this case “file”.