The S3 connector allows you to create a Shaped model directly from a set of Parquet, CSV, or JSONL files within an S3 bucket.
The schema must match the schema of the user, item and interactions mapped within the Create Model call.
Shaped fetches data from the given S3 bucket periodically each time the model is trained. To ensure it’s trained on the most recent data, make sure you push the latest data to S3 periodically.
Shaped needs access to the S3 bucket that contains your files. This can be done by granting explicit read access to the Shaped AWS Customer Data Access Role.
To grant access:
Create an IAM Trust Policy attached to your S3 bucket that grants the following permissions:
In this trust policy, grant the Shaped AWS Customer Data Access Role IAM permissions to assume the role:
The details of shaped_aws_account_id are available on request.
Below are the fields required for the File connector
|type||File||Specifies the connector type, in this case “File”.|
|id||file||Specifies the connector id, in this case “file”.|
|path||s3://file-path/path-key||Specifies the S3 path for the connector source.|