MongoDB

Preparation

To allow Shaped to connect to your MongoDB database, you need to create a read-only user and share those credentials through the Create Dataset endpoint. You can create a read-only user on a MongoDB database instance with the following commands:

# 1. Connect to MongoDB instance and switch to the database.
mongo --host <host> --port <port> --username <username> --password <password>
use <database_name>

# 2. Create a user with read-only access to the database.
db.createUser({
  user: "read_only_user",
  pwd: "<password>",
  roles: [{ role: "read", db: "<database_name>" }]
});

# 3. Grant Access to Specific Collection
db.createRole({
  role: "read_only_collection_role",
  privileges: [
    {
      resource: {
        db: "<database_name>",
        collection: "<collection_name>"
      },
      actions: ["find"]
    }
  ],
  roles: []
});

In MongoDB, read-only access at the database level will automatically grant read access to all collections in the database. If you want to restrict access to a specific collection, you will need to create a custom role.

db.grantRolesToUser("read_only_user", ["read_only_collection_role"]);

Replace host, port, username, password, database_name, and collection_name with the appropriate values for your MongoDB account.

Dataset Configuration

Below are the fields required for the MongoDB dataset connector:

Field	Example	Description
schema_type	MONGODB	Specifies the connector schema type, in this case "MongoDB".
config.user	your_user	Access account username.
config.password	pAssw0rd1!	Access account password.
config.collection	movies	The name of the MongoDB collection to sync.
config.database	movielens	The name of the database that contains the collection to sync.
config.start_date	2023-01-01T00:00:00	Optional. The start for which to start filtering records from, helpful for large collections.
config.replication_key	updated_at	Optional. The name of the column that contains a datetime key or ascending id for ordering data during incremental syncs.

Dataset Creation Example

Below is an example of a MongoDB dataset connector configuration:

dataset_name: mongodb_dataset
schema_type: MONGODB
config:
    user: your_user
    password: pAssw0rd1!
    collection: movies
    database: movielens
    replication_key: updated_at

The following payload will create a MongoDB dataset and begin syncing data from Shaped using the Shaped CLI.

shaped create-dataset --file dataset.yaml