MongoDB Database Clearing Process

Purpose

The MongoDB clearing process is implemented as a GitHub Action that runs before API deployments to develop and staging environments. This process serves several critical purposes:

1. Simulates Production Deployment: By clearing the database before each deployment, we simulate how our first release to production will work (with a clean database).

2. Validates Data Migrations: Ensures all initial data migrations run successfully during each environment deployment.

3. Prevents Data Contamination: Prevents old data structures from causing unexpected behavior after schema changes.

4. Consistent Testing Environments: Provides a consistent baseline for testing new features and migrations.

Workflow Design

The database clearing process is implemented as a reusable GitHub Action workflow that:

1. Is triggered automatically as part of the API CI/CD pipeline for develop and staging environments
2. Can also be triggered manually via workflow dispatch
3. Has appropriate safeguards to prevent running against production environments
4. Provides detailed logging and error reporting
5. Sends notifications upon completion

Configuration

Required Secrets

The workflow requires the following secrets to be configured in your GitHub repository:

• MONGO_CONNECTION_STRING: MongoDB connection string for our mongo cluster
• SLACK_WEBHOOK_URL: Webhook URL for Slack notifications

Trigger Configuration

The database clearing happens automatically as part of the API CI/CD pipeline through workflow calls:

1. For the develop branch, it clears the develop environment database
2. For the main branch, it clears the staging environment database

Security Considerations

1. Connection String Security: MongoDB connection strings are stored as GitHub secrets and never exposed in logs or outputs.

2. Environment Isolation: The workflow contains explicit checks to ensure it only runs against develop and staging environments.

3. Limited Database Operations: The script only deletes data from collections, preserving database structure and not affecting system collections.

4. Minimal Permissions: The MongoDB users specified in the connection strings should have only the necessary permissions for the environments they access.

5. Execution Logging: All actions are logged to enable audit trails and troubleshooting.

Maintenance Notes

After the initial production release, this workflow will be updated to:

1. Restore a sanitized copy of production data to staging environments instead of clearing the database
2. Maintain the database clearing for develop environments

Validation Process

The success of the database clearing and subsequent migrations is validated through:

1. GitHub Action logs showing successful collection clearing
2. API startup logs confirming successful migrations
3. Application functionality tests in the cleared environments

Troubleshooting

If the database clearing fails:

1. Check the GitHub Action logs for specific error messages
2. Verify the MongoDB connection strings are correct and the users have appropriate permissions
3. Ensure the MongoDB server is accessible from the GitHub Action runner
4. Check for any locks or active connections that might prevent collection clearing