Skip to main content

Cleanup and Reseed Plan

Overview​

Once all data models are reviewed and standardized, we can perform a complete database cleanup and reseed with the new standardized models. This is safe for staging environment.

Prerequisites Checklist​

Before running cleanup and reseed, verify:

✅ Data Model Review​

  • Cases - Base model created, all inconsistencies fixed
  • Users - Base model created, all inconsistencies fixed
  • Donations - Base model created, all inconsistencies fixed
  • Follows - Base model created, all inconsistencies fixed
  • Case Updates - Base model created, all inconsistencies fixed
  • Conversations - Base model created, all inconsistencies fixed
  • Support Tickets - Base model created, all inconsistencies fixed
  • Audit Logs - Skipped (backoffice infrastructure)

✅ Firestore Rules​

  • All collections have proper security rules
  • Field names match between rules and models
  • donorId → userId fixed in toto-app rules
  • requesterId → userId fixed in support tickets
  • socialMediaPosts rules added to toto-bo

✅ Seeding Scripts​

  • All seeding scripts updated to match base models
  • Field names consistent across all scripts
  • All required fields populated
  • Optional fields handled correctly

✅ Code Updates​

  • All interfaces updated to match base models
  • API routes updated with correct field names
  • Service files updated
  • UI components updated (where applicable)

Collections to Keep​

These collections will be preserved (not cleaned):

  1. socialMediaPosts - Created by social media bot, should be preserved

    • Reason: Generated by AI agents, not seeded
    • Action: Skip during cleanup

  2. guardians - Twitter bot guardian management

    • Reason: Managed by Twitter bot, not seeded
    • Action: Skip during cleanup

  3. guardianTweets - Guardian tweets

    • Reason: Generated by Twitter bot, not seeded
    • Action: Skip during cleanup

  4. twitterBotConfig - Twitter bot configuration

    • Reason: Configuration data, not seeded
    • Action: Skip during cleanup

Collections to Clean and Reseed​

These collections will be completely cleaned and reseeded:

  1. users - User accounts and profiles
  2. cases - Pet rescue cases
  3. donations - Donation records
  4. follows - User subscriptions
  5. updates - Case updates (top-level)
  6. support_tickets - Support tickets
  7. notifications - User notifications
  8. conversations - AI chat conversations
  9. audit_logs - System audit trail

Collections to Remove (Unused)​

These collections appear to be unused and can be removed:

  1. waitlist - External waitlist (now in users collection)
  2. chats - Replaced by conversations
  3. analytics - Check if used in Cloud Functions
  4. analytics_cache - Check if used in Cloud Functions
  5. accounts - Check if used for account management
  6. sessions - Check if used for session management
  7. verification_tokens - Check if used for email verification
  8. collaborators - Check if used for collaboration features

⚠️ Action Required: Before removing, verify these are not used in:

  • Cloud Functions
  • External services
  • Legacy code paths

Cleanup Script Modifications​

Update init-staging-complete/route.ts​

Modify the cleanup section to:

  1. Skip collections to preserve:
const collectionsToPreserve = [
  'socialMediaPosts',
  'guardians',
  'guardianTweets',
  'twitterBotConfig'
];
  1. Collections to clean and reseed:
const collectionsToReseed = [
  'users',
  'cases',
  'updates',
  'donations',
  'follows',
  'notifications',
  'support_tickets',
  'conversations',
  'audit_logs'
];
  1. Collections to remove (after verification):
const collectionsToRemove = [
  'waitlist',
  'chats',
  'analytics',
  'analytics_cache',
  'accounts',
  'sessions',
  'verification_tokens',
  'collaborators'
];

Execution Steps​

Step 1: Final Review​

  • Review all base models one final time
  • Verify all Firestore rules are correct
  • Check all seeding scripts
  • Verify no breaking changes in API routes
  • Export current database state (if needed for reference)
  • Document any custom data that needs to be preserved

Step 3: Update Cleanup Script​

  • Add collections to preserve list
  • Add collections to remove list (after verification)
  • Test cleanup logic

Step 4: Run Cleanup and Reseed​

  • Execute POST /api/init-staging-complete
  • Monitor logs for errors
  • Verify all collections are seeded correctly

Step 5: Verification​

  • Check all collections have data
  • Verify field names match base models
  • Test UI displays data correctly
  • Test API routes work correctly
  • Verify Firestore rules work correctly

Post-Cleanup Checklist​

After cleanup and reseed:

  • All entities display correctly in toto-app UI
  • All entities display correctly in toto-bo UI
  • API routes return correct data structure
  • Firestore rules allow/deny correctly
  • No console errors in browser/terminal
  • Social media bot can still create posts
  • Twitter bot can still access guardians

Rollback Plan​

If something goes wrong:

  1. Stop the process immediately
  2. Check logs for specific errors
  3. Fix the issue in seeding script or model
  4. Re-run cleanup (it's idempotent)

Notes​

  • Staging Only: This cleanup is for staging environment
  • Production: For production, create a migration script instead of full cleanup
  • Social Media Posts: These are generated by bots, not seeded, so they're preserved
  • Audit Logs: Will be reseeded but will continue to be generated by system

Timeline​

  1. Final Review: 1-2 hours
  2. Script Updates: 30 minutes
  3. Cleanup Execution: 5-10 minutes
  4. Verification: 30 minutes

Total Estimated Time: 2-3 hours