Release Notes for mongosync 1.8

1.8.1 Release

October 10, 2024

New Features:

• Support added for OpenID Connect (OIDC) authentication with the workload (machine) flow. For more information, see mongosync Authentication with Atlas Workload Identity Federation.

Issues Fixed:

• Fixed bug introduced in mongosync 1.8.0 to accommodate a server issue where replica sets that fail to set the replicaSetId configuration can cause mongosync to crash.

Updates to mongosync /progress API endpoint

• Returns a top-level boolean success field.

• Returns a new totalEventsApplied field.

• Reports per-partition progress tracking

See the progress documentation for more details.

Other Notes

Optimizations:

• Fixes and enhancements to significantly reduce the time needed to create partitions and adds logging that tracks the partition creation process.

Other Changes:

• mongosync now performs a round robin assignment of the primary shard for each database if the destination is a sharded cluster.

• mongosync now errors if it cannot create an index after 10 attempts. Previously, the system would attempt to create an index indefinitely.

• mongosync now sends hostnames as telemetry.

• mongosync now exits with an error if it stops and restarts with a different source or destination cluster than previously specified.

• Improved performance of the initialization process by eliminating the creation of unnecessary dummy indexes. mongosync now creates dummy indexes for sharded collections only if the destination cluster does not have an index to support the shard key.

Issues Fixed:

• Fixed a bug introduced in v1.0.0 that caused mongosync to fall off the source cluster's oplog if no writes occurred on the source cluster for an extended period of time.

• Fixed a bug introduced in v1.0.0 that caused mongosync to write certain logs to a location outside of the specified log directory.

• Fixed a bug introduced in v1.0.0 that caused the mongosync /progress endpoint to potentially return incorrect totalBytes for sharded clusters.

A->B->C Migrations

Starting in mongosync beta 1.8, you can perform A->B->C migrations. A->B->C migrations allow you to perform two consecutive migrations, where the destination cluster of the first migration acts as the source cluster for the second migration.

To learn more, see A->B->C Migrations.

Many-to-One Migrations

Starting in mongosync beta 1.8, you can perform Many-to-One migrations. Many-to-One migrations allow you to sync multiple source clusters simultaneously with a destination cluster. For example, you can consolidate data from many small clusters into a central cluster.

To learn more, see Many-to-One Migrations.

Document Filtering

Starting in mongosync beta 1.8, you can selectively migrate documents based on specific conditions. To further limit which documents migrate to the destination cluster, you can combine document filtering and namespace filtering.

To learn more, see Document Filtering.

Handle Destination Data with destinationDataHandling Option

Starting in mongosync beta 1.8, use the destinationDataHandling option in the start request to define what happens if the destination cluster already has user data. Earlier mongosync versions return an error if the destination cluster has user data.

For details, see Handle Pre-Existing Data on the Destination.

Namespace Remapping

Starting in mongosync beta 1.8, you can remap database names during sync. This allows you to take data in one database on the source cluster and migrate it into a different database on the destination cluster.

For more information, see Namespace Remapping.

Oplog Rollover Resilience

Starting in mongosync beta 1.8, you can enable Oplog Rollover Resilience (ORR). With ORR, mongosync applies changes made on the source cluster to the destination cluster concurrently with initial sync. For source clusters with a high write rate, ORR significantly lowers the risk of oplog rollover during initial sync and reduces the need to restart the sync.

To learn more, see Oplog Rollover Resilience.