René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Sync Model it already exists (duplicate model sync owing to multiple parent dependencies)","articleBody":"### Environment\r\n* DiffSync version:  2.0.0\r\n* Python version 3.12\r\n\r\n\u003c!-- What happened instead? --\u003e\r\n### Observed Behavior\r\n\r\nWhen launch `sync_from` function for a dataset with multiple model dependencies, in this example `Regions` parent of `Sites`. `sync_from` by default try to create Sites (child object) several times for each dependencies as you can see below.\r\n\r\n```bash\r\n2024-07-05 16:59:05,401 - DEBUG - 2024-07-05 14:59.05 [debug    ] Diff calculation between these two datasets will involve 17 models [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e src=\u003cNetboxAdapter \"netbox\"\u003e\r\n2024-07-05 16:59:05,401 - INFO - 2024-07-05 14:59.05 [info     ] Beginning diff calculation     [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e src=\u003cNetboxAdapter \"netbox\"\u003e\r\n2024-07-05 16:59:05,402 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Seclin\r\n2024-07-05 16:59:05,402 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Vendome\r\n2024-07-05 16:59:05,402 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Rennes\r\n2024-07-05 16:59:05,404 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=SCATEST\r\n2024-07-05 16:59:05,404 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=SCLTEST\r\n2024-07-05 16:59:05,405 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Vendome\r\n2024-07-05 16:59:05,405 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Belgium\r\n2024-07-05 16:59:05,405 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=brussels\r\n2024-07-05 16:59:05,406 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=SCATEST\r\n2024-07-05 16:59:05,406 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=SCLTEST\r\n2024-07-05 16:59:05,407 - DEBUG - 2024-07-05 14:59.05 [debug    ] Skipping due to SKIP_UNMATCHED_DST flag on source adapter [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=VDMTEST\r\n2024-07-05 16:59:05,407 - INFO - 2024-07-05 14:59.05 [info     ] Diff calculation complete      [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e src=\u003cNetboxAdapter \"netbox\"\u003e\r\n2024-07-05 16:59:05,408 - INFO - 2024-07-05 14:59.05 [info     ] Beginning sync                 [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e src=\u003cNetboxAdapter \"netbox\"\u003e\r\n2024-07-05 16:59:05,408 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model update        [diffsync.helpers] action=update diffs={'-': {'slug': 'courbevoie'}, '+': {'slug': 'courbevois'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Courbevoie\r\n2024-07-05 16:59:05,409 - INFO - 2024-07-05 14:59.05 [info     ] Updated successfully           [diffsync.helpers] action=update diffs={'-': {'slug': 'courbevoie'}, '+': {'slug': 'courbevois'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e status=success unique_id=Courbevoie\r\n2024-07-05 16:59:05,409 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model update        [diffsync.helpers] action=update diffs={'-': {'parent': 'France'}, '+': {'parent': 'france'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Rennes\r\n2024-07-05 16:59:05,409 - INFO - 2024-07-05 14:59.05 [info     ] Updated successfully           [diffsync.helpers] action=update diffs={'-': {'parent': 'France'}, '+': {'parent': 'france'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e status=success unique_id=Rennes\r\n2024-07-05 16:59:05,409 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model create        [diffsync.helpers] action=create diffs={'+': {'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=BOB\r\n2024-07-05 16:59:05,427 - INFO - Created Site in YML file: {'name': 'BOB', 'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}\r\n2024-07-05 16:59:05,428 - INFO - 2024-07-05 14:59.05 [info     ]                                [diffsync.helpers] action=create diffs={'+': {'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e status=success unique_id=BOB\r\n2024-07-05 16:59:05,429 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model update        [diffsync.helpers] action=update diffs={'-': {'parent': 'France'}, '+': {'parent': 'france'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Seclin\r\n2024-07-05 16:59:05,429 - INFO - 2024-07-05 14:59.05 [info     ] Updated successfully           [diffsync.helpers] action=update diffs={'-': {'parent': 'France'}, '+': {'parent': 'france'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=region src=\u003cNetboxAdapter \"netbox\"\u003e status=success unique_id=Seclin\r\n2024-07-05 16:59:05,429 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model create        [diffsync.helpers] action=create diffs={'+': {'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=BOB\r\n2024-07-05 16:59:05,430 - ERROR - 2024-07-05 14:59.05 [error    ] Failed to create site {'name': 'BOB'} - it already exists! [diffsync.helpers] action=create diffs={'+': {'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e status=error unique_id=BOB\r\n2024-07-05 16:59:05,430 - WARNING - 2024-07-05 14:59.05 [warning  ] No object resulted from sync, will not process child objects. [diffsync.helpers] action=create diffs={'+': {'slug': 'bob', 'status': 'Staging', 'region': 'Rennes'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=BOB\r\n2024-07-05 16:59:05,430 - DEBUG - 2024-07-05 14:59.05 [debug    ] Attempting model update        [diffsync.helpers] action=update diffs={'-': {'slug': 'courbevoie'}, '+': {'slug': 'courbevois'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e unique_id=Courbevoie\r\n2024-07-05 16:59:05,430 - INFO - 2024-07-05 14:59.05 [info     ] Updated successfully           [diffsync.helpers] action=update diffs={'-': {'slug': 'courbevoie'}, '+': {'slug': 'courbevois'}} dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e model=site src=\u003cNetboxAdapter \"netbox\"\u003e status=success unique_id=Courbevoie\r\n2024-07-05 16:59:05,430 - INFO - 2024-07-05 14:59.05 [info     ] Sync complete                  [diffsync.helpers] dst=\u003cLocal \"local\"\u003e flags=\u003cDiffSyncFlags.CONTINUE_ON_FAILURE|SKIP_UNMATCHED_DST: 5\u003e src=\u003cNetboxAdapter \"netbox\"\u003e\r\n```\r\nIt try here to create BOB site and it created it bind to Rennes regions but after finish child sync of Rennes it try a second time to create BOB it self because in site list, and this raise a duplication error.\r\n\r\n\u003c!-- What did you expect to happen? --\u003e\r\n### Expected Behavior\r\n\r\nIt should create once any objects even if it try to sync it in the child tree process.\r\n\r\n\u003c!--\r\n    Describe in detail the exact steps that someone else can take to reproduce\r\n    this bug using the current release.\r\n--\u003e\r\n\r\n","author":{"url":"https://github.com/jo-bzh","@type":"Person","name":"jo-bzh"},"datePublished":"2024-07-05T15:12:57.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/275/diffsync/issues/275"}