René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bug: reserve_jobs=True fails when dj.config.jobs.add_job_metadata = True and dj.config.jobs.keep_completed = True","articleBody":"## Bug Report\n\n### Description\nWhen testing the job metadata functionality i encountered following problem:\nWhen setting\n```\ndj.config.jobs.add_job_metadata = True\ndj.config.jobs.keep_completed = True\n```\ni cant run populate with reserve_jobs=True.\n\n### Reproducibility\n```\nimport datajoint as dj\ndj.config.jobs.add_job_metadata = True\ndj.config.jobs.keep_completed = True\ndj.conn() # using datajoint.json\nschema = dj.Schema(\"TEST_\")\n\n@schema\nclass Numbers(dj.Lookup):\n    definition = \"\"\"\n    number: int64\n    \"\"\"\n    contents = [(0,), (1,), (2,), (4,), (8,)]\n\n@schema\nclass NumbersSquared(dj.Computed):\n    definition = \"\"\"\n    -\u003e Numbers\n    ---\n    squared: int64\n    \"\"\"\n\n    def make(self, key):\n        self.insert1({**key, \"squared\": key[\"number\"] ** 2})\n\nNumbersSquared.populate(reserve_jobs=True)\n```\nFails with: \n```\n---------------------------------------------------------------------------\nDataJointError                            Traceback (most recent call last)\nCell In[2], line 1\n----\u003e 1 NumbersSquared.populate(reserve_jobs=True)\n\nFile ~/datajoint-python/src/datajoint/autopopulate.py:366, in AutoPopulate.populate(self, suppress_errors, return_exception_objects, reserve_jobs, max_calls, display_progress, processes, make_kwargs, priority, refresh, *restrictions)\n    363     raise DataJointError(\"Populate cannot be called during a transaction.\")\n    365 if reserve_jobs:\n--\u003e 366     return self._populate_distributed(\n    367         *restrictions,\n    368         suppress_errors=suppress_errors,\n    369         return_exception_objects=return_exception_objects,\n    370         max_calls=max_calls,\n    371         display_progress=display_progress,\n    372         processes=processes,\n    373         make_kwargs=make_kwargs,\n    374         priority=priority,\n    375         refresh=refresh,\n    376     )\n    377 else:\n    378     return self._populate_direct(\n    379         *restrictions,\n    380         suppress_errors=suppress_errors,\n   (...)    385         make_kwargs=make_kwargs,\n    386     )\n\nFile ~/datajoint-python/src/datajoint/autopopulate.py:490, in AutoPopulate._populate_distributed(self, suppress_errors, return_exception_objects, max_calls, display_progress, processes, make_kwargs, priority, refresh, *restrictions)\n    486     refresh = config.jobs.auto_refresh\n    487 if refresh:\n    488     # Use delay=-1 to ensure jobs are immediately schedulable\n    489     # (avoids race condition with scheduled_time \u003c= CURRENT_TIMESTAMP(3) check)\n--\u003e 490     self.jobs.refresh(*restrictions, priority=priority, delay=-1)\n    492 # Fetch pending jobs ordered by priority (use CURRENT_TIMESTAMP(3) for datetime(3) precision)\n    493 pending_query = self.jobs.pending \u0026 \"scheduled_time \u003c= CURRENT_TIMESTAMP(3)\"\n\nFile ~/datajoint-python/src/datajoint/jobs.py:398, in Job.refresh(self, delay, priority, stale_timeout, orphan_timeout, *restrictions)\n    394 # 2. Re-pend success jobs if keep_completed=True\n    395 if config.jobs.keep_completed:\n    396     # Success jobs whose keys are in key_source but not in target\n    397     # Disable semantic_check for Job table operations\n--\u003e 398     success_to_repend = self.completed.restrict(key_source, semantic_check=False) - self._target\n    399     repend_keys = success_to_repend.keys()\n    400     for key in repend_keys:\n\nFile ~/datajoint-python/src/datajoint/expression.py:282, in QueryExpression.__sub__(self, restriction)\n    276 def __sub__(self, restriction):\n    277     \"\"\"\n    278     Inverted restriction e.g. ``q1 - q2``.\n    279     :return: a restricted copy of the input argument\n    280     See QueryExpression.restrict for more detail.\n    281     \"\"\"\n--\u003e 282     return self.restrict(Not(restriction))\n\nFile ~/datajoint-python/src/datajoint/expression.py:234, in QueryExpression.restrict(self, restriction, semantic_check)\n    232         result._top = restriction\n    233     return result\n--\u003e 234 new_condition = make_condition(self, restriction, attributes, semantic_check=semantic_check)\n    235 if new_condition is True:\n    236     return self  # restriction has no effect, return the same object\n\nFile ~/datajoint-python/src/datajoint/condition.py:403, in make_condition(query_expression, condition, columns, semantic_check)\n    401 # restrict by another expression\n    402 if isinstance(condition, QueryExpression):\n--\u003e 403     assert_join_compatibility(query_expression, condition, semantic_check=semantic_check)\n    404     # Match on all non-hidden namesakes (hidden attributes excluded)\n    405     common_attributes = [q for q in condition.heading.names if q in query_expression.heading.names]\n\nFile ~/datajoint-python/src/datajoint/condition.py:256, in assert_join_compatibility(expr1, expr2, semantic_check)\n    254 # Semantic match requires both lineages to be non-None and equal\n    255 if lineage1 is None or lineage2 is None or lineage1 != lineage2:\n--\u003e 256     raise DataJointError(\n    257         f\"Cannot join on attribute `{name}`: \"\n    258         f\"different lineages ({lineage1} vs {lineage2}). \"\n    259         f\"Use .proj() to rename one of the attributes.\"\n    260     )\n\nDataJointError: Cannot join on attribute `number`: different lineages (TEST_.~~numbers_squared.number vs TEST_.#numbers.number). Use .proj() to rename one of the attributes.\n```\n\nUbuntu 24.04, python 3.12.11\nMySQL - remotely accessed myslq-latest docker (MySQL Server 9.6.0-1.el9)\nDataJoint Version : DataJoint 2.0.0a27\n\n\n","author":{"url":"https://github.com/arturoptophys","@type":"Person","name":"arturoptophys"},"datePublished":"2026-02-05T12:34:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/1379/datajoint-python/issues/1379"}