René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Serialize models with `exclude_unset`, not `exclude_defaults`","articleBody":"### What happened?\n\n`resource.update` (and `resource.update_status`) serialize Pydantic models with `model_dump(exclude_defaults=True)`. I think `exclude_unset=True` is more correct, and a recent change to the generated models has turned this from an academic distinction into a behavioural regression.\n\nSome background on how we got here. When I added `resource.update` in #98 I reached for `exclude_defaults` to stop `model_dump` serializing every optional field the function didn't set as an explicit `None`. That mostly worked, but it was too aggressive: in #114 we found it also strips `apiVersion` and `kind` when they equal their default, which for a generated model they always do, so we re-add them by hand. I floated switching to `exclude_none` and dropping the defaults from the generated models instead, but @haarchri pointed out that'd cost you the default values in editor/LSP completion, so we kept `exclude_defaults` plus the workaround. We didn't consider `exclude_unset` as an option.\n\nThe Crossplane CLI generates these models with `datamodel-code-generator`, and we're bumping the pinned image from a very old `0.31.2` to `0.59.0` to fix https://github.com/crossplane/cli/issues/63. Along the way the generator changed how it emits a field with an *object* default. Before:\n\n```python\nproviderConfigRef: Optional[ProviderConfigRef] = Field(\n    default_factory=lambda: ProviderConfigRef.model_validate(\n        {'kind': 'ClusterProviderConfig', 'name': 'default'}\n    )\n)\n```\n\nAfter:\n\n```python\nproviderConfigRef: ProviderConfigRef | None = Field(\n    {'kind': 'ClusterProviderConfig', 'name': 'default'}, validate_default=True\n)\n```\n\nThat change is deliberate upstream: see [koxudaxi/datamodel-code-generator#2228](https://github.com/koxudaxi/datamodel-code-generator/issues/2228).\n\nThe trouble is how that interacts with `exclude_defaults`. The two forms behave differently:\n\n- `default_factory`: an unset field is recognized as equal to its default, so `exclude_defaults=True` omits it. The composed resource has no `spec.providerConfigRef`.\n- raw default + `validate_default=True`: the default is validated into a `ProviderConfigRef` *instance* at construction. `exclude_defaults` compares that instance against the field's declared default, a plain `dict`. They're not equal, so the field is *not* excluded.\n\nSo every composed MR now carries an explicit `spec.providerConfigRef: {kind: ClusterProviderConfig, name: default}` that it previously omitted. Same goes for any field with an object default the function didn't set.\n\nIn most cases this isn't a problem. We generate the Pydantic models from the same OpenAPI schemas the Kubernetes API server uses, so if a function emits the default `providerConfigRef` it's just explicitly specifying what the API server would've defaulted anyway. So it's unlikely merging https://github.com/crossplane/cli/pull/64 would break anyone.\n\nSemantically though, I think `exclude_unset` is the correct option. We serialize these models into a `RunFunctionResponse` as desired resources, and Crossplane treats desired resources as server-side apply fully-specified intent. Under SSA the caller should only specify the fields it has an opinion about and wants to own. If a function doesn't explicitly want `providerConfigRef.name=default`, it shouldn't include it. Emitting it claims ownership of a field the function doesn't actually care about. That could mean spurious field-manager conflicts if some other entity *does* care about it, and it pins the field to the schema default even where the effective server-side default might differ (dynamic defaulting, admission webhooks, version skew between the model and the API server).\n\nThe distinction is:\n\n- `exclude_defaults` asks \"is this field's value different from its default?\"\n- `exclude_unset` asks \"did the caller set this field?\"\n\n`exclude_unset` matches the SSA intent: emit exactly the fields the function set, own those, leave the rest to the API server. It's also immune to the codegen change, because it doesn't care how a default is represented. A field the function didn't touch is absent from `model_fields_set`. That fixes the `providerConfigRef` regression for object defaults, scalar defaults, and any future representation the generator produces.\n\nIt is a behaviour change though, so we should think through the implications.\n\nFirst, `apiVersion` and `kind`. `exclude_unset` does *not* let us drop the manual re-add from #114. When a function builds a model with kwargs (the common case) it doesn't pass `apiVersion`/`kind`, so they're not in `model_fields_set` and `exclude_unset` omits them too. The workaround stays necessary either way.\n\nSecond, fields a function sets *to* their default value. Say a function writes `autoCreate=False` where `False` is the schema default. Today `exclude_defaults` drops it; `exclude_unset` keeps it. I'd argue that's more correct. If you set it, you have an opinion and should own it.\n\nThird, snapshot tests. Any function whose tests assert the exact `RunFunctionResponse` could see diffs if fields start appearing that `exclude_defaults` used to strip. (i.e. Fields the user explicitly set to the default value.)\n\n### How can we reproduce it?\n\nHand-write a model with an object-typed field that has a default, mirroring what the newer `datamodel-code-generator` emits, and dump it without setting that field:\n\n```python\nfrom pydantic import BaseModel, Field\n\nclass ProviderConfigRef(BaseModel):\n    kind: str | None = None\n    name: str\n\nclass Spec(BaseModel):\n    providerConfigRef: ProviderConfigRef | None = Field(\n        {\"kind\": \"ClusterProviderConfig\", \"name\": \"default\"}, validate_default=True\n    )\n\nprint(Spec().model_dump(exclude_defaults=True))\n# {'providerConfigRef': {'kind': 'ClusterProviderConfig', 'name': 'default'}}\nprint(Spec().model_dump(exclude_unset=True))\n# {}\n```\n\nThe `exclude_defaults` dump includes a field the caller never set; `exclude_unset` doesn't.\n\n### What environment did it happen in?\n\nFunction version: n/a — this is about `resource.update`/`update_status` in the SDK itself. The regression surfaces for anyone using models generated by `datamodel-code-generator \u003e= 0.54.0`, where the `validate_default=True` change landed, which the Crossplane CLI will produce once [crossplane/cli#64](https://github.com/crossplane/cli/pull/64) merges.\n","author":{"url":"https://github.com/negz","@type":"Person","name":"negz"},"datePublished":"2026-06-04T00:01:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/207/function-sdk-python/issues/207"}