René's URL Explorer Experiment

Title: Refactor/upgrade pydantic to v2 by georgeRobertson2 · Pull Request #107 · NHSDigital/data-validation-engine · GitHub

Open Graph Title: Refactor/upgrade pydantic to v2 by georgeRobertson2 · Pull Request #107 · NHSDigital/data-validation-engine

X Title: Refactor/upgrade pydantic to v2 by georgeRobertson2 · Pull Request #107 · NHSDigital/data-validation-engine

Description: Pydantic Changes: BaseModel: fields to model_fields dict() to model_dump() pydantic.main.ModelMetaclass removed and replaced with BaseModel Field: regex in pydantic.Field changed to pattern Pydantic Config: Removed Config subclass in Pydantic models and replaced with model_config e.g. model_config = {...} OR ConfigDict() allow_population_by_field_name in Confg renamed to populate_by_name fields in Config (now model_config) depreciated anystr_strip_whitespace in Config renamed to str_strip_whitespace Pydantic Validators: pre arg in @validator now mode. E.g. pre=True -> mode="before" allow_reuse arg in @validator no longer required, recommended to delete and no issue always=True arg in @validator and @root_validator depreciated and you now define within the Field with validate_default arg @root_validator changed to @model_validator kwarg skip_on_failure depreciated and the logic of skip_on_failure=True is now default behaviour added info kwarg to each field validator as that is how you now access the metadata of the model removed values as @field_validator now just has cls, value and ValidationInfo. Reference to other fields will be obtained through ValidationInfo.data @validate_arguments replaced with @validate_call and kwargs replaced with ConfigDict Pydantic Types: get_validators replaced with get_pydantic_core_schema The way you now call the validation functions has changed. You return a core_schema object which either returns a CoreSchema object or a ChainSchema consisting of CoreSchema objects. You can additionally pass in a type handler which makes checks like if value is None: return value irrelevant as the handler performs the check on the datatype based on whether the model allows optional or not. types.ConstrainedStr replaced with types.StringConstraints regex attr in ConstrainedStr replaced with pattern ConstrainedDecimal retired and replaced with condecimal type Optional for types when defined with no default will not allow nulls. So, in cases where it we seemed to be allowing nulls, I have placed a = None default curtail_length kwarg for ConstrainedStr has been depreciated Specifc DVE changes: Auditing: changed the schema generation the BaseAuditor so it can work with FieldInfo object changed the polars schema generation in DDBAuditor to use get_polars_type_from_annotation rather than using PYTHON_TYPE_TO_POLARS_TYPE map as the FieldInfo object needs to be evaluated into a Polars type changed ddb_create_table_sql in DDBAuditor to use get_duckdb_typ_from_annotation as FieldInfo object needs to be evaluated into a Polars type Removed is_complex() as no longer available due to model_fields returning FieldInfo objects. Replaced with a new custom is_field_complex() function Domain Types (custom types): _SimpleRegexValidator base type switched to types.StringConstraints per ConstrainedStr being depreciated NHSNumber: removed parent type as no longer required _warn_for_possible_invalid_number method has had loc argument removed as it can no longer be supplied with the way pydantic v2 validations are written validate method has value to allow str or int type in the arguments as that was the expectation in the tests validate method has had field argument removed as it can no longer be supplied with the way pydantic v2 validations are written added a new method called get_pydantic_core_schema to facilitate new method of creating custom domain type validations. This now configures the validations required to validate the inbound value function wrapper: _validator_kwargs has a number of arguments that have been depreciated in Pydantic v2 (pre, each_item, always, whole, allow_reuse) FieldSpecification: Added a check for Annotated types that now frequently pass due to Pyd types now mapping to Annotated types rather than Pydantic type objects Fixes post 23/05/26: Fixed file transformation error on Scenario: Validate complex nested XML data (duckdb) This was due to an error with checking whether a type was complicated wrote a new is_type_complex wrote a strip optional now check the type rather than the FieldInfo object when generating the template_row(s) in xml reading Change the "missing" check logic in the FeedbackMessage.from_pydantic_error to explicitly check for missing values as missing validation errors in Pydantic only trigger when a kwarg isn't provided

Open Graph Description: Pydantic Changes: BaseModel: fields to model_fields dict() to model_dump() pydantic.main.ModelMetaclass removed and replaced with BaseModel Field: regex in pydantic.Field changed to pattern ...

X Description: Pydantic Changes: BaseModel: fields to model_fields dict() to model_dump() pydantic.main.ModelMetaclass removed and replaced with BaseModel Field: regex in pydantic.Field changed to pattern ...

Opengraph URL: https://github.com/NHSDigital/data-validation-engine/pull/107

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width