René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"QAPage","mainEntity":{"@type":"Question","name":"REPOSTED: error 12728: Field 'tap_nom' is not between (or at) tap_min and tap_max for 1 transformer","text":"
Repost of a previous discussion conversation which is to be deleted
\n
willemijnbrus
\n3 days ago
\n
Hi PGM team,
\n
We are using PGM to do a loadflow on several low voltage grids. Since today we get the following error:
\nbatch #1, error 12728: Field 'tap_nom' is not between (or at) tap_min and tap_max for 1 transformer.
\nThis error sometimes seems to appear and sometimes it does not. Three weeks ago we had some runs on the same input files and we did not have this error. Today we sometimes get this error and sometimes we dont. When we read our input files (grid topology) we define the tap_min and tap_max following the input files. We do not define the tap_nom.  below is our code where we read the input files.
\nDo you have any suggestions on how we can debug this error?
\n
Best,
\nWillemijn
\n
def read_topology_pgm(\n    network_id: str, modify_tap_position: int = 0\n) -> tuple[GridTopology, dict[int, int]]:\n    \"\"\"\n    Reads network topology into a PGM-ready format.\n    :param network: the ID of the network to load or the path to the file\n    :return: the PGM-ready format for the given `network_id`\n    \"\"\"\n    # Use Polars for uniformity of file access (especially when loading from\n    # S3), but convert to Pandas dataframe for easy NumPy conversion\n    # TODO(BS): Drop dependency on Pandas by using Polars throughout\n    network_file = settings.input_dir / \"network\" / f\"network{network_id}.parquet\"\n    dataframe = pl.scan_parquet(network_file).collect().to_pandas()\n\n    ean_to_load_mapping: dict[int, int] = {\n        int(row[\"loads__EAN\"]): int(row[\"loads__id\"])\n        for idx, row in dataframe[[\"loads__id\", \"loads__EAN\"]].dropna().iterrows()\n    }\n\n    # loads\n    len(np.array(dataframe[\"loads__id\"]))\n    loads = np.array(dataframe[\"loads__id\"].unique())\n    loads = loads[~np.isnan(loads)]\n\n    loads_node = np.array(dataframe[\"loads__node\"])\n    loads_node = loads_node[~np.isnan(loads_node)]\n\n    loads_status = np.array(dataframe[\"loads__status\"])\n    loads_status = loads_status[~np.isnan(loads_status)]\n\n    loads_type = np.array(dataframe[\"loads__type\"])\n    loads_type = loads_type[~np.isnan(loads_type)]\n\n    loads_p = np.array(dataframe[\"loads__p_specified\"])\n    loads_p = loads_p[~np.isnan(loads_p)]\n\n    loads_q = np.array(dataframe[\"loads__q_specified\"])\n    loads_q = loads_q[~np.isnan(loads_q)]\n\n    # node\n    node = initialize_array(\n        DatasetType.input, ComponentType.node, len(dataframe[\"nodes__id\"].dropna())\n    )\n    node[\"id\"] = np.array(dataframe[\"nodes__id\"].dropna())\n    node[\"u_rated\"] = np.array(dataframe[\"nodes__u_rated\"].dropna())\n\n    # line\n    line_dt = dataframe[\"lines__id\"].dropna()\n    line = initialize_array(DatasetType.input, ComponentType.line, len(line_dt))\n    line[\"id\"] = np.array(dataframe[\"lines__id\"].dropna())\n    line[\"from_node\"] = np.array(dataframe[\"lines__from_node\"].dropna())\n    line[\"to_node\"] = np.array(dataframe[\"lines__to_node\"].dropna())\n    line[\"from_status\"] = np.array(dataframe[\"lines__from_status\"].dropna())\n    line[\"to_status\"] = np.array(dataframe[\"lines__to_status\"].dropna())\n    line[\"r1\"] = np.array(dataframe[\"lines__r1\"].dropna())\n    line[\"x1\"] = np.array(dataframe[\"lines__x1\"].dropna())\n    line[\"c1\"] = np.array(dataframe[\"lines__c1\"].dropna())\n    line[\"tan1\"] = np.array(dataframe[\"lines__tan1\"].dropna())\n    line[\"i_n\"] = np.array(dataframe[\"lines__i_n\"].dropna())\n\n    # load\n    sym_load = initialize_array(DatasetType.input, ComponentType.sym_load, len(loads))\n    sym_load[\"id\"] = loads\n    sym_load[\"node\"] = loads_node\n    sym_load[\"status\"] = loads_status\n    sym_load[\"type\"] = loads_type\n    sym_load[\"p_specified\"] = loads_p\n    sym_load[\"q_specified\"] = loads_q\n\n    # source\n    source = initialize_array(DatasetType.input, ComponentType.source, 1)\n    source[\"id\"] = np.array(dataframe[\"sources__id\"].unique())[\n        ~np.isnan(np.array(dataframe[\"sources__id\"].unique()))\n    ]\n    source[\"node\"] = np.array(dataframe[\"sources__node\"].unique())[\n        ~np.isnan(np.array(dataframe[\"sources__node\"].unique()))\n    ]\n    source[\"status\"] = np.array(dataframe[\"sources__status\"].unique())[\n        ~np.isnan(np.array(dataframe[\"sources__status\"].unique()))\n    ]\n    source[\"u_ref\"] = np.array(dataframe[\"sources__u_ref\"].unique())[\n        ~np.isnan(np.array(dataframe[\"sources__u_ref\"].unique()))\n    ]\n\n    # transformer\n    transformer_id = np.array(dataframe[\"transformers__id\"])[\n        ~np.isnan(dataframe[\"transformers__id\"])\n    ]\n    transformer = initialize_array(\n        DatasetType.input, ComponentType.transformer, len(transformer_id)\n    )\n    transformer[\"id\"] = transformer_id\n    transformer[\"u1\"] = np.array(dataframe[\"transformers__u1\"])[\n        ~np.isnan(dataframe[\"transformers__u1\"])\n    ]\n    transformer[\"u2\"] = np.array(dataframe[\"transformers__u2\"])[\n        ~np.isnan(dataframe[\"transformers__u2\"])\n    ]\n    transformer[\"sn\"] = np.array(dataframe[\"transformers__sn\"])[\n        ~np.isnan(dataframe[\"transformers__sn\"])\n    ]\n    transformer[\"pk\"] = np.array(dataframe[\"transformers__pk\"])[\n        ~np.isnan(dataframe[\"transformers__pk\"])\n    ]\n    transformer[\"i0\"] = np.array(dataframe[\"transformers__i0\"])[\n        ~np.isnan(dataframe[\"transformers__i0\"])\n    ]\n    transformer[\"p0\"] = np.array(dataframe[\"transformers__p0\"])[\n        ~np.isnan(dataframe[\"transformers__p0\"])\n    ]\n    transformer[\"from_node\"] = np.array(dataframe[\"transformers__from_node\"])[\n        ~np.isnan(dataframe[\"transformers__from_node\"])\n    ]\n    transformer[\"to_node\"] = np.array(dataframe[\"transformers__to_node\"])[\n        ~np.isnan(dataframe[\"transformers__to_node\"])\n    ]\n    transformer[\"from_status\"] = np.array(dataframe[\"transformers__from_status\"])[\n        ~np.isnan(dataframe[\"transformers__from_status\"])\n    ]\n    transformer[\"to_status\"] = np.array(dataframe[\"transformers__to_status\"])[\n        ~np.isnan(dataframe[\"transformers__to_status\"])\n    ]\n    transformer[\"uk\"] = np.array(dataframe[\"transformers__uk\"])[\n        ~np.isnan(dataframe[\"transformers__uk\"])\n    ]\n    transformer[\"winding_from\"] = np.array(dataframe[\"transformers__winding_from\"])[\n        ~np.isnan(dataframe[\"transformers__winding_from\"])\n    ]\n    transformer[\"winding_to\"] = np.array(dataframe[\"transformers__winding_to\"])[\n        ~np.isnan(dataframe[\"transformers__winding_to\"])\n    ]\n    transformer[\"clock\"] = np.array(dataframe[\"transformers__clock\"])[\n        ~np.isnan(dataframe[\"transformers__clock\"])\n    ]\n    transformer[\"tap_side\"] = np.array(dataframe[\"transformers__tap_side\"])[\n        ~np.isnan(dataframe[\"transformers__tap_side\"])\n    ]\n    transformer[\"tap_pos\"] = np.array(\n        dataframe[\"transformers__tap_pos\"] + modify_tap_position\n    )[~np.isnan(dataframe[\"transformers__tap_pos\"])]\n    transformer[\"tap_min\"] = np.array(dataframe[\"transformers__tap_min\"])[\n        ~np.isnan(dataframe[\"transformers__tap_min\"])\n    ]\n    transformer[\"tap_max\"] = np.array(dataframe[\"transformers__tap_max\"])[\n        ~np.isnan(dataframe[\"transformers__tap_max\"])\n    ]\n    transformer[\"tap_size\"] = np.array(dataframe[\"transformers__tap_size\"])[\n        ~np.isnan(dataframe[\"transformers__tap_size\"])\n    ]\n\n    # transformer tap regulator\n    transformer_tap_regulator = initialize_array(\n        DatasetType.input, ComponentType.transformer_tap_regulator, 1\n    )\n    transformer_tap_regulator[\"id\"] = [888888888]\n    transformer_tap_regulator[\"regulated_object\"] = transformer_id\n    transformer_tap_regulator[\"status\"] = [1]\n    transformer_tap_regulator[\"control_side\"] = [BranchSide.to_side]\n    transformer_tap_regulator[\"u_set\"] = [400.0]\n    transformer_tap_regulator[\"u_band\"] = [20.0]\n    transformer_tap_regulator[\"line_drop_compensation_r\"] = [0.0]\n    transformer_tap_regulator[\"line_drop_compensation_x\"] = [0.0]\n\n    # TODO: Ensure that all `sym_load`s have CONSTANT POWER type\n    # # Filter all arrays that have constant power\n    # selected_loads = np.array([load for load in pgm_network['sym_load'] if (load[3] == LoadGenType.const_power).any()])\n    # col_names = selected_loads['id']\n    # # Assign the ID's of the objects that should be changed / updated within the model to loads in the loadfile.\n    # loads = e_load\n\n    pgm_topology = {\n        ComponentType.node: node,\n        ComponentType.line: line,\n        ComponentType.transformer: transformer,\n        ComponentType.sym_load: sym_load,\n        ComponentType.source: source,\n        ComponentType.transformer_tap_regulator: transformer_tap_regulator,\n    }
\n
mgovers
\n3 days ago
\nMaintainer
\nHi @willemijnbrus , thank you for your question.
\n
While I cannot know for sure without seeing the actual data, I think you are running into the following edge case:
\n
If you look at the input data for Transformers ( https://power-grid-model.readthedocs.io/en/stable/user_manual/components.html#transformer ), then you can see that:
\n
tap_nom is optional and defaults to 0 if you do not provide it.
\ntap_nom has a valid value if and only if (tap_min <= tap_nom <= tap_max) or (tap_min >= tap_nom >= tap_max)
\nif not, you will receive the error message you ran into.
\nIt seems you are running into the edge case that 0 is not in the range [tap_min, tap_max], i.e.:
\n
tap_min > 0 and tap_max > 0 or tap_min < 0 and tap_max < 0
\nIn that case, the default value for tap_nom is not a good value for the transformer and you will need to set it yourself.
\n
As a solution, I propose the following. If tap_nom is not relevant for your use case, then you may e.g. set it to any of the following values, whichever makes sense:
\n
tap_pos (if you set it in the input data)
\ntap_min
\ntap_max
\n(tap_min + tap_max) // 2
\ntap_min or tap_max, whichever is closer to 0.
\nEDIT: cfr. https://github.com/orgs/PowerGridModel/discussions/43#discussioncomment-11817887 (credits to @TonyXiang8787 )
\n
It is not a problem that tap_min is higher than tap_max. It happens actually regularly. PGM requires that tap_pos and tap_nom have to be between tap_min and tap_max. The following combination of values is valid:
\n
tap_min = 5
\ntap_max = 1
\ntap_nom = 3
\ntap_pos = 4
\nHowever, tap_nom=0 is not valid value.
\n
1 reply
\n@willemijnbrus
\nwillemijnbrus
\n3 days ago
\nAuthor
\nHi Martijn, we will have a check if 0 is always in the range of tapmin_tap_max. and also thank you for your suggestion of setting it another value if tap_nom is not important. Very helpful tips. We will try those!
\n
Answer selected by mgovers
\n
BartSchuurmans
\n5 hours ago
\nAdditional information: in some cases, tap_min was higher than tap_max in our input data, which was why no valid value for tap_pos / tap_nom was possible.
\n
2 replies 1 new
\n@TonyXiang8787
\nTonyXiang8787
\n5 hours ago
\nMaintainer
\nHi @BartSchuurmans,
\n
It is not a problem that tap_min is higher than tap_max. It happens actually regularly. PGM requires that tap_pos and tap_nom have to be between tap_min and tap_max. The following value is valid:
\n
tap_min = 5
\ntap_max = 1
\ntap_nom = 3
\ntap_pos = 4
\nHowever, tap_nom=0 is not valid value.
\n
@BartSchuurmans
\nBartSchuurmans
\n5 hours ago
\nHi @TonyXiang8787, thanks for your quick response. This is good to know, I assumed there was an error in our input data.
","upvoteCount":1,"answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"
Answer Copy-pasted from OP:
\n
Hi @willemijnbrus , thank you for your question.
\n
While I cannot know for sure without seeing the actual data, I think you are running into the following edge case:
\n
If you look at the input data for Transformers ( https://power-grid-model.readthedocs.io/en/stable/user_manual/components.html#transformer ), then you can see that:
\n
\n
tap_nom is optional and defaults to 0 if you do not provide it.
\n
tap_nom has a valid value if and only if (tap_min <= tap_nom <= tap_max) or (tap_min >= tap_nom >= tap_max)
\n
If not, you will receive the error message you ran into.
\n
It seems you are running into the edge case that 0 is not in the range [tap_min, tap_max], i.e.:
\n
\n
tap_min > 0 and tap_max > 0 or tap_min < 0 and tap_max < 0
\nIn that case, the default value for tap_nom is not a good value for the transformer and you will need to set it yourself.
\n
As a solution, I propose the following. If tap_nom is not relevant for your use case, then you may e.g. set it to any of the following values, whichever makes sense:
\n
\n
tap_pos (if you set it in the input data)
\n
tap_min
\n
tap_max
\n
(tap_min + tap_max) // 2
\n
tap_min or tap_max, whichever is closer to 0.
\n
\n
EDIT: (credits to @TonyXiang8787 )
\n
It is not a problem that tap_min is higher than tap_max. It happens actually regularly. PGM requires that tap_pos and tap_nom have to be between tap_min and tap_max. The following combination of values is valid:
\n
tap_min = 5\ntap_max = 1\ntap_nom = 3\ntap_pos = 4\n
\n
However, tap_nom=0 is not valid value.
","upvoteCount":1,"url":"https://github.com/orgs/PowerGridModel/discussions/45#discussioncomment-11821329"}}}