René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"test_df_pivot tests failed at the system_prerelease session","articleBody":"After fixing failed tests mentioned #337, still have other tests failed:\r\n\r\n```\r\nFAILED tests/system/small/test_dataframe.py::test_dataframe_bool_aggregates[all_axis0] - AssertionError: Series.index are different\r\nFAILED tests/system/small/test_dataframe.py::test_dataframe_bool_aggregates[any_axis0] - AssertionError: Series.index are different\r\nFAILED tests/system/small/test_dataframe.py::test_df_pivot[values2-int64_too-columns2] - AssertionError: DataFrame.iloc[:, 0] (column name=\"('int64_col', \u003cNA\u003e)\") are different\r\nFAILED tests/system/small/test_groupby.py::test_dataframe_groupby_analytic[cumprod] - AssertionError: DataFrame.iloc[:, 1] (column name=\"int64_col\") are different\r\nFAILED tests/system/small/test_dataframe.py::test_to_pandas_downsampling_option_override - assert 1.3427486419677734 == 1 ± 3.0e-01\r\nFAILED tests/system/small/test_series.py::test_series_add_prefix - AssertionError: Series.index are different\r\nFAILED tests/system/small/test_series.py::test_series_add_suffix - AssertionError: Series.index are different\r\nFAILED tests/system/small/test_series.py::test_groupby_window_ops[cumprod] - AssertionError: Series are different\r\nFAILED tests/system/small/test_series.py::test_string_astype_int - AssertionError: Series.index are different\r\n```\r\n\r\n@tswast mentioned, the distinction of RangeIndex vs Int64Index issue could be unblock by setting `check_index_type=False`: https://pandas.pydata.org/docs/reference/api/pandas.testing.assert_series_equal.html#pandas.testing.assert_series_equal\r\n\r\nBut the iloc errors may be real issues. The callstack is shown as below:\r\n\r\n```\r\n=================================== FAILURES ===================================\r\n__________________ test_df_pivot[values2-int64_too-columns2] ___________________\r\n\r\nscalars_dfs = (          bool_col                                          bytes_col  \\\r\nrowindex                                    ......  2038-01-19 03:14:17.999999+00:00\r\n8            False  ...                              \u003cNA\u003e\r\n\r\n[9 rows x 13 columns])\r\nvalues = ['int64_col', 'float64_col'], index = 'int64_too'\r\ncolumns = ['string_col']\r\n\r\n    @pytest.mark.parametrize(\r\n        (\"values\", \"index\", \"columns\"),\r\n        [\r\n            (\"int64_col\", \"int64_too\", [\"string_col\"]),\r\n            ([\"int64_col\"], \"int64_too\", [\"string_col\"]),\r\n            ([\"int64_col\", \"float64_col\"], \"int64_too\", [\"string_col\"]),\r\n        ],\r\n    )\r\n    def test_df_pivot(scalars_dfs, values, index, columns):\r\n        scalars_df, scalars_pandas_df = scalars_dfs\r\n    \r\n        bf_result = scalars_df.pivot(\r\n            values=values, index=index, columns=columns\r\n        ).to_pandas()\r\n        pd_result = scalars_pandas_df.pivot(values=values, index=index, columns=columns)\r\n    \r\n        # Pandas produces NaN, where bq dataframes produces pd.NA\r\n\u003e       pd.testing.assert_frame_equal(bf_result, pd_result, check_dtype=False)\r\n\r\ntests/system/small/test_dataframe.py:2294: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nleft = array([nan, nan, nan, nan])\r\nright = array([nan, nan, \u003cNA\u003e, nan], dtype=object), err_msg = None\r\n\r\n    def _raise(left, right, err_msg) -\u003e NoReturn:\r\n        if err_msg is None:\r\n            if left.shape != right.shape:\r\n                raise_assert_detail(\r\n                    obj, f\"{obj} shapes are different\", left.shape, right.shape\r\n                )\r\n    \r\n            diff = 0\r\n            for left_arr, right_arr in zip(left, right):\r\n                # count up differences\r\n                if not array_equivalent(left_arr, right_arr, strict_nan=strict_nan):\r\n                    diff += 1\r\n    \r\n            diff = diff * 100.0 / left.size\r\n            msg = f\"{obj} values are different ({np.round(diff, 5)} %)\"\r\n\u003e           raise_assert_detail(obj, msg, left, right, index_values=index_values)\r\nE           AssertionError: DataFrame.iloc[:, 0] (column name=\"('int64_col', \u003cNA\u003e)\") are different\r\nE           \r\nE           DataFrame.iloc[:, 0] (column name=\"('int64_col', \u003cNA\u003e)\") values are different (25.0 %)\r\nE           [index]: [-2345, 0, 1, 2]\r\nE           [left]:  [nan, nan, nan, nan]\r\nE           [right]: [nan, nan, \u003cNA\u003e, nan]\r\n\r\n.nox/system_prerelease/lib/python3.11/site-packages/pandas/_testing/asserters.py:684: AssertionError\r\n```","author":{"url":"https://github.com/chelsea-lin","@type":"Person","name":"chelsea-lin"},"datePublished":"2024-01-23T17:58:24.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/341/python-bigquery-dataframes/issues/341"}