Title: fix: resolve the issue where rpc timeout of 0 is used when timeout expires by parthea · Pull Request #776 · googleapis/python-api-core · GitHub
Open Graph Title: fix: resolve the issue where rpc timeout of 0 is used when timeout expires by parthea · Pull Request #776 · googleapis/python-api-core
X Title: fix: resolve the issue where rpc timeout of 0 is used when timeout expires by parthea · Pull Request #776 · googleapis/python-api-core
Description: In PR #462, the deadline argument of google.api_core.retry.Retry was refactored and marked as deprecated. Although the comment below mentions that the deadline argument would override the timeout, this is not the behavior that we see when the deadline argument of default_retry is set as part of the wrapped method as is the case in this line, similar to the bug report in #654. python-api-core/google/api_core/retry/retry_unary.py Lines 194 to 200 in b1fae31 Unfortunately, in the past this class (and the api-core library as a whole) has not been properly distinguishing the concepts of "timeout" and "deadline", and the ``deadline`` parameter has meant ``timeout``. That is why ``deadline`` has been deprecated and ``timeout`` should be used instead. If the ``deadline`` parameter is set, it will override the ``timeout`` parameter. In other words, ``retry.deadline`` should be treated as just a deprecated alias for ``retry.timeout``. We actually have 2 separate values deadline and timeout which behave in a different manner. The deadline argument of default_retry is an overall invocation timeout, while timeout is the rpc timeout. The client will send requests (respecting the configured backoff) during the specified timeout allowed. Once the timeout is up, the rpc timeout ends up being 0, but the requests keep going out because the deadline hasn't expired. A simple fix is to have a reasonableness check on the rpc timeout to avoid sending requests with rpc timeout of 0. A longer term fix is being considered where we have separate timeouts for overall timeout and rpc timeout. Currently, we just have a single timeout defined in pubsub_grpc_service_config.json does not provide flexibility to configure both an overall timeout or rpc timeout. Initial testing shows that this should fix #654 Output of test log. Previously I would see gRPC timeout:0 seconds as reported in #654 Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/usr/local/google/home/partheniou/git/gapic-generator-python', '--capture=no', '/usr/local/google/home/partheniou/git/gapic-generator-python/tests/system/test_retry.py::test_lro[grpc]'] ============================= test session starts ============================== platform linux -- Python 3.9.16, pytest-7.4.0, pluggy-1.5.0 rootdir: /usr/local/google/home/partheniou/git/gapic-generator-python plugins: localserver-0.8.1, cov-5.0.0, asyncio-0.23.5 asyncio: mode=strict collected 1 item tests/system/test_retry.py {"timestamp": "2025-01-12 13:11:39,035", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.client", "message": "Created client `google.showcase_v1beta1.EchoClient`.", "serviceName": "google.showcase.v1beta1.Echo", "credentialsType": "builtins.NoneType", "universeDomain": ""} gRPC timeout: 60.0 seconds {"timestamp": "2025-01-12 13:11:39,042", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:12:39,048", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 0.0s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:12:39,084", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:13:39,088", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 0.0s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:13:39,114", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:14:39,118", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 11.0s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:14:50,085", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:15:50,089", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 20.4s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:16:10,474", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:17:10,477", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 27.2s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:17:37,729", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:18:37,733", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 5.6s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:18:43,345", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:19:43,349", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 1.5s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:19:44,817", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} {"timestamp": "2025-01-12 13:20:44,821", "severity": "DEBUG", "name": "google.api_core.retry", "message": "Retrying due to 504 Deadline Exceeded, sleeping 39.8s ..."} gRPC timeout: 60 seconds {"timestamp": "2025-01-12 13:21:24,597", "severity": "DEBUG", "name": "google.showcase_v1beta1.services.echo.transports.grpc", "message": "Sending request for /google.showcase.v1beta1.Echo/Block", "rpcName": "/google.showcase.v1beta1.Echo/Block", "serviceName": "google.showcase.v1beta1.Echo", "request": {"payload": "{\n \"responseDelay\": \"60s\"\n}", "requestMethod": "grpc", "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}}, "metadata": {"x-goog-api-version": "v1_20240408", "x-goog-api-client": "gl-python/3.9.16 grpc/1.67.1 gax/2.24.0 gapic/0.0.0"}} Fixes #654 Fixes b/387530669
Open Graph Description: In PR #462, the deadline argument of google.api_core.retry.Retry was refactored and marked as deprecated. Although the comment below mentions that the deadline argument would override the timeout, ...
X Description: In PR #462, the deadline argument of google.api_core.retry.Retry was refactored and marked as deprecated. Although the comment below mentions that the deadline argument would override the timeout, ...
Opengraph URL: https://github.com/googleapis/python-api-core/pull/776
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:d550f17d-388f-98d1-0cd5-2d53d88c5654 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 94A2:633B1:EDEAB1:14CF2B4:6A4E91BA |
| html-safe-nonce | 77b065bef5dcfedafc6f7951b7415f0de2b57c69dd42ca7eacd27c9b5dce4af7 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5NEEyOjYzM0IxOkVERUFCMToxNENGMkI0OjZBNEU5MUJBIiwidmlzaXRvcl9pZCI6IjE4MTM4MzY4NTQ5MjQ4NDE0MDIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | d42da9e04af57d97dacd881ead40995d25663ce691663ab9b0b2646bc34bafa0 |
| hovercard-subject-tag | pull_request:2272545248 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/googleapis/python-api-core/pull/776/files |
| twitter:image | https://avatars.githubusercontent.com/u/5184014?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/5184014?s=400&v=4 |
| og:image:alt | In PR #462, the deadline argument of google.api_core.retry.Retry was refactored and marked as deprecated. Although the comment below mentions that the deadline argument would override the timeout, ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 41b6ab3ba6d20a71766ac245b5a4a94c6fc672a9cd4da7d44c1b33ab8bf6a21c |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/googleapis/python-api-core git https://github.com/googleapis/python-api-core.git |
| octolytics-dimension-user_id | 16785467 |
| octolytics-dimension-user_login | googleapis |
| octolytics-dimension-repository_id | 226992456 |
| octolytics-dimension-repository_nwo | googleapis/python-api-core |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 226992456 |
| octolytics-dimension-repository_network_root_nwo | googleapis/python-api-core |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | e6a744804e8e70f97b4d5a18a94dcc63db22f97a |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width