Title: Timestamp loss of precision in insert_rows · Issue #392 · googleapis/python-bigquery · GitHub
Open Graph Title: Timestamp loss of precision in insert_rows · Issue #392 · googleapis/python-bigquery
X Title: Timestamp loss of precision in insert_rows · Issue #392 · googleapis/python-bigquery
Description: Certain values of datetime lose a single microsecond on their way into BigQuery. This appears to happen because of the JSON rendering of timestamps as a floating point number of seconds. Environment details OS type and version: Ubuntu Bi...
Open Graph Description: Certain values of datetime lose a single microsecond on their way into BigQuery. This appears to happen because of the JSON rendering of timestamps as a floating point number of seconds. Environmen...
X Description: Certain values of datetime lose a single microsecond on their way into BigQuery. This appears to happen because of the JSON rendering of timestamps as a floating point number of seconds. Environmen...
Opengraph URL: https://github.com/googleapis/python-bigquery/issues/392
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Timestamp loss of precision in insert_rows","articleBody":"Certain values of `datetime` lose a single microsecond on their way into BigQuery. This appears to happen because of the JSON rendering of timestamps as a floating point number of seconds.\r\n\r\n#### Environment details\r\n\r\n - OS type and version: Ubuntu Bionic\r\n - Python version: 3.6.9\r\n - pip version: 9.0.1\r\n - `google-cloud-bigquery` version: 1.28.0\r\n\r\n#### Steps to reproduce\r\n\r\nHere is an example of a `datetime` value that loses a microsecond:\r\n```py\r\ndatetime(2020, 11, 17, 1, 6, 52, 353795, tzinfo=datetime.timezone(datetime.timedelta(0), 'UTC'))\r\n```\r\n\r\nAs a full integration repro, you could try something like this:\r\n\r\n 1. Create a BigQuery table with a `TIMESTAMP` column.\r\n 2. Use `insert_rows` to insert the value above.\r\n 3. Inspect the results in the Console, where it will show as `2020-11-17 01:06:52.353794 UTC` instead of `2020-11-17 01:06:52.353795 UTC`.\r\n\r\n#### Code example\r\n\r\nAn easier repro hints at the root cause. By looking at the output of `_timestamp_to_json_row`, it appears that the problematic values are rounded down to microsecond precision.\r\n\r\n```python\r\nimport datetime\r\nfrom google.cloud import bigquery as bq\r\nts = datetime.datetime(2020, 11, 17, 1, 6, 52, 353795, tzinfo=datetime.timezone(datetime.timedelta(0), 'UTC'))\r\nprint(bq._helpers._timestamp_to_json_row(ts))\r\n```\r\nThis will print:\r\n```\r\n1605575212.3537948\r\n```\r\n\r\nPerhaps the solution is to add a half microsecond (0.5e-6) so that truncating results in rounding to the nearest microsecond.\r\n","author":{"url":"https://github.com/mhfrantz","@type":"Person","name":"mhfrantz"},"datePublished":"2020-11-18T00:34:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/392/python-bigquery/issues/392"}
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:117aee02-ad71-a58c-1023-72d50bc1f440 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | AD7E:CD446:2D9442A:3ED46EB:6A4F15CC |
| html-safe-nonce | da1a80562ee3a1b019d59a3c918df579438efcee01e601bab205a5b6b98cc33c |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBRDdFOkNENDQ2OjJEOTQ0MkE6M0VENDZFQjo2QTRGMTVDQyIsInZpc2l0b3JfaWQiOiI2NzQyODYwNzAyNDMwNDY3NTMyIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | c618540d7dae3c0bef3adf5c7e051966c6032eb87cd667e1c95db63bca520eec |
| hovercard-subject-tag | issue:745192830 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/python-bigquery/392/issue_layout |
| twitter:image | https://opengraph.githubassets.com/9311b63aeea77ab0be04893c93e05b177549c2017409757c1bcc7bd6475ca4a7/googleapis/python-bigquery/issues/392 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/9311b63aeea77ab0be04893c93e05b177549c2017409757c1bcc7bd6475ca4a7/googleapis/python-bigquery/issues/392 |
| og:image:alt | Certain values of datetime lose a single microsecond on their way into BigQuery. This appears to happen because of the JSON rendering of timestamps as a floating point number of seconds. Environmen... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | mhfrantz |
| hostname | github.com |
| expected-hostname | github.com |
| None | b92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9 |
| turbo-cache-control | no-preview |
| go-import | github.com/googleapis/python-bigquery git https://github.com/googleapis/python-bigquery.git |
| octolytics-dimension-user_id | 16785467 |
| octolytics-dimension-user_login | googleapis |
| octolytics-dimension-repository_id | 226992475 |
| octolytics-dimension-repository_nwo | googleapis/python-bigquery |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 226992475 |
| octolytics-dimension-repository_network_root_nwo | googleapis/python-bigquery |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 2b8f23afb982271f1b22258a94aede67a6b77760 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width