Title: BigQuery: 'load_table_from_dataframe' raises OSError with STRUCT / RECORD columns. · Issue #9024 · googleapis/google-cloud-python · GitHub
Open Graph Title: BigQuery: 'load_table_from_dataframe' raises OSError with STRUCT / RECORD columns. · Issue #9024 · googleapis/google-cloud-python
X Title: BigQuery: 'load_table_from_dataframe' raises OSError with STRUCT / RECORD columns. · Issue #9024 · googleapis/google-cloud-python
Description: pyarrow-0.14.0 pandas '0.24.2' windows 10 Hi, I am tring to load dataframe to big query that looks like that uid_first agg_col 1001 [{'page_type': 1}, {'record_type': 1}, {'non_consectutive_home': 0}] the agg_col is list of dicts I also ...
Open Graph Description: pyarrow-0.14.0 pandas '0.24.2' windows 10 Hi, I am tring to load dataframe to big query that looks like that uid_first agg_col 1001 [{'page_type': 1}, {'record_type': 1}, {'non_consectutive_home': ...
X Description: pyarrow-0.14.0 pandas '0.24.2' windows 10 Hi, I am tring to load dataframe to big query that looks like that uid_first agg_col 1001 [{'page_type': 1}, {'record_type': 1}, {&...
Opengraph URL: https://github.com/googleapis/google-cloud-python/issues/9024
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"BigQuery: 'load_table_from_dataframe' raises OSError with STRUCT / RECORD columns.","articleBody":"pyarrow-0.14.0\r\npandas '0.24.2'\r\nwindows 10\r\nHi,\r\nI am tring to load dataframe to big query that looks like that\r\n\r\n```\r\nuid_first\tagg_col\r\n1001\t[{'page_type': 1}, {'record_type': 1}, {'non_consectutive_home': 0}]\r\n```\r\nthe agg_col is list of dicts\r\nI also tried dict\r\n\r\nSchema config:\r\n```\r\nschema = [\r\n bigquery.SchemaField(\"uid_first\",\"STRING\",mode=\"NULLABLE\"),\r\n bigquery.SchemaField(\"agg_col\",\"RECORD\",mode=\"NULLABLE\",fields=[\r\n bigquery.SchemaField(\"page_type\", \"INTEGER\", mode=\"NULLABLE\"),\r\n bigquery.SchemaField(\"record_type\", \"INTEGER\", mode=\"NULLABLE\"),\r\n bigquery.SchemaField(\"non_consectutive_home\", \"INTEGER\", mode=\"NULLABLE\")])]\r\n\r\n```\r\nload command\r\n```\r\ndataset_ref = client.dataset('dataset')\r\ntable_ref = dataset_ref.table('table')\r\ntable = bigquery.Table(table_ref,schema=schema)\r\ntable = client.load_table_from_dataframe(dff, table).result()\r\n```\r\nThe error message\r\n\r\n```\r\nTraceback (most recent call last):\r\n\r\n File \"\u003cipython-input-167-60a73e366976>\", line 4, in \u003cmodule>\r\n table = client.load_table_from_dataframe(dff, table).result()\r\n\r\n File \"C:\\ProgramData\\Anaconda3\\envs\\p37\\lib\\site-packages\\google\\cloud\\bigquery\\client.py\", line 1546, in load_table_from_dataframe\r\n os.remove(tmppath)\r\n\r\nFileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\\\Users\\\\ADMINI~1\\\\AppData\\\\Local\\\\Temp\\\\tmpcxotr6mb_job_5c94b06f.parquet'\r\n```\r\n\r\n\r\n\r\n\r\n","author":{"url":"https://github.com/dabasmoti","@type":"Person","name":"dabasmoti"},"datePublished":"2019-08-13T14:59:47Z","url":"https://github.com/googleapis/google-cloud-python/issues/9024"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:0d10ade3-6bd6-93e7-29ea-1e078b1c7269 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | EB00:ADCD1:686C15:89BC60:6AFE6E16 |
| html-safe-nonce | 4a8dceed05f93beaf58b044fa5780d7f2747485db1ab19e5d87449733a4c7fc3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQjAwOkFEQ0QxOjY4NkMxNTo4OUJDNjA6NkFGRTZFMTYiLCJ2aXNpdG9yX2lkIjoiODE4NDIyNTIzOTkxNjI0NDUwMiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | a6d0c9b4c26a12e17bc164f3ec3844d4e7ad531422758f75701e7891cd649c2a |
| hovercard-subject-tag | issue:480207067 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/google-cloud-python/9024/issue_layout |
| twitter:image | https://opengraph.githubassets.com/29f44fbca1c48d7852185226b4e7314d120be735498a8af031395c2d133c1dc9/googleapis/google-cloud-python/issues/9024 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/29f44fbca1c48d7852185226b4e7314d120be735498a8af031395c2d133c1dc9/googleapis/google-cloud-python/issues/9024 |
| og:image:alt | pyarrow-0.14.0 pandas '0.24.2' windows 10 Hi, I am tring to load dataframe to big query that looks like that uid_first agg_col 1001 [{'page_type': 1}, {'record_type': 1}, {'non_consectutive_home': ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | dabasmoti |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-preview |
| go-import | github.com/googleapis/google-cloud-python git https://github.com/googleapis/google-cloud-python.git |
| octolytics-dimension-user_id | 16785467 |
| octolytics-dimension-user_login | googleapis |
| octolytics-dimension-repository_id | 16316451 |
| octolytics-dimension-repository_nwo | googleapis/google-cloud-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 16316451 |
| octolytics-dimension-repository_network_root_nwo | googleapis/google-cloud-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width