Title: Add support to custom `JSONEncoder` · Issue #656 · googleapis/python-logging · GitHub
Open Graph Title: Add support to custom `JSONEncoder` · Issue #656 · googleapis/python-logging
X Title: Add support to custom `JSONEncoder` · Issue #656 · googleapis/python-logging
Description: Is your feature request related to a problem? Please describe. My company is moving out from structlog to start using this package, as it is intended to have better integration with google products. We're using StructuredLogHandler to se...
Open Graph Description: Is your feature request related to a problem? Please describe. My company is moving out from structlog to start using this package, as it is intended to have better integration with google products...
X Description: Is your feature request related to a problem? Please describe. My company is moving out from structlog to start using this package, as it is intended to have better integration with google products...
Opengraph URL: https://github.com/googleapis/python-logging/issues/656
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add support to custom `JSONEncoder`","articleBody":" **Is your feature request related to a problem? Please describe.**\r\nMy company is moving out from [structlog](https://www.structlog.org/en/stable/) to start using this package, as it is intended to have better integration with google products. We're using `StructuredLogHandler` to send the log records to google cloud logging. But something structlog used to do out-of-the-box was the ability to automatically convert the extra data we provide to the logger to be sent to the stream, and therefore, we need to check all the calls we're making to `logger.info()`, `debug()`, `warning()`, etc. to check if the data type is supported by JSONEncoder, otherwise the call to `hadler.format()` will raise `TypeError`, stating that `Object of type PosixPath is not JSON serializable` (this is one example of error we're getting, because the default encoder cannot encode `pathlib.Path` instances) and the log record will not be pushed to gcloud logging\r\n **Describe the solution you'd like**\r\nI'd like to be able to specify a custom class to encode the message to JSON, so the formatting is handled by the encoder, not the caller of `logger.info()` and its siblings\r\n **Describe alternatives you've considered**\r\nAn alternative would be having a filter that prepares the data to be used by the handler, I'm even wrote a filter to handle that:\r\n```python\r\nclass JsonableConverterFilter(logging.Filter):\r\n \"\"\"Prepare log record to be JSON encoded\r\n\r\n :class:`google.cloud.logging_v2.handlers.StructuredLogHandler` encodes directly to default JSON encoder without\r\n the possibility to set up a custom encoder, so we need to hydrate the log record to not break when emitting\r\n (because broken logs break silently)\r\n\r\n The ideal solution would be to have a LogFormatter, but as GCloud's log handler formats to JSON inside the handler,\r\n we need to circumvent this issue with a :class:`logging.Filter`\r\n \"\"\"\r\n\r\n def filter(self, record: logging.LogRecord) -\u003e bool:\r\n fields: Dict[str, Any] = getattr(record, \"json_fields\", {})\r\n\r\n for key, value in fields.items():\r\n fields[key] = JsonableConverterFilter.prepare_field(value)\r\n\r\n return True\r\n\r\n @classmethod\r\n def prepare_field(cls, value: Any) -\u003e TypeJSON:\r\n \"\"\"Converts the provided type into a plain JSON type\"\"\"\r\n # basic types\r\n if isinstance(value, (int, str, float, bool)) or value is None:\r\n return value\r\n\r\n try:\r\n iterator = iter(value)\r\n except TypeError:\r\n # probably a custom class\r\n if not hasattr(value, \"__dict__\"):\r\n return str(value)\r\n\r\n iterator = iter(value.__dict__)\r\n\r\n # hashmaps\r\n if hasattr(value, \"keys\") and hasattr(value, \"values\"):\r\n return {key: cls.prepare_field(val) for key, val in value.items()}\r\n\r\n # sequences\r\n return [cls.prepare_field(item) for item in iterator]\r\n\r\n```\r\nbut it seems to me I'm deviating the purpose of log filters by doing that\r\n **Additional context**\r\nn/a for now\r\n","author":{"url":"https://github.com/thiromi","@type":"Person","name":"thiromi"},"datePublished":"2022-10-25T11:03:01.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/656/python-logging/issues/656"}
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:08a3cc49-859a-de5d-2d21-a6470cafb3eb |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 928E:34DA5A:26EB46D:363AE5E:6A4F57E5 |
| html-safe-nonce | 492add88342b8fc0de145f411ac2da97b1da1895b58b5d4af12dd34be4e73247 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5MjhFOjM0REE1QToyNkVCNDZEOjM2M0FFNUU6NkE0RjU3RTUiLCJ2aXNpdG9yX2lkIjoiNjIwNjY0NTg3MjQ1NDc1MjIyOSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | ab5049566baa9b4d183c0dac881af78e6f7139286ac5e6cb16c86fe6c2a123b1 |
| hovercard-subject-tag | issue:1422279865 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/googleapis/python-logging/656/issue_layout |
| twitter:image | https://opengraph.githubassets.com/5f7872a543f4a5d9fc147f09b35306b3fe5e1a4a1ac47e6f3187e81b123f6770/googleapis/python-logging/issues/656 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/5f7872a543f4a5d9fc147f09b35306b3fe5e1a4a1ac47e6f3187e81b123f6770/googleapis/python-logging/issues/656 |
| og:image:alt | Is your feature request related to a problem? Please describe. My company is moving out from structlog to start using this package, as it is intended to have better integration with google products... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | thiromi |
| hostname | github.com |
| expected-hostname | github.com |
| None | b92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9 |
| turbo-cache-control | no-preview |
| go-import | github.com/googleapis/python-logging git https://github.com/googleapis/python-logging.git |
| octolytics-dimension-user_id | 16785467 |
| octolytics-dimension-user_login | googleapis |
| octolytics-dimension-repository_id | 226992562 |
| octolytics-dimension-repository_nwo | googleapis/python-logging |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 226992562 |
| octolytics-dimension-repository_network_root_nwo | googleapis/python-logging |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | b14b8c6d72fdb96ca48aba85dec94fe96ce5adbe |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width