Title: RemoteException causes IPython 8 to crash · Issue #13538 · ipython/ipython · GitHub
Open Graph Title: RemoteException causes IPython 8 to crash · Issue #13538 · ipython/ipython
X Title: RemoteException causes IPython 8 to crash · Issue #13538 · ipython/ipython
Description: Exceptions in subprocesses in multiprocessing.pool.Pool and concurrent.futures.ProcessPoolExecutor (and possibly more) cause IPython 8 to crash. At first I thought this might be a duplicate of #12467 but while the symptoms are similar, t...
Open Graph Description: Exceptions in subprocesses in multiprocessing.pool.Pool and concurrent.futures.ProcessPoolExecutor (and possibly more) cause IPython 8 to crash. At first I thought this might be a duplicate of #124...
X Description: Exceptions in subprocesses in multiprocessing.pool.Pool and concurrent.futures.ProcessPoolExecutor (and possibly more) cause IPython 8 to crash. At first I thought this might be a duplicate of #124...
Opengraph URL: https://github.com/ipython/ipython/issues/13538
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"RemoteException causes IPython 8 to crash","articleBody":"Exceptions in subprocesses in `multiprocessing.pool.Pool` and `concurrent.futures.ProcessPoolExecutor` (and possibly more) cause IPython 8 to crash.\r\n\r\nAt first I thought this might be a duplicate of #12467 but while the symptoms are similar, the cause seems to be different\r\n\r\nIn `IPython.core.ultratb.VerboseTB.structured_traceback` exists a while loop which loops over the chain of exceptions. If it stems from a subprocess though, the type of the traceback might be `RemoteTraceback` (or `_RemoteTraceback` for `ProcessPoolExecutor`) instead of a regular traceback object which has neither a `__cause__` nor a `__context__`. This in turn leads to `etb` being `None` in `structured_traceback` and an exception being triggered by the newly introduced `assert etb is not None` in `format_exception_as_a_whole` which leads to the crash.\r\n\r\n## Steps to reproduce:\r\n\r\n```python\r\nfrom multiprocessing.pool import Pool\r\n\r\ndef foo(x):\r\n raise Exception()\r\n\r\nPool(1).map(foo, [1])\r\n```\r\n\r\nTested in latest Python 3.10 docker container with IPython 8.0.1:\r\n\r\n\u003cdetails>\r\n\r\n```python\r\nroot@0d7056b87b09:/# ipython\r\nPython 3.10.2 (main, Feb 8 2022, 04:44:29) [GCC 10.2.1 20210110]\r\nType 'copyright', 'credits' or 'license' for more information\r\nIPython 8.0.1 -- An enhanced Interactive Python. Type '?' for help.\r\n\r\nIn [1]: from multiprocessing.pool import Pool\r\n\r\nIn [2]: def foo(x):\r\n ...: raise Exception()\r\n\r\nIn [3]: Pool(1).map(foo, [1])\r\nmultiprocessing.pool.RemoteTraceback: \r\n\"\"\"\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/multiprocessing/pool.py\", line 125, in worker\r\n result = (True, func(*args, **kwds))\r\n File \"/usr/local/lib/python3.10/multiprocessing/pool.py\", line 48, in mapstar\r\n return list(map(*args))\r\n File \"\u003cipython-input-2-f1f53f013b9a>\", line 2, in foo\r\n raise Exception()\r\nException\r\n\"\"\"\r\n\r\nThe above exception was the direct cause of the following exception:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3251, in run_code\r\n exec(code_obj, self.user_global_ns, self.user_ns)\r\n File \"\u003cipython-input-3-6128068d2903>\", line 1, in \u003cmodule>\r\n Pool(1).map(foo, [1])\r\n File \"/usr/local/lib/python3.10/multiprocessing/pool.py\", line 364, in map\r\n return self._map_async(func, iterable, mapstar, chunksize).get()\r\n File \"/usr/local/lib/python3.10/multiprocessing/pool.py\", line 771, in get\r\n raise self._value\r\nException\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'Exception' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3191, in run_ast_nodes\r\n if await self.run_code(code, result, async_=asy):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3268, in run_code\r\n self.showtraceback(running_compiled_code=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'AssertionError' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2814, in _run_cell\r\n return runner(coro)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/async_helpers.py\", line 129, in _pseudo_sync_runner\r\n coro.send(None)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3012, in run_cell_async\r\n has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3210, in run_ast_nodes\r\n self.showtraceback()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'AssertionError' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/ipython\", line 8, in \u003cmodule>\r\n sys.exit(start_ipython())\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/__init__.py\", line 123, in start_ipython\r\n return launch_new_instance(argv=argv, **kwargs)\r\n File \"/usr/local/lib/python3.10/site-packages/traitlets/config/application.py\", line 846, in launch_instance\r\n app.start()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/ipapp.py\", line 316, in start\r\n self.shell.mainloop()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py\", line 611, in mainloop\r\n self.interact()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py\", line 604, in interact\r\n self.run_cell(code, store_history=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2768, in run_cell\r\n result = self._run_cell(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2819, in _run_cell\r\n self.showtraceback(running_compiled_code=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nIf you suspect this is an IPython 8.0.1 bug, please report it at:\r\n https://github.com/ipython/ipython/issues\r\nor send an email to the mailing list at ipython-dev@python.org\r\n\r\nYou can print a more detailed traceback right now with \"%tb\", or use \"%debug\"\r\nto interactively debug it.\r\n\r\nExtra-detailed tracebacks for bug-reporting purposes can be enabled via:\r\n %config Application.verbose_crash=True\r\n\r\nroot@0d7056b87b09:/# \r\n```\r\n\r\n\u003c/details>\r\n\r\nAlso with `ProcessPoolExecutor`:\r\n\r\n```python\r\nfrom concurrent.futures import ProcessPoolExecutor\r\n\r\ndef foo():\r\n raise Exception(\"foo\")\r\n\r\npool = ProcessPoolExecutor(max_workers=1)\r\nfuture = pool.submit(foo)\r\nfuture.result()\r\n```\r\n\r\n\r\n\u003cdetails>\r\n\r\n```python\r\nroot@8a840b5a619f:/# ipython\r\nPython 3.10.2 (main, Feb 8 2022, 04:44:29) [GCC 10.2.1 20210110]\r\nType 'copyright', 'credits' or 'license' for more information\r\nIPython 8.0.1 -- An enhanced Interactive Python. Type '?' for help.\r\n\r\nIn [1]: from concurrent.futures import ProcessPoolExecutor\r\n ...: \r\n ...: \r\n ...: def foo():\r\n ...: raise Exception(\"foo\")\r\n ...: \r\n ...: \r\n ...: pool = ProcessPoolExecutor(max_workers=1)\r\n ...: future = pool.submit(foo)\r\n ...: future.result()\r\nconcurrent.futures.process._RemoteTraceback: \r\n\"\"\"\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/concurrent/futures/process.py\", line 243, in _process_worker\r\n r = call_item.fn(*call_item.args, **call_item.kwargs)\r\n File \"\u003cipython-input-1-86d971dad3a2>\", line 5, in foo\r\n raise Exception(\"foo\")\r\nException: foo\r\n\"\"\"\r\n\r\nThe above exception was the direct cause of the following exception:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3251, in run_code\r\n exec(code_obj, self.user_global_ns, self.user_ns)\r\n File \"\u003cipython-input-1-86d971dad3a2>\", line 10, in \u003cmodule>\r\n future.result()\r\n File \"/usr/local/lib/python3.10/concurrent/futures/_base.py\", line 446, in result\r\n return self.__get_result()\r\n File \"/usr/local/lib/python3.10/concurrent/futures/_base.py\", line 391, in __get_result\r\n raise self._exception\r\nException: foo\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'Exception' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3191, in run_ast_nodes\r\n if await self.run_code(code, result, async_=asy):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3268, in run_code\r\n self.showtraceback(running_compiled_code=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'AssertionError' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2814, in _run_cell\r\n return runner(coro)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/async_helpers.py\", line 129, in _pseudo_sync_runner\r\n coro.send(None)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3012, in run_cell_async\r\n has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 3210, in run_ast_nodes\r\n self.showtraceback()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1934, in showtraceback\r\n stb = value._render_traceback_()\r\nAttributeError: 'AssertionError' object has no attribute '_render_traceback_'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/ipython\", line 8, in \u003cmodule>\r\n sys.exit(start_ipython())\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/__init__.py\", line 123, in start_ipython\r\n return launch_new_instance(argv=argv, **kwargs)\r\n File \"/usr/local/lib/python3.10/site-packages/traitlets/config/application.py\", line 846, in launch_instance\r\n app.start()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/ipapp.py\", line 316, in start\r\n self.shell.mainloop()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py\", line 611, in mainloop\r\n self.interact()\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py\", line 604, in interact\r\n self.run_cell(code, store_history=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2768, in run_cell\r\n result = self._run_cell(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 2819, in _run_cell\r\n self.showtraceback(running_compiled_code=True)\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/interactiveshell.py\", line 1936, in showtraceback\r\n stb = self.InteractiveTB.structured_traceback(etype,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 1105, in structured_traceback\r\n return FormattedTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 999, in structured_traceback\r\n return VerboseTB.structured_traceback(\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 871, in structured_traceback\r\n formatted_exceptions += self.format_exception_as_a_whole(etype, evalue, etb, lines_of_context,\r\n File \"/usr/local/lib/python3.10/site-packages/IPython/core/ultratb.py\", line 775, in format_exception_as_a_whole\r\n assert etb is not None\r\nAssertionError\r\n\r\nIf you suspect this is an IPython 8.0.1 bug, please report it at:\r\n https://github.com/ipython/ipython/issues\r\nor send an email to the mailing list at ipython-dev@python.org\r\n\r\nYou can print a more detailed traceback right now with \"%tb\", or use \"%debug\"\r\nto interactively debug it.\r\n\r\nExtra-detailed tracebacks for bug-reporting purposes can be enabled via:\r\n %config Application.verbose_crash=True\r\n\r\nroot@8a840b5a619f:/# \r\n```\r\n\r\n\u003c/details>\r\n\r\n","author":{"url":"https://github.com/jstucke","@type":"Person","name":"jstucke"},"datePublished":"2022-02-21T11:32:42Z","url":"https://github.com/ipython/ipython/issues/13538"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:f4dd0b1b-dd6c-ad3e-b6d5-172fde082470 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D4C6:134361:57C8BF:75D73E:6B00CF83 |
| html-safe-nonce | eb42769a910186e2ee13e17d4489d8880ea9db2ea4c43ea7feacb48197466504 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENEM2OjEzNDM2MTo1N0M4QkY6NzVENzNFOjZCMDBDRjgzIiwidmlzaXRvcl9pZCI6IjE5OTQzNzY4ODMyNjA2NzM5MjMiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 8665af939bf0e3a7143afdb7121b08bb97aa339f4e8d56344c7b6dce69dbee88 |
| hovercard-subject-tag | issue:1145679525 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/ipython/ipython/13538/issue_layout |
| twitter:image | https://opengraph.githubassets.com/fc5a206ea2f27cf975a8659c38a7b77c49c67ebcf7fe57b16c48e5e64de45b9e/ipython/ipython/issues/13538 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/fc5a206ea2f27cf975a8659c38a7b77c49c67ebcf7fe57b16c48e5e64de45b9e/ipython/ipython/issues/13538 |
| og:image:alt | Exceptions in subprocesses in multiprocessing.pool.Pool and concurrent.futures.ProcessPoolExecutor (and possibly more) cause IPython 8 to crash. At first I thought this might be a duplicate of #124... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | jstucke |
| hostname | github.com |
| expected-hostname | github.com |
| None | 85bbdfe2b6ba16e234513bc6ceff5d848ef8449820c900692a2b4e67061931e5 |
| turbo-cache-control | no-preview |
| go-import | github.com/ipython/ipython git https://github.com/ipython/ipython.git |
| octolytics-dimension-user_id | 230453 |
| octolytics-dimension-user_login | ipython |
| octolytics-dimension-repository_id | 658518 |
| octolytics-dimension-repository_nwo | ipython/ipython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 658518 |
| octolytics-dimension-repository_network_root_nwo | ipython/ipython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4943239a6d2c591325dad6e5c313e1cafda877f6 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width