René's URL Explorer Experiment


Title: Thinking about improving data updates · Issue #647 · plotly/plotly.js · GitHub

Open Graph Title: Thinking about improving data updates · Issue #647 · plotly/plotly.js

X Title: Thinking about improving data updates · Issue #647 · plotly/plotly.js

Description: tl;dr There's more and more code that couples the aspect of plotting logic with the aspect of incrementally propagating changes, e.g. see all things going on in Plotly.restyle. Would be good to discuss ways to improve on the situation. M...

Open Graph Description: tl;dr There's more and more code that couples the aspect of plotting logic with the aspect of incrementally propagating changes, e.g. see all things going on in Plotly.restyle. Would be good to dis...

X Description: tl;dr There's more and more code that couples the aspect of plotting logic with the aspect of incrementally propagating changes, e.g. see all things going on in Plotly.restyle. Would be good to...

Opengraph URL: https://github.com/plotly/plotly.js/issues/647

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Thinking about improving data updates","articleBody":"**tl;dr**\nThere's more and more code that couples the aspect of plotting logic with the aspect of incrementally propagating changes, e.g. see all things going on in `Plotly.restyle`. Would be good to discuss ways to improve on the situation. Manual code leads to a tangle and some small, simple library focused on change propagation e.g. [MobX](https://docs.google.com/presentation/d/1tP0VWjproXsZex3R8zWCA7Bra7RtMcu4daJXCUbs8pc/edit#slide=id.g1102c17e6b_0_31) would be worth looking into.\n\n**Plotting turns a stream of user intent into a stream of side effects such as DOM updates**\n\nPlotting can be conceived of as a black box:\n- input streams are plot specifications, typically the payloads in `Plotly.plot`, `Plotly.restyle`, `Plotly.relayout`, animation inducing user calls as well as DOM events such as `window.resize` and `mousedown`\n- output is a stream of side effecting operations, e.g. DOM mutations, WebGL API calls, and sometimes event callbacks\n- currently, some output is provided by encouraging users to read directly from internal object state but it's something to move away from, by providing a query API and/or event callbacks with meaningful data, so I'll ignore this\n\nThe use of 'stream' highlights the fact that with user pointer operations, restyle/relayout, animation etc. _generally_ make plotting a temporal process, rather than something that can be modeled with a function with some input JSON and an output SVG - even if some of the uses are as simple as this special case.\n\n**Plotting logic is a directed acyclic graph of computation nodes**\n\nWe have multiple pieces of input (e.g. `data[0].x`) at the input and DOM mutating calls as the output. However there's complex calculation in the middle that can be thought of as a DAG. For example, \n- the above `x` vector serves as the basis for calculating a `[min, max]` domain that will determine the bounds of the X axis\n- the `x` vector is also trivially input to scatter point positions, however, a `scale` transform converts domain values to e.g. pixel coordinates\n- for things like the boxplot, there may be various aggregations building atop of the `x` vector\n- aesthetics might depend on things like how long the `x` vector is; maybe defaulting from scatterplot to a density plot at some threshold\n\nAll such calculations themselves can be input to downstream calculations.\n\n**Plotting needs to be economical**\n\nWhile it would be possible to make a single function whose inputs are `{domRoot, userIntentHistory}`, it's impractical: response times with a naive implementation would be too high (keeping `userIntentHistory is merely of modest size impact). There's no way to recompute everything from scratch and expect a 60FPS frame rate when turning a WebGL plot or animating something.\n\nThis means that there needs to be some kind of caching, therefore state management. The sole purpose of maintaining state is caching (besides this, we may retain `userIntentHistory` to allow time travel, and of course the output streams are linked to calls that modify the DOM).\n\n**Means of reducing recomputation costs**\n\nIdeally we'd like to \n- Only recompute what's strictly needed. For example, if I add a new highest value to vector `x` it needs to lead to an increased visible X axis domain, provided it's set to automatic. However, if the newly inserted value is inside the bounds, there's no need to recalculate anything that depends only on the `[min, max]` domain. Sure, sometimes there's no harm due to speed of recalculation or lack of need for speed, but there _are_ cases when it's useful to be fairly granular about recalculations due to some specific performance need. Solving these specific performance needs one by one, without a formal change propagation approach is brittle.\n- It may even be useful, necessary and easy to pick calculation algorithms to be incremental. For example, a newly arriving X value can be directly used to update the `[min, max]` bounds, as opposed to inserting it in the preexisting large vector and applying the vector `extent` calculation. Similarly, many types of aggregates can be calculated _on-line_ as well as _batch_. For example, mean, variance and standard deviation.\n\n**Some possible tools**\n\nHandwritten userland JavaScript isn't quite good for managing a dependency graph, because given enough nodes and optimization rounds, there will be inevitable cache invalidation issues, and potentially, memory leaks. Keeping things consistent and in in sync is also a challenge especially in the presence of asynchronous events. Most importantly, coupling the plot logic aspect with the incremental recalculation aspect makes both aspects hard to decipher, debug and further develop.\n\nThere are a lot of tools that provide some kind of framework for calculating and propagating values that can change over time, responding to input, inspired by Functional Reactive Programming. Without endorsing any of these excellent libraries ([xstream](https://github.com/staltz/xstream), [most.js](https://github.com/cujojs/most) etc.) perhaps [MobX](https://github.com/mobxjs/mobx) would feel closest to the current architecture in that it gives you objects that have properties acting like calculated spreadsheet cells, and as @etpinard suggested in the 2.0 wishlist, object-oriented, but investigation would be needed to see how it fits. All these libs are around 10k compressed.\n\n**History**\n\nWe've touched on related topics in the past; a few inspirations:\n- Wishlist suggestions such as using more OO; not storing data in the DOM; using pure functions; more complete data in callbacks: https://github.com/plotly/plotly.js/issues/420\n- Customer filed PRs for faster (re)calculation and way faster incremental calculation, e.g. on large WebGL meshes\n- Most of the Plotly.restyle function, whose 500 lines do heavy amounts of manual work, e.g. https://github.com/plotly/plotly.js/blob/master/src/plot_api/plot_api.js#L1736-L1814 and https://github.com/plotly/plotly.js/pull/617#discussion_r67130816\n- Previous discussions e.g. on the animation topic\n","author":{"url":"https://github.com/monfera","@type":"Person","name":"monfera"},"datePublished":"2016-06-15T11:39:19Z","url":"https://github.com/plotly/plotly.js/issues/647"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:32b7ce73-8090-8139-bee7-21e3e5760e42
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id8C8A:2D40DE:1C0ABC4:23A9285:6B019728
html-safe-nonce742288643e39521a2b0a8c7ae1962479cb077f8fe724538ef080f166f22c8647
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4QzhBOjJENDBERToxQzBBQkM0OjIzQTkyODU6NkIwMTk3MjgiLCJ2aXNpdG9yX2lkIjoiMTI3OTc5MTc4MTI4Njc2NjQ4IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac6390cf1e688cac6f8ca14b6632dd7b0db8b2b73a0b4167cedbff8c81c075bd0b
hovercard-subject-tagissue:160401302
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/plotly/plotly.js/647/issue_layout
twitter:imagehttps://opengraph.githubassets.com/f8926cf609a7bd64895cec2012d5e4f4fdef0478237274d13b5cd2e2f1f738be/plotly/plotly.js/issues/647
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/f8926cf609a7bd64895cec2012d5e4f4fdef0478237274d13b5cd2e2f1f738be/plotly/plotly.js/issues/647
og:image:alttl;dr There's more and more code that couples the aspect of plotting logic with the aspect of incrementally propagating changes, e.g. see all things going on in Plotly.restyle. Would be good to dis...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamemonfera
hostnamegithub.com
expected-hostnamegithub.com
Nonecc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c
turbo-cache-controlno-preview
go-importgithub.com/plotly/plotly.js git https://github.com/plotly/plotly.js.git
octolytics-dimension-user_id5997976
octolytics-dimension-user_loginplotly
octolytics-dimension-repository_id45646037
octolytics-dimension-repository_nwoplotly/plotly.js
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id45646037
octolytics-dimension-repository_network_root_nwoplotly/plotly.js
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8c77c9e8439031ad54c75a250694842892f693e1
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/plotly/plotly.js/issues/647#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fplotly%2Fplotly.js%2Fissues%2F647
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fplotly%2Fplotly.js%2Fissues%2F647
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=plotly%2Fplotly.js
Reloadhttps://github.com/plotly/plotly.js/issues/647
Reloadhttps://github.com/plotly/plotly.js/issues/647
Reloadhttps://github.com/plotly/plotly.js/issues/647
plotlyhttps://github.com/plotly
plotly.jshttps://github.com/plotly/plotly.js
Codehttps://github.com/plotly/plotly.js
Issues726 (726)https://github.com/plotly/plotly.js/issues
Pull requests84 (84)https://github.com/plotly/plotly.js/pulls
Actionshttps://github.com/plotly/plotly.js/actions
Security and qualityhttps://github.com/plotly/plotly.js/security
Insightshttps://github.com/plotly/plotly.js/insights
https://github.com/plotly/plotly.js
New issuehttps://github.com/login?return_to=https://github.com/plotly/plotly.js/issues/647
New issuehttps://github.com/login?return_to=https://github.com/plotly/plotly.js/issues/647
Thinking about improving data updateshttps://github.com/plotly/plotly.js/issues/647#top
https://github.com/monfera
monferahttps://github.com/monfera
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#issue-160401302
MobXhttps://docs.google.com/presentation/d/1tP0VWjproXsZex3R8zWCA7Bra7RtMcu4daJXCUbs8pc/edit#slide=id.g1102c17e6b_0_31
xstreamhttps://github.com/staltz/xstream
most.jshttps://github.com/cujojs/most
MobXhttps://github.com/mobxjs/mobx
@etpinardhttps://github.com/etpinard
wishlist for potential breaking changes since v1 #420https://github.com/plotly/plotly.js/issues/420
https://github.com/plotly/plotly.js/blob/master/src/plot_api/plot_api.js#L1736-L1814https://github.com/plotly/plotly.js/blob/master/src/plot_api/plot_api.js#L1736-L1814
Color gradient for scatter3d snail trail lines [WIP] #617 (comment)https://github.com/plotly/plotly.js/pull/617#discussion_r67130816
mdtuszhttps://github.com/mdtusz
status: discussion neededhttps://github.com/plotly/plotly.js/issues?q=state%3Aopen%20label%3A%22status%3A%20discussion%20needed%22
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#event-693283655
https://github.com/etpinard
etpinardhttps://github.com/etpinard
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226208945
@monferahttps://github.com/monfera
https://github.com/mdtusz
mdtuszhttps://github.com/mdtusz
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226211312
https://github.com/monfera
monferahttps://github.com/monfera
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226323494
axes, lineshttp://bl.ocks.org/nsonnad/4175202
pointshttps://2804c37c4fc0e52ab1e7443363b203c03d956048-www.googledrive.com/host/0BwcmXsFAhzhoRWh5SjNyV2hLNlE/dashboard.html
is it this one?http://husk.eecs.berkeley.edu/courses/cs260-fall10/images/3/3a/FinalPaper-HongLiuShin.pdf
https://github.com/monfera
monferahttps://github.com/monfera
on Jun 15, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226326804
@mdtuszhttps://github.com/mdtusz
https://github.com/mdtusz
mdtuszhttps://github.com/mdtusz
on Jun 16, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226561014
https://github.com/monfera
monferahttps://github.com/monfera
on Jun 18, 2016https://github.com/plotly/plotly.js/issues/647#issuecomment-226969155
#648https://github.com/plotly/plotly.js/issues/648
monferahttps://github.com/monfera
completedhttps://github.com/plotly/plotly.js/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Jun 18, 2016https://github.com/plotly/plotly.js/issues/647#event-696830687
Sign up for freehttps://github.com/signup?return_to=https://github.com/plotly/plotly.js/issues/647
Sign in to commenthttps://github.com/login?return_to=https://github.com/plotly/plotly.js/issues/647
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.