René's URL Explorer Experiment


Title: Lazy restoration of cached modules · Issue #1728 · microsoft/python-language-server · GitHub

Open Graph Title: Lazy restoration of cached modules · Issue #1728 · microsoft/python-language-server

X Title: Lazy restoration of cached modules · Issue #1728 · microsoft/python-language-server

Description: Currently module analysis is restored from the database as regular analysis with dependencies is running. I.e., instead of evaluating code, result is read from the database. This allowed to integrate caching into the existing analysis wo...

Open Graph Description: Currently module analysis is restored from the database as regular analysis with dependencies is running. I.e., instead of evaluating code, result is read from the database. This allowed to integra...

X Description: Currently module analysis is restored from the database as regular analysis with dependencies is running. I.e., instead of evaluating code, result is read from the database. This allowed to integra...

Opengraph URL: https://github.com/microsoft/python-language-server/issues/1728

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Lazy restoration of cached modules","articleBody":"Currently module analysis is restored from the database as regular analysis with dependencies is running. I.e., instead of evaluating code, result is read from the database. This allowed to integrate caching into the existing analysis workflow (the dependency walk) easily, but, unfortunately, there is a drawback.\r\n\r\nNamely, often IO operations (i.e. reading from the database) take almost as long as the original analysis. It is not the database performance issue, raw serialization into flat binary file and reading back is not much faster than binary JSON used by the LiteDB. \r\n\r\nChanges:\r\n- Instead of restoring everything, only main module (such as numpy) is restored. Module the claims to have no dependencies so analysis does not run on its submodules. This should provide first level of completions. \r\n- Types coming from submodules should be kept lazy – for example, class that comes from a submodule, remains lazy. When deeper member is requested, such as when user types ‘numpy.A.’ where ‘A’ is from ‘numpy.core’, then ‘numpy.core’ analysis is restored from the db, and so on – on demand. \r\n- Types that never used may never be restored yielding memory savings. Obviously, there may be hit for really complex members than chain multiple modules, but it should be one time hit. \r\n- Top-level module database will then need to store more information about inner types (such as function signature and documentation rather than relying on fetching those off the inner restored type).\r\n\r\nThe approach does not require changes to the analysis workflow. If module is cached, it will be restored and if analysis of some non-cached module requires deeper members, process is the same as if user requested said members.\r\n","author":{"url":"https://github.com/MikhailArkhipov","@type":"Person","name":"MikhailArkhipov"},"datePublished":"2019-10-31T15:25:00Z","url":"https://github.com/microsoft/python-language-server/issues/1728"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:e4d345a9-2a7c-f6f8-59e5-c322f72d3a0e
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id8BBA:5B76F:4FDCC6:6BEBED:6AFE5C15
html-safe-nonceb4c47b4718df32cb3f4710d4868149e6feb0a2699a6fd3beedfae31952082442
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4QkJBOjVCNzZGOjRGRENDNjo2QkVCRUQ6NkFGRTVDMTUiLCJ2aXNpdG9yX2lkIjoiOTAwNjg2MDg0NzQ3Mjg2MDE4MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac2a43e20fef6a07430aa75046e3d8b69081e411927218f8dbb73bcefce15319e5
hovercard-subject-tagissue:515522019
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/python-language-server/1728/issue_layout
twitter:imagehttps://opengraph.githubassets.com/23a28b0eb458e9a0ceed24a67baf4a42b243aa3d749e65172b62f774e2832983/microsoft/python-language-server/issues/1728
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/23a28b0eb458e9a0ceed24a67baf4a42b243aa3d749e65172b62f774e2832983/microsoft/python-language-server/issues/1728
og:image:altCurrently module analysis is restored from the database as regular analysis with dependencies is running. I.e., instead of evaluating code, result is read from the database. This allowed to integra...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameMikhailArkhipov
hostnamegithub.com
expected-hostnamegithub.com
Noneb3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86
turbo-cache-controlno-preview
go-importgithub.com/microsoft/python-language-server git https://github.com/microsoft/python-language-server.git
octolytics-dimension-user_id6154722
octolytics-dimension-user_loginmicrosoft
octolytics-dimension-repository_id145901977
octolytics-dimension-repository_nwomicrosoft/python-language-server
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id145901977
octolytics-dimension-repository_network_root_nwomicrosoft/python-language-server
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4a10310af51a1bfffbedc4fd81cb582acf0cb0ff
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/microsoft/python-language-server/issues/1728#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fpython-language-server%2Fissues%2F1728
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fpython-language-server%2Fissues%2F1728
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=microsoft%2Fpython-language-server
Reloadhttps://github.com/microsoft/python-language-server/issues/1728
Reloadhttps://github.com/microsoft/python-language-server/issues/1728
Reloadhttps://github.com/microsoft/python-language-server/issues/1728
microsofthttps://github.com/microsoft
python-language-serverhttps://github.com/microsoft/python-language-server
Codehttps://github.com/microsoft/python-language-server
Issues309 (309)https://github.com/microsoft/python-language-server/issues
Pull requests6 (6)https://github.com/microsoft/python-language-server/pulls
Projectshttps://github.com/microsoft/python-language-server/projects
Wikihttps://github.com/microsoft/python-language-server/wiki
Security and qualityhttps://github.com/microsoft/python-language-server/security
Insightshttps://github.com/microsoft/python-language-server/insights
https://github.com/microsoft/python-language-server
#1757https://github.com/microsoft/python-language-server/pull/1757
Lazy restoration of cached moduleshttps://github.com/microsoft/python-language-server/issues/1728#top
#1757https://github.com/microsoft/python-language-server/pull/1757
https://github.com/MikhailArkhipov
enhancementNew feature or requesthttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22enhancement%22
feature: cachinghttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22feature%3A%20caching%22
performancehttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22performance%22
November 2019.1https://github.com/microsoft/python-language-server/milestone/29
https://github.com/MikhailArkhipov
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
on Oct 31, 2019https://github.com/microsoft/python-language-server/issues/1728#issue-515522019
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
enhancementNew feature or requesthttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22enhancement%22
performancehttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22performance%22
feature: cachinghttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22feature%3A%20caching%22
on Oct 31, 2019https://github.com/microsoft/python-language-server/issues/1728#event-2760274639
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
November 2019.1https://github.com/microsoft/python-language-server/milestone/29
on Oct 31, 2019https://github.com/microsoft/python-language-server/issues/1728#event-2760274651
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
on Oct 31, 2019https://github.com/microsoft/python-language-server/issues/1728#event-2760274669
https://github.com/MikhailArkhipov
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
on Nov 5, 2019https://github.com/microsoft/python-language-server/issues/1728#issuecomment-549667429
https://user-images.githubusercontent.com/12820357/68180608-33e74200-ff49-11e9-8b4d-36239ab728cf.png
https://github.com/MikhailArkhipov
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
on Nov 6, 2019https://github.com/microsoft/python-language-server/issues/1728#issuecomment-550137992
#1753https://github.com/microsoft/python-language-server/issues/1753
https://github.com/MikhailArkhipov
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
on Nov 8, 2019https://github.com/microsoft/python-language-server/issues/1728#issuecomment-551972880
#1533https://github.com/microsoft/python-language-server/issues/1533
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
completedhttps://github.com/microsoft/python-language-server/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
#1757https://github.com/microsoft/python-language-server/pull/1757
on Dec 9, 2019https://github.com/microsoft/python-language-server/issues/1728#event-2867948840
Sign up for freehttps://github.com/signup?return_to=https://github.com/microsoft/python-language-server/issues/1728
Sign in to commenthttps://github.com/login?return_to=https://github.com/microsoft/python-language-server/issues/1728
Mikhail Arkhipov (MikhailArkhipov)https://github.com/MikhailArkhipov
enhancementNew feature or requesthttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22enhancement%22
feature: cachinghttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22feature%3A%20caching%22
performancehttps://github.com/microsoft/python-language-server/issues?q=state%3Aopen%20label%3A%22performance%22
November 2019.1https://github.com/microsoft/python-language-server/milestone/29
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.