René's URL Explorer Experiment


Title: Reduce allocations for MarkdownTextChunker by 80-95 % by gmanvel · Pull Request #9 · managedcode/graphrag · GitHub

Open Graph Title: Reduce allocations for MarkdownTextChunker by 80-95 % by gmanvel · Pull Request #9 · managedcode/graphrag

X Title: Reduce allocations for MarkdownTextChunker by 80-95 % by gmanvel · Pull Request #9 · managedcode/graphrag

Description: A modular graph-based Retrieval-Augmented Generation (RAG) system - Reduce allocations for MarkdownTextChunker by 80-95 % by gmanvel · Pull Request #9 · managedcode/graphrag

Open Graph Description: Description This PR dramatically reduces memory allocations in MarkdownTextChunker.Chunk() - a hot-path component in the GraphRAG indexing pipeline. The method is called for every document during t...

X Description: Description This PR dramatically reduces memory allocations in MarkdownTextChunker.Chunk() - a hot-path component in the GraphRAG indexing pipeline. The method is called for every document during t...

Opengraph URL: https://github.com/managedcode/graphrag/pull/9

X: @github

direct link

Domain: github.com

route-pattern/_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format)
route-controllervoltron_pull_requests_fragments
route-actionpull_request_layout
fetch-noncev2:15250649-a6d0-2a1e-ed7f-b178408bc289
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idC7C6:70240:EA1DFC:1463D50:6A4E94B2
html-safe-nonce65d235a97ebf62d66d2ea74ad65d18f9a39c77175d67533f37d1b965c638d740
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDN0M2OjcwMjQwOkVBMURGQzoxNDYzRDUwOjZBNEU5NEIyIiwidmlzaXRvcl9pZCI6IjE0MDY3NzEwMTI5MzQ5OTcxNzAiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac0cda737a584a76b2e19dde697e4b12b8187a9d5e069bd9346aa8ba95954ef8cb
hovercard-subject-tagpull_request:3089304744
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/pull_requests_fragments/pull_request_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/managedcode/graphrag/9/pull_request_layout
twitter:imagehttps://opengraph.githubassets.com/d957bfdc7193f615f3fa0ca8abf9c900957479fabcd33bf9ce561ec851c02ae3/managedcode/graphrag/pull/9
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/d957bfdc7193f615f3fa0ca8abf9c900957479fabcd33bf9ce561ec851c02ae3/managedcode/graphrag/pull/9
og:image:altDescription This PR dramatically reduces memory allocations in MarkdownTextChunker.Chunk() - a hot-path component in the GraphRAG indexing pipeline. The method is called for every document during t...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamegmanvel
hostnamegithub.com
expected-hostnamegithub.com
None41b6ab3ba6d20a71766ac245b5a4a94c6fc672a9cd4da7d44c1b33ab8bf6a21c
turbo-cache-controlno-cache
go-importgithub.com/managedcode/graphrag git https://github.com/managedcode/graphrag.git
octolytics-dimension-user_id78253589
octolytics-dimension-user_loginmanagedcode
octolytics-dimension-repository_id1083331438
octolytics-dimension-repository_nwomanagedcode/graphrag
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forktrue
octolytics-dimension-repository_parent_id778431525
octolytics-dimension-repository_parent_nwomicrosoft/graphrag
octolytics-dimension-repository_network_root_id778431525
octolytics-dimension-repository_network_root_nwomicrosoft/graphrag
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasee6a744804e8e70f97b4d5a18a94dcc63db22f97a
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/managedcode/graphrag/pull/9#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmanagedcode%2Fgraphrag%2Fpull%2F9
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmanagedcode%2Fgraphrag%2Fpull%2F9
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fpull_requests_fragments%2Fpull_request_layout&source=header-repo&source_repo=managedcode%2Fgraphrag
Reloadhttps://github.com/managedcode/graphrag/pull/9
Reloadhttps://github.com/managedcode/graphrag/pull/9
Reloadhttps://github.com/managedcode/graphrag/pull/9
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
managedcode https://github.com/managedcode
graphraghttps://github.com/managedcode/graphrag
microsoft/graphraghttps://github.com/microsoft/graphrag
Notifications https://github.com/login?return_to=%2Fmanagedcode%2Fgraphrag
Fork 10 https://github.com/login?return_to=%2Fmanagedcode%2Fgraphrag
Star 82 https://github.com/login?return_to=%2Fmanagedcode%2Fgraphrag
Code https://github.com/managedcode/graphrag
Issues 0 https://github.com/managedcode/graphrag/issues
Pull requests 1 https://github.com/managedcode/graphrag/pulls
Actions https://github.com/managedcode/graphrag/actions
Models https://github.com/managedcode/graphrag/models
Security and quality 0 https://github.com/managedcode/graphrag/security
Insights https://github.com/managedcode/graphrag/pulse
Code https://github.com/managedcode/graphrag
Issues https://github.com/managedcode/graphrag/issues
Pull requests https://github.com/managedcode/graphrag/pulls
Actions https://github.com/managedcode/graphrag/actions
Models https://github.com/managedcode/graphrag/models
Security and quality https://github.com/managedcode/graphrag/security
Insights https://github.com/managedcode/graphrag/pulse
KSemenenkohttps://github.com/KSemenenko
managedcode:mainhttps://github.com/managedcode/graphrag/tree/main
gmanvel:mainhttps://github.com/gmanvel/graphrag/tree/main
Conversationhttps://github.com/managedcode/graphrag/pull/9
Commits2 (2)https://github.com/managedcode/graphrag/pull/9/commits
Checkshttps://github.com/managedcode/graphrag/pull/9/checks
Files changedhttps://github.com/managedcode/graphrag/pull/9/files
Reduce allocations for MarkdownTextChunker by 80-95 %https://github.com/managedcode/graphrag/pull/9#top
KSemenenkohttps://github.com/KSemenenko
managedcode:mainhttps://github.com/managedcode/graphrag/tree/main
gmanvel:mainhttps://github.com/gmanvel/graphrag/tree/main
https://github.com/gmanvel
gmanvelhttps://github.com/gmanvel
Dec 10, 2025https://github.com/managedcode/graphrag/pull/9#issue-3715138858
https://private-user-images.githubusercontent.com/11509911/524822228-affbbe93-ec35-4730-8091-557fbb62a2a6.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODM1MzUwNzAsIm5iZiI6MTc4MzUzNDc3MCwicGF0aCI6Ii8xMTUwOTkxMS81MjQ4MjIyMjgtYWZmYmJlOTMtZWMzNS00NzMwLTgwOTEtNTU3ZmJiNjJhMmE2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA3MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwNzA4VDE4MTkzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVmNTQzMDhlZDI1ZTM2NDJlZGQ0ODA0OWJjOTVhODVjOGE3MzhjNGU2ODY3NThhMmJiZTlhNDFhMTEyYjIyZWMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.2brz6U76fkHhLNMydPRTxY_twrIZdSKp02NxrKOhdOQ
https://private-user-images.githubusercontent.com/11509911/524823192-9f320eb3-8547-46f7-9a98-80561e772b79.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODM1MzUwNzAsIm5iZiI6MTc4MzUzNDc3MCwicGF0aCI6Ii8xMTUwOTkxMS81MjQ4MjMxOTItOWYzMjBlYjMtODU0Ny00NmY3LTlhOTgtODA1NjFlNzcyYjc5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA3MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwNzA4VDE4MTkzMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThiYzdiMjhiYmViNDU1YjE1MmU2NDY4YTQ0NDVlMWU2NGM5NzllNjZhN2NmNzZlNjViZTRhMzhiODliYjk0NTEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.9sbVxdP8isWnfMF1gIxJEZrSiR58SAyn5ShV2deEAso
regression testshttps://github.com/gmanvel/graphrag/blob/perf/optimize-markdownchunker/tests/ManagedCode.GraphRag.Tests/Chunking/MarkdownTextChunkerTests.cs#L902
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
gmanvelhttps://github.com/gmanvel
December 10, 2025 13:14https://github.com/managedcode/graphrag/pull/9#commits-pushed-c6d82c8
https://github.com/gmanvel
Optimize the MarkdownTextChunk allocation footprinthttps://github.com/managedcode/graphrag/pull/9/commits/c6d82c844c6d04f28390b7633dceed4f847fc077
c6d82c8https://github.com/managedcode/graphrag/pull/9/commits/c6d82c844c6d04f28390b7633dceed4f847fc077
https://github.com/gmanvel
Cleanup leftover methodhttps://github.com/managedcode/graphrag/pull/9/commits/b6873ee0b86ae051c2c206091b2b3cc912449709
b6873eehttps://github.com/managedcode/graphrag/pull/9/commits/b6873ee0b86ae051c2c206091b2b3cc912449709
https://github.com/apps/codecov
codecovhttps://github.com/apps/codecov
Dec 10, 2025https://github.com/managedcode/graphrag/pull/9#issuecomment-3637039716
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
Codecovhttps://app.codecov.io/gh/managedcode/graphrag/pull/9?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
22e277dhttps://app.codecov.io/gh/managedcode/graphrag/commit/22e277d42dd014ade2053a4f9d81247cc3c6ba30?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
b6873eehttps://app.codecov.io/gh/managedcode/graphrag/commit/b6873ee0b86ae051c2c206091b2b3cc912449709?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
Files with missing lineshttps://app.codecov.io/gh/managedcode/graphrag/pull/9?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
...nagedCode.GraphRag/Chunking/MarkdownTextChunker.cshttps://app.codecov.io/gh/managedcode/graphrag/pull/9?src=pr&el=tree&filepath=src%2FManagedCode.GraphRag%2FChunking%2FMarkdownTextChunker.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode#diff-c3JjL01hbmFnZWRDb2RlLkdyYXBoUmFnL0NodW5raW5nL01hcmtkb3duVGV4dENodW5rZXIuY3M=
35 Missing and 6 partials ⚠️ https://app.codecov.io/gh/managedcode/graphrag/pull/9?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
☔ View full report in Codecov by Sentryhttps://app.codecov.io/gh/managedcode/graphrag/pull/9?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
Share it herehttps://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=managedcode
Test Analyticshttps://docs.codecov.com/docs/test-analytics
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/KSemenenko
KSemenenkohttps://github.com/KSemenenko
Dec 11, 2025https://github.com/managedcode/graphrag/pull/9#issuecomment-3641577151
@gmanvelhttps://github.com/gmanvel
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/KSemenenko
KSemenenkohttps://github.com/KSemenenko
dc9c98ehttps://github.com/managedcode/graphrag/commit/dc9c98ea9a025988b1074804e14b32e6ac6070de
Dec 11, 2025https://github.com/managedcode/graphrag/pull/9#event-21502390072
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/gmanvel
gmanvelhttps://github.com/gmanvel
Dec 11, 2025https://github.com/managedcode/graphrag/pull/9#issuecomment-3641863589
@KSemenenkohttps://github.com/KSemenenko
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/KSemenenko
KSemenenkohttps://github.com/KSemenenko
Dec 11, 2025https://github.com/managedcode/graphrag/pull/9#issuecomment-3641929106
@gmanvelhttps://github.com/gmanvel
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/gmanvel
gmanvelhttps://github.com/gmanvel
Dec 11, 2025https://github.com/managedcode/graphrag/pull/9#issuecomment-3642003935
@gmanvelhttps://github.com/gmanvel
https://devblogs.microsoft.com/dotnet/introducing-data-ingestion-building-blocks-preview/#split-your-data-into-chunkshttps://devblogs.microsoft.com/dotnet/introducing-data-ingestion-building-blocks-preview/#split-your-data-into-chunks
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.co/hiddenchars
https://github.com/managedcode/graphrag/pull/{{ revealButtonHref }}
Sign up for freehttps://github.com/join?source=comment-repo
Sign in to commenthttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmanagedcode%2Fgraphrag%2Fpull%2F9
Please reload this pagehttps://github.com/managedcode/graphrag/pull/9
https://github.com/gmanvel
https://github.com/KSemenenko
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.