René's URL Explorer Experiment


Title: MaD: update generator scripts to support writing JSON data extensions by nickrolfe · Pull Request #22798 · github/codeql · GitHub

Open Graph Title: MaD: update generator scripts to support writing JSON data extensions by nickrolfe · Pull Request #22798 · github/codeql

X Title: MaD: update generator scripts to support writing JSON data extensions by nickrolfe · Pull Request #22798 · github/codeql

Description: This is split into two commits: Updates the scripts so that they store and build models using an in-memory representation that is independent of the YAML format currently used. This means rows are represented as Python tuples, where previously they were created as YAML-formatted strings during the merging process. Scalar values inside those tuples are represented as Python strings and booleans. Only the DataExtensionFile.write_yaml function now has to deal with YAML formatting concerns. Adds support for writing JSON and choosing which format to use. For now, YAML is still the default. We can change this when CLI support for JSON extensions is widely available and the existing models in this repo have been converted to JSON. (I will be making a separate PR with a conversion script). I tested these changes by running both misc/scripts/models-as-data/generate_mad.py and java/ql/src/utils/modelconverter/ConvertExtensions.py and manually inspecting the output.

Open Graph Description: This is split into two commits: Updates the scripts so that they store and build models using an in-memory representation that is independent of the YAML format currently used. This means rows are...

X Description: This is split into two commits: Updates the scripts so that they store and build models using an in-memory representation that is independent of the YAML format currently used. This means rows are...

Opengraph URL: https://github.com/github/codeql/pull/22798

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:d3443f2b-7f93-51ce-1061-d41f584dedea
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idE4F8:72087:1477882:1B164B2:6AF8912B
html-safe-nonce904e23a66aac66ecf70d672d16de8db1b91ec869dc3a19d7c0a6504be1a90961
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNEY4OjcyMDg3OjE0Nzc4ODI6MUIxNjRCMjo2QUY4OTEyQiIsInZpc2l0b3JfaWQiOiI4NjIwMDYwNzE0MTc5NTMxMDUxIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmaca095cdc269d3fd0c5b536771d1b9a5fb7caa3b8c1971522a8c5fb672abe05c9f
hovercard-subject-tagpull_request:4802376918
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/github/codeql/pull/22798/files
twitter:imagehttps://avatars.githubusercontent.com/u/158306?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/158306?s=400&v=4
og:image:altThis is split into two commits: Updates the scripts so that they store and build models using an in-memory representation that is independent of the YAML format currently used. This means rows are...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None612d3e6ffd693e7ea8d9fcf452370995872baef31d825b1fedef704cd3d24ac7
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/github/codeql git https://github.com/github/codeql.git
octolytics-dimension-user_id9919
octolytics-dimension-user_logingithub
octolytics-dimension-repository_id143040428
octolytics-dimension-repository_nwogithub/codeql
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id143040428
octolytics-dimension-repository_network_root_nwogithub/codeql
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release072e34035c13c31543fba9f025cbbbda4210dae4
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/github/codeql/pull/22798/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F22798%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgithub%2Fcodeql%2Fpull%2F22798%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=github%2Fcodeql
Reloadhttps://github.com/github/codeql/pull/22798/files
Reloadhttps://github.com/github/codeql/pull/22798/files
Reloadhttps://github.com/github/codeql/pull/22798/files
githubhttps://github.com/github
codeqlhttps://github.com/github/codeql
Codehttps://github.com/github/codeql
Issues1k (1k)https://github.com/github/codeql/issues
Pull requests516 (516)https://github.com/github/codeql/pulls
Discussionshttps://github.com/github/codeql/discussions
Actionshttps://github.com/github/codeql/actions
Projectshttps://github.com/github/codeql/projects
Security and qualityhttps://github.com/github/codeql/security
Insightshttps://github.com/github/codeql/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fgithub%2Fcodeql%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fgithub%2Fcodeql%2Fissues%2Fnew%2Fchoose
nickrolfehttps://github.com/nickrolfe
mainhttps://github.com/github/codeql/tree/main
nickrolfe/mad-generator-jsonhttps://github.com/github/codeql/tree/nickrolfe/mad-generator-json
Conversation 12 https://github.com/github/codeql/pull/22798
Commits 2 https://github.com/github/codeql/pull/22798/commits
Checks 15 https://github.com/github/codeql/pull/22798/checks
Findings https://github.com/github/codeql/pull/22798/findings
Files changed https://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
MaD: update generator scripts to support writing JSON data extensions https://github.com/github/codeql/pull/22798/files#top
Show all changes 2 commits https://github.com/github/codeql/pull/22798/files
96ec1ed MaD: update generator scripts to be format-agnostic nickrolfe Oct 9, 2026 https://github.com/github/codeql/pull/22798/commits/96ec1eddfcc1e7a4f51985add8137e6b7de616e2
3872102 MaD: update generator scripts to support writing JSON data extensions nickrolfe Oct 9, 2026 https://github.com/github/codeql/pull/22798/commits/38721028d75d52cfe0672391d8dc5d633cc172a1
Clear filters https://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
bulk_generate_mad.py https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
convert_extensions.py https://github.com/github/codeql/pull/22798/files#diff-c4e4c92c52716e8378cec3060f10bf85787cc5f28dd592eb11419cd81bc0a84e
data_extension_file.py https://github.com/github/codeql/pull/22798/files#diff-3680036fa05204ab9a05ca5cfd4c1805afbafcf7f280305c3a9d4562f889e160
generate_mad.py https://github.com/github/codeql/pull/22798/files#diff-f48c430f423ff34f220d6088442e29f5f5f4404b204d139f6f053fcfe91f445d
helpers.py https://github.com/github/codeql/pull/22798/files#diff-f08f9bc883b777164823bcc0f8d15e48250e19c66a6bd092633659d0b40c3fcd
https://github.com/github/codeql/blob/main/CODEOWNERS#L2
misc/scripts/models-as-data/bulk_generate_mad.pyhttps://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
View file https://github.com/github/codeql/blob/38721028d75d52cfe0672391d8dc5d633cc172a1/misc/scripts/models-as-data/bulk_generate_mad.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/22798/{{ revealButtonHref }}
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/pull/22798/files#diff-267ed3726802be833947f890eed1c2872425a7de62a76fdb0c0b7813ce46cf30
https://github.com/github/codeql/blob/main/CODEOWNERS#L2
misc/scripts/models-as-data/convert_extensions.pyhttps://github.com/github/codeql/pull/22798/files#diff-c4e4c92c52716e8378cec3060f10bf85787cc5f28dd592eb11419cd81bc0a84e
View file https://github.com/github/codeql/blob/38721028d75d52cfe0672391d8dc5d633cc172a1/misc/scripts/models-as-data/convert_extensions.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/22798/{{ revealButtonHref }}
https://github.com/github/codeql/pull/22798/files#diff-c4e4c92c52716e8378cec3060f10bf85787cc5f28dd592eb11419cd81bc0a84e
https://github.com/github/codeql/pull/22798/files#diff-c4e4c92c52716e8378cec3060f10bf85787cc5f28dd592eb11419cd81bc0a84e
https://github.com/github/codeql/pull/22798/files#diff-c4e4c92c52716e8378cec3060f10bf85787cc5f28dd592eb11419cd81bc0a84e
https://github.com/github/codeql/blob/main/CODEOWNERS#L2
misc/scripts/models-as-data/data_extension_file.pyhttps://github.com/github/codeql/pull/22798/files#diff-3680036fa05204ab9a05ca5cfd4c1805afbafcf7f280305c3a9d4562f889e160
View file https://github.com/github/codeql/blob/38721028d75d52cfe0672391d8dc5d633cc172a1/misc/scripts/models-as-data/data_extension_file.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/github/codeql/pull/22798/{{ revealButtonHref }}
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
Please reload this pagehttps://github.com/github/codeql/pull/22798/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.