René's URL Explorer Experiment


Title: Turn specs into evals for any agent with ASSERT - Command Line

Open Graph Title: Turn specs into evals for any agent with ASSERT

X Title: Turn specs into evals for any agent with ASSERT

Description: ASSERT is an open-source framework for converting natural language behavior requirements into executable evaluations of AI models and agents.

Open Graph Description: ASSERT is an open-source framework for converting natural language behavior requirements into executable evaluations of AI models and agents.

X Description: ASSERT is an open-source framework for converting natural language behavior requirements into executable evaluations of AI models and agents.

Opengraph URL: https://commandline.microsoft.com/assert-written-intent-executable-evals/

direct link

Domain: commandline.microsoft.com

Hey, it has json ld scripts: 
{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#article","isPartOf":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/"},"author":{"name":"Lisa Brown Jaloza","@id":"https:\/\/commandline.microsoft.com\/#\/schema\/person\/973f271f274488f0846804fbab859a2b"},"headline":"Turn specs into evals for any agent with ASSERT","datePublished":"2026-06-02T17:29:45+00:00","dateModified":"2026-10-08T14:21:08+00:00","mainEntityOfPage":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/"},"wordCount":2113,"commentCount":0,"publisher":{"@id":"https:\/\/commandline.microsoft.com\/#organization"},"image":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#primaryimage"},"thumbnailUrl":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/06\/turn-specs-into-evals-for-any-agent-with-assert-card.svg","inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/","url":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/","name":"Turn specs into evals for any agent with ASSERT - Command Line","isPartOf":{"@id":"https:\/\/commandline.microsoft.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#primaryimage"},"image":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#primaryimage"},"thumbnailUrl":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/06\/turn-specs-into-evals-for-any-agent-with-assert-card.svg","datePublished":"2026-06-02T17:29:45+00:00","dateModified":"2026-10-08T14:21:08+00:00","description":"ASSERT is an open-source framework for converting natural language behavior requirements into executable evaluations of AI models and agents.","breadcrumb":{"@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#primaryimage","url":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/06\/turn-specs-into-evals-for-any-agent-with-assert-card.svg","contentUrl":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/06\/turn-specs-into-evals-for-any-agent-with-assert-card.svg","caption":"Turn specs into evals for any agent with ASSERT card visual"},{"@type":"BreadcrumbList","@id":"https:\/\/commandline.microsoft.com\/assert-written-intent-executable-evals\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/commandline.microsoft.com\/"},{"@type":"ListItem","position":2,"name":"Turn specs into evals for any agent with ASSERT"}]},{"@type":"WebSite","@id":"https:\/\/commandline.microsoft.com\/#website","url":"https:\/\/commandline.microsoft.com\/","name":"Command Line","description":"By builders, for builders.","publisher":{"@id":"https:\/\/commandline.microsoft.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/commandline.microsoft.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/commandline.microsoft.com\/#organization","name":"Command Line","url":"https:\/\/commandline.microsoft.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/commandline.microsoft.com\/#\/schema\/logo\/image\/","url":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/05\/command-line-icon.png","contentUrl":"https:\/\/commandline.microsoft.com\/wp-content\/uploads\/2026\/05\/command-line-icon.png","width":512,"height":512,"caption":"Command Line"},"image":{"@id":"https:\/\/commandline.microsoft.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/commandline.microsoft.com\/#\/schema\/person\/973f271f274488f0846804fbab859a2b","name":"Lisa Brown Jaloza","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/83a688101e15a88112553abbddf9c7fc8f61b8ea0741dbc26d801491212c5b5d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/83a688101e15a88112553abbddf9c7fc8f61b8ea0741dbc26d801491212c5b5d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/83a688101e15a88112553abbddf9c7fc8f61b8ea0741dbc26d801491212c5b5d?s=96&d=mm&r=g","caption":"Lisa Brown Jaloza"},"url":"https:\/\/commandline.microsoft.com\/author\/lbrownjalozamicrosoft-com\/"}]}
og:localeen_US
og:typearticle
og:site_nameCommand Line
article:published_time2026-06-02T17:29:45+00:00
article:modified_time2026-10-08T14:21:08+00:00
og:imagehttps://commandline.microsoft.com/wp-content/themes/command-line/assets/images/logo.svg?v=1794397501
og:image:width2560
og:image:height1440
og:image:typeimage/png
authorLisa Brown Jaloza
twitter:cardsummary_large_image
twitter:label1Written by
twitter:data1Lisa Brown Jaloza
twitter:label2Est. reading time
twitter:data210 minutes
msapplication-TileImagehttps://commandline.microsoft.com/wp-content/uploads/2026/05/favicon-300x300.png

Links:

Skip to main contenthttps://commandline.microsoft.com/assert-written-intent-executable-evals/#main-content
https://www.microsoft.com
Source https://news.microsoft.com/source
Signal bloghttps://news.microsoft.com/signal/
Official Microsoft Bloghttps://blogs.microsoft.com
Command Linehttps://commandline.microsoft.com/
Microsoft On The Issueshttps://blogs.microsoft.com/on-the-issues/
Asiahttps://news.microsoft.com/apac/
Canadahttps://news.microsoft.com/source/canada/
Europe, Middle East and Africahttps://news.microsoft.com/europe/
Latin Americahttps://news.microsoft.com/es-xl/
The Code of Ushttps://news.microsoft.com/codeofus
What's newhttps://news.microsoft.com/signal/
AIhttps://news.microsoft.com/source/topics/ai/
Innovationhttps://news.microsoft.com/source/topics/innovation/
Digital Transformationhttps://news.microsoft.com/source/topics/digital-transformation/
Sustainabilityhttps://news.microsoft.com/source/topics/sustainability/
Securityhttps://news.microsoft.com/source/topics/security/
Work & Lifehttps://news.microsoft.com/source/topics/work-life/
Diversity & Inclusionhttps://news.microsoft.com/source/topics/diversity-inclusion/
Unlockedhttps://unlocked.microsoft.com/
Microsoft 365https://www.microsoft.com/microsoft-365
Azurehttps://azure.microsoft.com/en-us?ocid=cmm4r4ppnhp
Copilothttps://copilot.com/?fromcode=cmmuaql67jy
Windowshttps://www.microsoft.com/en-us/windows/
Surfacehttps://www.microsoft.com/surface
XBOXhttps://www.xbox.com/
Dealshttps://www.microsoft.com/en-us/store/b/sale?icid=DSM_TopNavDeals
Small Businesshttps://www.microsoft.com/en-us/store/b/business
Supporthttps://support.microsoft.com/en-us
Windows Appshttps://apps.microsoft.com/home
Outlookhttps://www.microsoft.com/en-us/microsoft-365/outlook/email-and-calendar-software-microsoft-outlook
OneDrivehttps://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage
Microsoft Teamshttps://www.microsoft.com/en-us/microsoft-teams/group-chat-software
OneNotehttps://www.microsoft.com/en-us/microsoft-365/onenote/digital-note-taking-app
Microsoft Edgehttps://www.microsoft.com/edge
Moving from Skype to Teamshttps://support.microsoft.com/en-us/office/moving-from-skype-to-microsoft-teams-free-3c0caa26-d9db-4179-bcb3-930ae2c87570?icid=DSM_All_Skype
Computershttps://www.microsoft.com/en-us/store/b/pc?icid=CNavDevicesPC
Shop XBOXhttps://www.microsoft.com/en-us/store/b/xbox?icid=DSM_All_ShopXbox
Accessorieshttps://www.microsoft.com/en-us/store/b/accessories?icid=CNavDevicesAccessories
VR & mixed realityhttps://www.microsoft.com/en-us/store/b/virtualreality?icid=CNavVirtualReality
Certified Refurbishedhttps://www.microsoft.com/en-us/store/b/certified-refurbished-products
Trade-in for cashhttps://www.microsoft.com/en-us/store/b/microsoft-trade-in
XBOX Game Pass Ultimatehttps://www.xbox.com/en-us/games/store/xbox-game-pass-ultimate/cfq7ttc0khs0?icid=DSM_All_XboxGamePassUltimate
PC Game Passhttps://www.xbox.com/en-us/games/store/pc-game-pass/cfq7ttc0kgq8?icid=CNavAllPCGamePass
XBOX gameshttps://www.microsoft.com/en-us/store/b/xboxgames?icid=DSM_All_XboxGames
PC gameshttps://apps.microsoft.com/games
Microsoft AIhttps://www.microsoft.com/en-us/ai?icid=DSM_All_AI
Microsoft Securityhttps://www.microsoft.com/en-us/security
Dynamics 365https://www.microsoft.com/en-us/dynamics-365
Microsoft 365 for businesshttps://www.microsoft.com/en-us/microsoft-365/business
Microsoft Power Platformhttps://www.microsoft.com/en-us/power-platform
Windows 365https://www.microsoft.com/en-us/windows-365
Small Businesshttps://www.microsoft.com/en-us/store/b/business?icid=CNavBusinessStore
Digital Sovereigntyhttps://www.microsoft.com/en-us/sovereignty?icid=DSM_More_Sovereignty
Azurehttps://azure.microsoft.com/en-us/
Microsoft Developerhttps://developer.microsoft.com/en-us/
Microsoft Learnhttps://learn.microsoft.com/
Support for AI marketplace appshttps://www.microsoft.com/software-development-companies/offers-benefits/isv-success?icid=DSM_All_SupportAIMarketplace&ocid=cmm3atxvn98
Microsoft Tech Communityhttps://techcommunity.microsoft.com/
Microsoft Marketplacehttps://marketplace.microsoft.com?icid=DSM_All_Marketplace&ocid=cmm3atxvn98
Software companieshttps://www.microsoft.com/software-development-companies?icid=DSM_All_SoftwareCompanies&ocid=cmm3atxvn98
Visual Studiohttps://visualstudio.microsoft.com/
Microsoft Rewardshttps://www.microsoft.com/rewards
Free downloads & securityhttps://www.microsoft.com/en-us/download
Educationhttps://www.microsoft.com/en-us/education
Gift cardshttps://www.microsoft.com/en-us/store/b/gift-cards?icid=DSM_All_GiftCards
Licensinghttps://www.microsoft.com/licensing/
Unlocked storieshttps://unlocked.microsoft.com/
View Sitemaphttps://www.microsoft.com/en-us/sitemap
https://commandline.microsoft.com/
Open Sourcehttps://commandline.microsoft.com/?type=Open%20Source
inhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
rhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
xhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
fhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
Responsible AI at Microsofthttps://commandline.microsoft.com/author/responsible-ai-at-microsoft
Mehrnoosh Samekihttps://commandline.microsoft.com/author/mehrnoosh-sameki
Sandeep Atlurihttps://commandline.microsoft.com/author/sandeep-atluri
Minsoo Thigpenhttps://commandline.microsoft.com/author/minsoo-thigpen
Abhinav Paliahttps://commandline.microsoft.com/author/abby-palia
Chang Liuhttps://commandline.microsoft.com/author/chang-lui
Adaptive Spec-driven Scoring for Evaluation and Regression Testing (ASSERT)https://github.com/responsibleai/ASSERT
Read the paper https://arxiv.org/abs/2608.13840
Agarwal et al. (2026)https://arxiv.org/abs/2605.26001
examples/travel_planner_langgraph/https://github.com/microsoft/ASSERT/tree/main/examples/travel_planner_langgraph
https://github.com/responsibleai/ASSERThttps://github.com/responsibleai/ASSERT
responsibleai.github.io/ASSERThttps://responsibleai.github.io/ASSERT/
travel-planning agenthttps://github.com/microsoft/ASSERT/tree/main/examples/travel_planner_langgraph
https://arxiv.org/abs/2608.13840https://arxiv.org/abs/2608.13840
Read the paper https://arxiv.org/abs/2608.13840
inhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
rhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
xhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
fhttps://commandline.microsoft.com/assert-written-intent-executable-evals/
Introducing run-assert-eval: Find the risk, fix it, prove it Open Source Today, we’re introducing run-assert-eval, a skill that discovers the risks that matter for a given agent, measures how often the agent fails, generates runtime policy directly from those findings, and reruns the eval to prove whether the fix worked. [ read ] September 24, 2026 https://commandline.microsoft.com/run-assert-eval-responsible-ai-agent-risk-discovery-at-runtime/
PyRIT: Democratizing AI red teaming through open-source tooling Open Source Learn how the Python Risk Identification Tool for generative AI evolved over time to open up AI red teaming to subject matter experts from virtually any domain. [ read ] September 10, 2026 https://commandline.microsoft.com/pyrit-python-risk-identification-tool-ai-red-teaming-subject-matter-experts/
https://microsoft.com
Subscribe for updateshttps://adoption.microsoft.com/command-line-newsletter
English (United States) https://www.microsoft.com/en-us/locale
Your Privacy Choices https://aka.ms/yourcaliforniaprivacychoices
Consumer Health Privacyhttps://go.microsoft.com/fwlink/?linkid=2259814
Sitemap https://commandline.microsoft.com/sitemap_index.xml
Contact Microsoft https://support.microsoft.com/contactus
Privacy https://go.microsoft.com/fwlink/?LinkId=521839
Terms of use https://go.microsoft.com/fwlink/?LinkID=206977
Trademarks https://go.microsoft.com/fwlink/?linkid=2196228
Safety & eco https://go.microsoft.com/fwlink/?linkid=2196227
Recycling https://www.microsoft.com/en-us/legal/compliance/recycling
About our ads https://choice.microsoft.com
Manage cookieshttps://commandline.microsoft.com/assert-written-intent-executable-evals/
RSShttps://commandline.microsoft.com/feed/
Surface Laptop Ultrahttps://www.microsoft.com/en-us/surface/devices/surface-laptop-ultra?icid=DSM_Footer_WhatsNew_SurfaceLaptopUltra
Surface Prohttps://www.microsoft.com/surface/devices/surface-pro
Surface Laptophttps://www.microsoft.com/surface/devices/surface-laptop
Surface RTX Spark Dev Boxhttps://www.microsoft.com/en-us/surface/devices/surface-rtx-spark-dev-box?icid=DSM_Footer_WhatsNew_SurfaceRTXSparkDevBox
Explore Copilothttps://www.microsoft.com/en-us/copilot?icid=DSM_Footer_WhatsNew_Copilot
Explore Microsoft productshttps://www.microsoft.com/en-us/microsoft-products-and-apps
Windows 11 appshttps://www.microsoft.com/en-us/windows/apps-for-windows?icid=DSM_Footer_WhatsNew_Windows11apps
Account profilehttps://account.microsoft.com/
Download Centerhttps://www.microsoft.com/en-us/download
Microsoft Store supporthttps://go.microsoft.com/fwlink/?linkid=2139749
Returnshttps://www.microsoft.com/en-us/store/b/returns
Order trackinghttps://www.microsoft.com/en-us/store/b/order-tracking
Certified Refurbishedhttps://www.microsoft.com/en-us/store/b/certified-refurbished-products
Why buy from Microsoft Storehttps://www.microsoft.com/en-us/store/b/why-microsoft-store?icid=footer_why-msft-store?icid=DSM_Footer_MicrosoftStore_WhyBuy
Flexible Paymentshttps://www.microsoft.com/en-us/store/b/payment-financing-options?icid=footer_financing_vcc
Microsoft in educationhttps://www.microsoft.com/en-us/education
Devices for educationhttps://www.microsoft.com/en-us/education/devices/overview
Microsoft Teams for Educationhttps://www.microsoft.com/en-us/education/products/teams
Microsoft 365 Educationhttps://www.microsoft.com/en-us/education/products/microsoft-365
How to buy for your schoolhttps://www.microsoft.com/education/how-to-buy
Educator training and developmenthttps://education.microsoft.com/
Deals for students and parentshttps://www.microsoft.com/en-us/store/b/education
AI for educationhttps://www.microsoft.com/en-us/education/ai-in-education
Microsoft AIhttps://www.microsoft.com/en-us/ai?icid=DSM_Footer_AI
Microsoft Securityhttps://www.microsoft.com/en-us/security
Dynamics 365https://www.microsoft.com/en-us/dynamics-365
Microsoft 365https://www.microsoft.com/en-us/microsoft-365/business
Microsoft Power Platformhttps://www.microsoft.com/en-us/power-platform
Microsoft Teamshttps://www.microsoft.com/en-us/microsoft-teams/group-chat-software
Microsoft Copilothttps://www.microsoft.com/en-us/copilot?icid=DSM_Footer_Business_Copilot
Small Businesshttps://www.microsoft.com/en-us/store/b/business?icid=CNavBusinessStore
Azurehttps://azure.microsoft.com/en-us/
Microsoft Developerhttps://developer.microsoft.com/en-us/
Microsoft Learnhttps://learn.microsoft.com/
Support for AI marketplace appshttps://www.microsoft.com/software-development-companies/offers-benefits/isv-success?icid=DSM_Footer_SupportAIMarketplace&ocid=cmm3atxvn98
Microsoft Tech Communityhttps://techcommunity.microsoft.com/
Microsoft Marketplacehttps://marketplace.microsoft.com?icid=DSM_Footer_Marketplace&ocid=cmm3atxvn98
Software companieshttps://www.microsoft.com/software-development-companies?icid=DSM_Footer_SoftwareCompanies&ocid=cmm3atxvn98
Visual Studiohttps://visualstudio.microsoft.com/
Careershttps://careers.microsoft.com/
About Microsofthttps://www.microsoft.com/about
Company newshttps://news.microsoft.com/source/?icid=DSM_Footer_Company_CompanyNews
Privacy at Microsofthttps://www.microsoft.com/en-us/privacy?icid=DSM_Footer_Company_Privacy
Investorshttps://www.microsoft.com/investor/default.aspx
Diversity and inclusionhttps://www.microsoft.com/en-us/diversity/default?icid=DSM_Footer_Company_Diversity
Accessibilityhttps://www.microsoft.com/en-us/accessibility
Sustainabilityhttps://www.microsoft.com/en-us/corporate-responsibility/sustainability?icid=DSM_Footer_Sustainability
English (United States)https://www.microsoft.com/en-us/locale
Your Privacy Choices Opt-Out Icon Your Privacy Choices https://aka.ms/yourcaliforniaprivacychoices
Your Privacy Choices Opt-Out Icon Your Privacy Choices https://aka.ms/yourcaliforniaprivacychoices
Consumer Health Privacyhttps://go.microsoft.com/fwlink/?linkid=2259814
Sitemaphttps://www.microsoft.com/en-us/sitemap1.aspx
Contact Microsofthttps://support.microsoft.com/contactus
Privacy https://go.microsoft.com/fwlink/?LinkId=521839
Manage cookieshttps://commandline.microsoft.com/assert-written-intent-executable-evals/
Terms of usehttps://go.microsoft.com/fwlink/?LinkID=206977
Trademarkshttps://go.microsoft.com/fwlink/?linkid=2196228
Safety & ecohttps://go.microsoft.com/fwlink/?linkid=2196227
Recyclinghttps://www.microsoft.com/en-us/legal/compliance/recycling
About our adshttps://choice.microsoft.com

Viewport: width=device-width, initial-scale=1

Robots: index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1

URLs of crawlers that visited me.