Title: Support pure C# DSL keywords in PowerShell · Issue #3243 · PowerShell/PowerShell · GitHub
Open Graph Title: Support pure C# DSL keywords in PowerShell · Issue #3243 · PowerShell/PowerShell
X Title: Support pure C# DSL keywords in PowerShell · Issue #3243 · PowerShell/PowerShell
Description: This is slightly premature, given that it assumes #3169 , but I want to lay out a possible roadmap for improving the DSL/DynamicKeyword functionality in that PR. #3169 provides C# syntax for specifying a dynamic keyword syntax, but achie...
Open Graph Description: This is slightly premature, given that it assumes #3169 , but I want to lay out a possible roadmap for improving the DSL/DynamicKeyword functionality in that PR. #3169 provides C# syntax for specif...
X Description: This is slightly premature, given that it assumes #3169 , but I want to lay out a possible roadmap for improving the DSL/DynamicKeyword functionality in that PR. #3169 provides C# syntax for specif...
Opengraph URL: https://github.com/PowerShell/PowerShell/issues/3243
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Support pure C# DSL keywords in PowerShell","articleBody":"This is slightly premature, given that it assumes #3169 , but I want to lay out a possible roadmap for improving the DSL/DynamicKeyword functionality in that PR.\r\n\r\n#3169 provides C# syntax for specifying a dynamic keyword syntax, but achieves the actual keyword functionality by invoking PowerShell functions of the form `DslModuleName\\DslKeywordName`. This works by rewriting the AST at compile time so that what the user has written as keyword invocations get turned into function calls, where the \"bodies\" of keywords (scriptblocks or hashtables) get passed in as arguments to the function (essentially DSLs are syntactic sugar for nested function calls). This works well, but some users may prefer to write everything within the same file, just like Cmdlets. There are a few steps to make here.\r\n\r\nWork out just how much we can change the public, but relatively unused, `DynamicKeyword` API\r\n---\r\nAt the moment, this looks like:\r\n```powershell\r\n$kw = [System.Management.Automation.DynamicKeyword]::new()\r\n$kw.Keyword = \"MyKeyword\"\r\n$kw.PostParse = { ... }\r\n...\r\n[System.Management.Automation.DynamicKeyword]::AddKeyword($kw)\r\n```\r\nThis API is currently available through and stored by a thread-static variable (as seen above). However, because keywords are specific to a parser session, they are likely better stored as parser-local. This would be a breaking change, but with a good cause -- especially since PowerShell currently keeps the parser and the REPL in different threads. While this API might always have a use, it might be nice to prefer a more fluent way to define DSL keywords within PowerShell (such as a keyword-defining DSL).\r\n\r\nUpdate [DynamicKeywordStatementRule](https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/parser/Parser.cs#L3379)\r\n---\r\nCurrently this is relatively specific for DSC keywords, and in particular has special logic for names in keywords, and does not allow keywords to use parameters -- parameters passed to a scriptblock- or hashtable-bodied keyword will be ignored. For a suggestion on how an update to this might look, see [a rough prototype I wrote](https://github.com/rjmholt/PowerShell/blob/dsl-csharp-prototype/src/System.Management.Automation/engine/parser/Parser.cs#L3753).\r\n\r\nSeparate DSC keywords from `DynamicKeyword`s\r\n---\r\nThis class is currently relatively specific to DSC keywords, and might be better as a base type that DSC keywords can inherit from. It contains assumptions such as `ResourceName`, as well as an extension class (`DynamicKeywordExtension`) which would be a good candidate for features on a `DscKeyword` class.\r\n\r\nBreak up the `DynamicKeyword` class\r\n---\r\n Currently `DynamicKeyword` holds information like keyword name and parameter type constraints, that can be read in without loading an assembly, as well as delegates like `SemanticCheck` that do require an assembly load. Moving to a pure C# implementation without changing `DynamicKeyword` might result in it becoming a catch-all monolith. Instead, breaking it up might pay some dividends:\r\n- A `DynamicKeywordMetadata` class could hold any information that can be obtained without loading the keyword's containing assembly. Ideally this class would be immutable after construction like other elements in the AST. This could inherit from `CommandMetadata`, containing things like `ParameterMetadata`.\r\n- A `DynamicKeywordInfo` class, just like `CmdletInfo` would contain information evaluated from loading the user-defined type that defines a keyword, and would inherit from `CommandInfo`. This would similarly be immutable, but more aimed at storing parse-time delegates and being used at compile-time. For a rough prototype, see [here](https://github.com/rjmholt/PowerShell/blob/dsl-csharp-prototype/src/System.Management.Automation/engine/parser/DslSpecification.cs#L29).\r\n- `DynamicKeyword` itself would perhaps be better repurposed in the namespace in the same way as `Cmdlet`, so as the abstract base class that user-defined keywords inherit from, with virtual methods to provide useful abstractions over the complexity of the PowerShell internals. A benefit here is that DSC keywords could be defined by inheriting from `DscKeyword`. Again, see [here](https://github.com/rjmholt/PowerShell/blob/dsl-csharp-prototype/src/System.Management.Automation/engine/parser/DslSpecification.cs#L313).\r\n- Finally, the state managed in the static part of `DynamicKeyword` should be taken out and managed with structures like `DynamicKeywordScope` and `DynamicKeywordRuntimeState` and moved to be parser-local. This is already in #3169 [here](https://github.com/rjmholt/PowerShell/blob/dynamic-keyword-metadata-reader/src/System.Management.Automation/engine/parser/DynamicKeyword.cs#L200) and [here](https://github.com/rjmholt/PowerShell/blob/dynamic-keyword-metadata-reader/src/System.Management.Automation/engine/parser/DynamicKeyword.cs#L389),\r\n\r\n\r\nCompiling the C# keyword\r\n---\r\nThis seemed originally to be a hard problem (and may still be), since users must be able to specify when the expressions they pass to their keywords get evaluated -- in case they are side-effectful. Meaning that arbitrary PowerShell expressions that may reference runtime state (like `$y + Get-Result -Param ($z+1)`) must be passed into the keyword functions written in C# without executing them, and then executed at a time when the C# code prescribes.\r\n\r\n(Consider in Pester: I can put a `BeforeAll` block after all other keywords, but if the keywords above it are executed before it, they might not have the state set up that they need.)\r\n\r\nThis is solved in the current implementation by passing the [body as an expression](https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/parser/ast.cs#L7220) to the function that implements the keyword -- essentially canning the keyword body like a [thunk](https://en.wikipedia.org/wiki/Thunk) so that the implementing user can choose when (of if) to evaluate it.\r\n\r\nI think a similar method of creating a C# `ScriptBlock` object for keywords to take in as an argument might work well, which would effectively manage the problem of how state set up by outer keywords is communicated to inner keywords. But it certainly bodes discussion.\r\n\r\nPassing parameters to keywords\r\n---\r\nThis is the actual hard problem. In order to use the parameter binding algorithm to set parameters on keywords implemented as C#/.NET objects, we need to use the parameter binder logic to make sure that parameters passed to keywords work and are resolved in exactly the same way as parameters passed to any other PowerShell command -- that users' expectations are met.\r\n\r\nFrom what I've read in the code so far, much of the existing logic is spread across `ParameterBinderBase`, `ParameterBinderController`, `CmdletParameterBinderController` and several others. My understanding is that there are two parameter binders in use. `ReflectionParameterBinder` and `ScriptParameterBinder` do most of the heavy lifting with some common code in the base class, and `PseudoParameterBinder` uses a combination of the two internally, while `NativeCommandParameterBinder` does its best to just pass arguments through as strings. The difficulty arises when `CommandProcess` [assumes that the command invoking it could only be a ScriptBlock or a Cmdlet](https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/CommandProcessor.cs#L114). However, this might be easier to fix than I think, or more likely I am wrong about how this all works.\r\n\r\nThis is complicated by the fact that the parameters on keywords (not their values) are known at compile time -- so it's less complicated to perform parameter and parameter type checks than for Cmdlets.\r\n\r\nFor what it's worth, I have a [small start on a C#-keyword parameter binder that doesn't work](https://github.com/rjmholt/PowerShell/blob/dsl-csharp-prototype/src/System.Management.Automation/engine/parser/DslSpecification.cs#L673), but similarly doesn't crash PowerShell (I consider this an achievement...). It seems like the alternatives are:\r\n- Rewrite some of the current parameter binder logic so that base classes make less assumptions about subclasses and also provide more of the parameter binding logic in a generic way.\r\n- Partially re-implement the parameter binding algorithm in a new subclass of `ParameterBinderBase` and only touch what needs to be changed in the existing parts of code like `CommandProcessor`.\r\n\r\nThis is as far as I've gotten with moving dynamic keywords to a pure C# implementation -- and I wanted to record it for discussion.","author":{"url":"https://github.com/rjmholt","@type":"Person","name":"rjmholt"},"datePublished":"2017-03-03T00:44:38Z","url":"https://github.com/PowerShell/PowerShell/issues/3243"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:9248cc28-8da0-3f94-a2ab-a124c4db542d |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | AA4A:3C5A36:1B5933:274426:6AFDD045 |
| html-safe-nonce | e1826992deea73c091883852d4a588eb2c143c545b8e24fc229b9a98811ffba2 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQTRBOjNDNUEzNjoxQjU5MzM6Mjc0NDI2OjZBRkREMDQ1IiwidmlzaXRvcl9pZCI6IjE1Njg2NzEyOTEyMjc4ODU2MzciLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | b75859d4ee758cf80b9733e320343ba277d4069999ea914dbed59394f0f343fa |
| hovercard-subject-tag | issue:211563093 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/PowerShell/PowerShell/3243/issue_layout |
| twitter:image | https://opengraph.githubassets.com/ee60af2cbbddf60e4b4fb191e38dc68b82b1e2931fb846cd77f812618aaea2a9/PowerShell/PowerShell/issues/3243 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/ee60af2cbbddf60e4b4fb191e38dc68b82b1e2931fb846cd77f812618aaea2a9/PowerShell/PowerShell/issues/3243 |
| og:image:alt | This is slightly premature, given that it assumes #3169 , but I want to lay out a possible roadmap for improving the DSL/DynamicKeyword functionality in that PR. #3169 provides C# syntax for specif... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | rjmholt |
| hostname | github.com |
| expected-hostname | github.com |
| None | 26886a40b582b821a7a76351e0a2f42b6d6160854b6e2520db24f31fd7be5da7 |
| turbo-cache-control | no-preview |
| go-import | github.com/PowerShell/PowerShell git https://github.com/PowerShell/PowerShell.git |
| octolytics-dimension-user_id | 11524380 |
| octolytics-dimension-user_login | PowerShell |
| octolytics-dimension-repository_id | 49609581 |
| octolytics-dimension-repository_nwo | PowerShell/PowerShell |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 49609581 |
| octolytics-dimension-repository_network_root_nwo | PowerShell/PowerShell |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 68ebaa631a1d248d69ebde4b9c5defd3a19aeb2f |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width