Title: Pre-Compiler Plugin Proposal · Issue #38736 · microsoft/TypeScript · GitHub
Open Graph Title: Pre-Compiler Plugin Proposal · Issue #38736 · microsoft/TypeScript
X Title: Pre-Compiler Plugin Proposal · Issue #38736 · microsoft/TypeScript
Description: Search Terms Extension, Plugin, Vue, Custom extensions I found some issues but no proposal :/ Suggestion Hi! I'm an author of the fork-ts-checker-webpack-plugin. Last few months I was busy with rewriting this plugin to pay off the techno...
Open Graph Description: Search Terms Extension, Plugin, Vue, Custom extensions I found some issues but no proposal :/ Suggestion Hi! I'm an author of the fork-ts-checker-webpack-plugin. Last few months I was busy with rew...
X Description: Search Terms Extension, Plugin, Vue, Custom extensions I found some issues but no proposal :/ Suggestion Hi! I'm an author of the fork-ts-checker-webpack-plugin. Last few months I was busy with...
Opengraph URL: https://github.com/microsoft/TypeScript/issues/38736
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Pre-Compiler Plugin Proposal","articleBody":"## Search Terms\r\nExtension, Plugin, Vue, Custom extensions\r\n\r\nI found some issues but no proposal :/\r\n\r\n## Suggestion\r\nHi! I'm an author of the [`fork-ts-checker-webpack-plugin`](https://github.com/TypeStrong/fork-ts-checker-webpack-plugin). Last few months I was busy with [rewriting this plugin to pay off the technological debt](https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/pull/404).\r\n\r\nOne of the features that this plugin provides is support for the [_Vue.js Single File Components_](https://vuejs.org/v2/guide/single-file-components.html). I was able to implement it but in order to work with the `Watch` program and `SolutionBuilder` [I had to do some workarounds](https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/blob/feature/refactor-to-version-5/src/typescript-reporter/extension/TypeScriptEmbeddedExtension.ts). It's because there is an assertion in the TypeScript source code that files cannot have a custom extension.\r\n\r\nThe main issue with this approach is that you have to implement these workarounds in every TypeScript's API client and they can behave differently (like [`appendTsSuffixTo`](https://github.com/TypeStrong/ts-loader#appendtssuffixto) in the `ts-loader`). I know that there are already Language Service Plugins, but they don't work if you use different API, like `Watch`.\r\n\r\n### The proposal\r\nA new type of TypeScript plugins that can pre-compile custom source code to the format that TypeScript understands. The simplified interface would be:\r\n```typescript\r\n////////////////////////////////////////////////\r\n// update existing implementation and typings //\r\n\r\n/**\r\n * Keep it for the backward compatibility\r\n * @deprecated\r\n */\r\ninterface PluginCreateInfo {\r\n project: Project;\r\n languageService: LanguageService;\r\n languageServiceHost: LanguageServiceHost;\r\n serverHost: ServerHost;\r\n config: any;\r\n}\r\n// the new name for this type\r\ntype LanguageServicePluginCreateInfo = PluginCreateInfo;\r\n\r\n/**\r\n * Keep it for the backward compatibility\r\n * @deprecated\r\n */\r\ninterface PluginModule {\r\n type?: 'language-service-plugin'; // not required due to backward compatibility\r\n getExternalFiles?(project: Project): string[];\r\n onConfigurationChanged?(config: any): void;\r\n create(createInfo: PluginCreateInfo): LanguageService;\r\n}\r\n// the new name for this type\r\ntype LanguageServicePluginModule = PluginModule & {\r\n type: 'language-service-plugin'; // required in the new type\r\n}\r\n\r\n/////////////////////////////\r\n// add new type of plugins //\r\n\r\ninterface PreCompilerPluginModuleCreateInfo {\r\n moduleResolutionHost: ModuleResolutionHost;\r\n sourceMapHost: SourceMapHost;\r\n config: any;\r\n}\r\ninterface PreCompilerPluginModule {\r\n type: 'pre-compiler-plugin'; // to distinguish between LanguageServicePlugin and PreCompilerPlugin\r\n onConfigurationChanged?(config: any): void;\r\n create(createInfo: PreCompilerPluginModuleCreateInfo): PreCompilerPlugin;\r\n}\r\ninterface PreCompilerPlugin {\r\n extensions: string[]; // list of supported extensions, for example ['.vue'] or ['.mdx']\r\n createPreCompiledSourceFile(\r\n fileName: string,\r\n sourceText: string,\r\n languageVersion: ScriptTarget,\r\n setParentNodes?: boolean\r\n ): PreCompiledSourceFile;\r\n}\r\n// we need to define an extended version of the SourceFile to support additional sourceMap\r\ninterface PreCompiledSourceFile extends SourceFile {\r\n preSourceMapText: string; // we need to provide source map to calculate diagnostics positions and source maps\r\n}\r\n\r\n// we need to define a function to create a PreCompiledSourceFile \r\n// (as we can't infer ScriptKind so it's not an optional parameter)\r\nfunction createPreCompiledSourceFile(\r\n fileName: string,\r\n sourceText: string,\r\n languageVersion: ScriptTarget,\r\n scriptKind: ScriptKind,\r\n setParentNodes?: boolean,\r\n sourceMapText?: string\r\n): PreCompiledSourceFile\r\n\r\n// this will help with source map generation\r\ninterface SourceTextNavigator {\r\n getLineAndCharacterOfPosition(position: number): LineAndCharacter;\r\n getPositionOfLineAndCharacter(line: number, character: number): number;\r\n getLineEndOfPosition(position: number): number;\r\n getLineStarts(): number[];\r\n}\r\ninterface SourceMapHost {\r\n createSourceMapGenerator(fileName: string): SourceMapGenerator;\r\n createSourceTextNavigator(text: string): SourceTextNavigator;\r\n}\r\n\r\n\r\n////////////////////////////////////////////////\r\n// update existing implementation and typings //\r\n\r\ntype AnyPluginModule = PluginModule | PreCompilerPluginModule;\r\n\r\ninterface PluginModuleWithName {\r\n name: string;\r\n module: AnyPluginModule;\r\n}\r\ntype PluginModuleFactory = (mod: {\r\n typescript: typeof ts;\r\n}) => AnyPluginModule\r\n```\r\n\r\nThis architecture would allow adding new plugin types in the future (so we could add `ModuleResolutionPlugin` to support Yarn PnP in the future)\r\n\r\nThe maintenance cost of this feature should be low because the exposed API is pretty simple.\r\n\r\nThere is a point in the [TypeScript Design Goals](https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals)\r\n> 4. Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.\r\n\r\nIf you think that this feature is incompatible with this point, we could limit it to emitting only `.d.ts` files.\r\n\r\nI could try to implement it, but first I need to know if it's something that you would like to add to the TypeScript and if this API is a good direction :)\r\n\r\n## Use Cases\r\n\r\nI imagine that community would create `PreCompilerPlugin`s for a lot of use cases. For example:\r\n * `vue-typescript-plugin` - support for the Single File Components\r\n * `mdx-typescript-plugin` - support for embedded TypeScript in the MDX files\r\n * `graphql-typescript-pluging` - support for generation of GraphQL types / clients in the runtime (so we don't have to use generators anymore)\r\n * `css-modules-typescript-plugin` - support for typed css modules (instead of `declare module '*.css';`)\r\n\r\n## Examples\r\n\r\n#### tsconfig.json\r\n```json\r\n{\r\n \"compilerOptions\": {\r\n \"target\": \"ES6\",\r\n \"module\": \"CommonJS\",\r\n \"lib\": [\"ES6\"],\r\n \"moduleResolution\": \"Node\",\r\n \"resolveJsonModule\": true,\r\n \"esModuleInterop\": true,\r\n \"baseUrl\": \"./src\",\r\n \"outDir\": \"./lib\",\r\n \"plugins\": [{ \"name\": \"vue-typescript-plugin\" }]\r\n },\r\n \"include\": [\"./src\"],\r\n \"exclude\": [\"node_modules\"]\r\n}\r\n```\r\n\r\n#### node_modules/vue-typescript-plugin/index.ts\r\nIt's a pseudo-code based on the `fork-ts-checker-webpack-plugin` implementation. It adds support for the `.vue` files. \r\n```typescript\r\nimport compiler from 'vue-template-compiler';\r\n\r\nfunction init(modules: { typescript: typeof import(\"typescript/lib/typescript\") }) {\r\n const ts = modules.typescript;\r\n\r\n function create({ sourceMapHost }: ts.PreCompilerPluginModuleCreateInfo): ts.PreCompilerPlugin {\r\n function getScriptKindByLang(lang: string | undefined) {\r\n switch (lang) {\r\n case 'ts':\r\n return ts.ScriptKind.TS;\r\n case 'tsx':\r\n return ts.ScriptKind.TSX;\r\n case 'jsx':\r\n return ts.ScriptKind.JSX;\r\n case 'json':\r\n return ts.ScriptKind.JSON;\r\n case 'js':\r\n default:\r\n return ts.ScriptKind.JS;\r\n }\r\n }\r\n\r\n function createNoScriptSourceFile(\r\n fileName: string,\r\n sourceText: string,\r\n languageVersion: ts.ScriptTarget,\r\n setParentNodes: boolean | undefined\r\n ): ts.PreCompilerSourceFile {\r\n const compiledText = 'export default {};';\r\n\r\n // generate source map\r\n const sourceMap = sourceMapHost.createSourceMapGenerator(fileName);\r\n const sourceIndex = sourceMap.addSource(fileName);\r\n sourceMap.setSourceContent(sourceIndex, sourceText);\r\n\r\n const sourceTextNavigator = sourceMapHost.createSourceTextNavigator(sourceText);\r\n\r\n const sourceStart = sourceTextNavigator.getLineAndCharacterOfPosition(0);\r\n const sourceEnd = sourceTextNavigator.getLineAndCharacterOfPosition(sourceText.length);\r\n\r\n sourceMap.addMapping(0, 0, sourceIndex, sourceStart.line, sourceStart.character);\r\n sourceMap.addMapping(0, compiledText.length, sourceIndex, sourceEnd.line, sourceEnd.character);\r\n\r\n // create source file\r\n return ts.createPreCompiledSourceFile(\r\n fileName,\r\n 'export default {};',\r\n languageVersion,\r\n ts.ScriptKind.JS,\r\n setParentNodes\r\n );\r\n }\r\n\r\n function createSrcScriptSourceFile(\r\n fileName: string,\r\n sourceText: string,\r\n languageVersion: ts.ScriptTarget,\r\n setParentNodes: boolean | undefined,\r\n sourceStartPosition: number,\r\n sourceEndPosition: number,\r\n scriptTagSrc: string,\r\n scriptTagLang: string | undefined,\r\n ): ts.PreCompilerSourceFile {\r\n // import path cannot be end with '.ts[x]'\r\n const compiledText = `export * from \"${scriptTagSrc.replace(/\\.tsx?$/i, '')}\";`;\r\n\r\n // generate source map\r\n const sourceMap = sourceMapHost.createSourceMapGenerator(fileName);\r\n const sourceIndex = sourceMap.addSource(fileName);\r\n sourceMap.setSourceContent(sourceIndex, sourceText);\r\n\r\n const sourceTextNavigator = sourceMapHost.createSourceTextNavigator(sourceText);\r\n const sourceStart = sourceTextNavigator.getLineAndCharacterOfPosition(sourceStartPosition);\r\n const sourceEnd = sourceTextNavigator.getLineAndCharacterOfPosition(sourceEndPosition);\r\n\r\n sourceMap.addMapping(0, 0, sourceIndex, sourceStart.line, sourceStart.character);\r\n sourceMap.addMapping(0, compiledText.length, sourceIndex, sourceEnd.line, sourceEnd.character);\r\n\r\n // create source file\r\n return ts.createPreCompiledSourceFile(\r\n fileName,\r\n compiledText,\r\n languageVersion,\r\n getScriptKindByLang(scriptTagLang),\r\n setParentNodes,\r\n sourceMap.toString()\r\n );\r\n }\r\n\r\n function createInlineScriptSourceFile(\r\n fileName: string,\r\n sourceText: string,\r\n languageVersion: ts.ScriptTarget,\r\n setParentNodes: boolean | undefined,\r\n sourceStartPosition: number,\r\n scriptTagContent: string,\r\n scriptTagLang: string | undefined\r\n ): ts.PreCompilerSourceFile {\r\n const compiledText = sourceTagContent;\r\n\r\n // generate source map\r\n const sourceMap = sourceMapHost.createSourceMapGenerator(fileName);\r\n const sourceIndex = sourceMap.addSource(fileName);\r\n sourceMap.setSourceContent(sourceIndex, sourceText);\r\n\r\n const sourceTextNavigator = sourceMapHost.createSourceTextNavigator(sourceText);\r\n const compiledTextNavigator = sourceMapHost.createSourceTextNavigator(compiledText);\r\n\r\n const compiledLineStarts = compiledTextNavigator.getLineStarts();\r\n compiledLineStarts.forEach((compiledLineStart) => {\r\n // map line by line\r\n const sourceStart = sourceTextNavigator.getLineAndCharacterOfPosition(sourceStartPosition + compiledLineStart);\r\n const sourceEnd = sourceTextNavigator.getLineAndCharacterOfPosition(sourceTextNavigator.getLineEndOfPosition(sourceStartPosition + compiledLineStart));\r\n const compiledStart = compiledTextNavigator.getLineAndCharacterOfPosition(compiledLineStart);\r\n const compiledEnd = compiledTextNavigator.getLineAndCharacterOfPosition(compiledTextNavigator.getLineEndOfPosition(compiledLineStart));\r\n\r\n sourceMap.addMapping(compiledStart.line, compiledStart.character, sourceIndex, sourceStart.line, sourceStart.character);\r\n sourceMap.addMapping(compiledEnd.line, compiledEnd.character, sourceIndex, sourceEnd.line, sourceEnd.character);\r\n });\r\n\r\n // create source file\r\n return ts.createPreCompiledSourceFile(\r\n fileName,\r\n compiledText,\r\n languageVersion,\r\n getScriptKindByLang(scriptTagLang),\r\n setParentNodes,\r\n sourceMap.toString()\r\n );\r\n }\r\n\r\n\r\n return {\r\n extensions: ['.vue'],\r\n createPreCompiledSourceFile(fileName, sourceText, languageVersion, setParentNodes) {\r\n const { script } = compiler.parseComponent(sourceText, { pad: 'space' });\r\n \r\n if (!script) {\r\n // No \u003cscript> block\r\n return createNoScriptSourceFile(fileName, sourceText, languageVersion, setParentNodes);\r\n } else if (script.attrs.src) {\r\n // \u003cscript src=\"file.ts\" /> block\r\n return createSrcScriptSourceFile(\r\n fileName, \r\n sourceText, \r\n languageVersion, \r\n setParentNodes, \r\n script.start, \r\n script.end, \r\n script.attrs.src, \r\n script.attrs.lang\r\n );\r\n } else {\r\n // \u003cscript lang=\"ts\">\u003c/script> block\r\n return createInlineScriptSourceFile(\r\n fileName, \r\n sourceText, \r\n languageVersion, \r\n setParentNodes, \r\n script.start,\r\n sourceText.slice(script.start, script.end), \r\n script.attrs.lang\r\n );\r\n }\r\n }\r\n }\r\n }\r\n\r\n return {\r\n type: 'pre-compiler-plugin',\r\n create\r\n };\r\n}\r\n\r\nexport = init;\r\n```\r\n\r\n## Checklist\r\n\r\nMy suggestion meets these guidelines:\r\n\r\n* [x] This wouldn't be a breaking change in existing TypeScript/JavaScript code\r\n* [x] This wouldn't change the runtime behavior of existing JavaScript code\r\n* [x] This could be implemented without emitting different JS based on the types of the expressions\r\n* [x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)\r\n* [x] This feature would agree with the rest of [TypeScript's Design Goals](https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals).\r\n","author":{"url":"https://github.com/piotr-oles","@type":"Person","name":"piotr-oles"},"datePublished":"2020-05-22T17:37:57Z","url":"https://github.com/microsoft/TypeScript/issues/38736"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:ba50f8e1-2f7e-7c2d-43b3-d3f93ebed557 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | CBA6:EA09B:5FC64F:80EBD8:6B00D47B |
| html-safe-nonce | 3d086f9b76b6359da13b58767e2f609eeb5d830c66462fd77ad172aa757a19dc |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDQkE2OkVBMDlCOjVGQzY0Rjo4MEVCRDg6NkIwMEQ0N0IiLCJ2aXNpdG9yX2lkIjoiNDQ2MjQyODM5NDc4ODgwMzcwNyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 9980a73db504483b65f552712a8dfe615e0c29462e5221ab487d7e4953a56b85 |
| hovercard-subject-tag | issue:623379629 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/microsoft/TypeScript/38736/issue_layout |
| twitter:image | https://opengraph.githubassets.com/cca59c1e55e21d51736bb269fd3300830d999344674ead1151feaf86cea6ae53/microsoft/TypeScript/issues/38736 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/cca59c1e55e21d51736bb269fd3300830d999344674ead1151feaf86cea6ae53/microsoft/TypeScript/issues/38736 |
| og:image:alt | Search Terms Extension, Plugin, Vue, Custom extensions I found some issues but no proposal :/ Suggestion Hi! I'm an author of the fork-ts-checker-webpack-plugin. Last few months I was busy with rew... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | piotr-oles |
| hostname | github.com |
| expected-hostname | github.com |
| None | 85bbdfe2b6ba16e234513bc6ceff5d848ef8449820c900692a2b4e67061931e5 |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/TypeScript git https://github.com/microsoft/TypeScript.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 20929025 |
| octolytics-dimension-repository_nwo | microsoft/TypeScript |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 20929025 |
| octolytics-dimension-repository_network_root_nwo | microsoft/TypeScript |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4943239a6d2c591325dad6e5c313e1cafda877f6 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width