L-2 policy

L-2/L-n policy means that the plugin should support only the latest n versions behind the current one.
In regards to WordPress & WooCommerce for sure would help to limit the number of combinations we test and check. I'd add one additional rule to that, to be able not to limit the innovation, use the root fixes, instead of working them around over and over again. I’d say we should make it so: “We support version of main dependency no older than n, but occasionally we may still use the newer versions of granular dependencies”

It's to be implemented on the plugin level only.
It helps with Problems 2-6 but puts pressure on merchants to frequently update, so we need to balance the n. Plus, it does not solve the problem at all, just limits its range.

I think it's a nice policy to start with, but we should not stop here, as we still face all the DevX frustration even with n=2.

Build/automate WP, WC, plugin version maps

I think that all the data is there (somewhere) we just need a nice and automated way to gather it all in a single place. As mentioned in the comment by @jsnajdr

We could be maintaining machine-readable information about which WordPress version ships which versions of the Gutenberg plugin and the NPM libraries

I’d only add to make it machine and human-readable, and for WooCommerce as well.

Conceptually it’s gathering all package.jsons for respective WP, WC, DEWP, and plugin versions and putting them in a nice table. I’d also add there the WP versions supported by a specific WC version.

Thanks to that a developer, support engineer, merchant, further tools could inspect what versions are used where and decide upon that.
It will help in

• debugging issues, to know in which version of a package should we look for a bug.
• support engineers communicating with customers, as one could easily check that a given plugin uses a granular dependency that's far behind the WP they use, which may be a reason for problems
• developing a plugin, as a developer could check:
  • which packages were actually extracted, where they should put additional caution
  • what granular dependency versions range is supported in the range of WP & WC they support, so know which features could be used, and which should be worked around
  • versions in the upcoming WP/WC release to be able to anticipate it and fix compatibility problems before they arrive to support engineers
• testing a plugin, as we could set up automated tests for the combinations that would be actually in use
• implementing further solutions for the dependency problems, as the build- and run-time code could take and process that information, to serve the code to be imported/executed.
• gathering the data about the scale of the version problems, as we will finally see if/how far the ranges do not match. We could even track/report that data on production builds.

It solves 3, 5, 6; helps with 2, 4.
Requires action/changes in WordPress, WooCommerce, or DEWP repos.

I think it's something we should start with, as it would give us an insight and data to reason about the problem.

Make Dependency Extraction Plugin handle dependency versions

I believe we have all the data in place to solve the problem, assuming we already have a way to get granular dependencies’ versions for a given WordPress version. The local package.json gives the versions for dependencies in us. WP version (range) is also given in the repo.

I’m not very experienced with Webpack. But to me, the feature that DEWP brings is similar to what native Import maps do: “Check what dependencies are already available in WP/WC and map those imports to external modules” (instead of adding and looking them up in the local bundle).

What’s cool about native import maps (besides the fact, it’s native, already available for free in Chromium, and does not require us to complicate our tools and stack) is that they seem to solve the problem of multiple versions problem – with scopes. So if someday we’ll switch to native ESM, we could use a single map for all plugins, without a need for each plugin adding DEWP to their tool stack.

I’ll use import maps syntax as I believe it’s clear and declarative enough to express the behavior we’d like to have:

Consider the given WooCommerce version uses

"@woocommerce/components": "5.1.2",
"@woocommerce/currency": "3.1.0",
"@woocommerce/number": "1.2.3",

Then the import map for a plugin that would like to use shared dependencies would look like

"imports": {
    "@woocommerce/components": "/wp-content/…/woocommerce-admin/…/components.js",
    "@woocommerce/currency": "/wp-content/…/woocommerce-admin/…/currency.js",
    "@woocommerce/number": "/wp-content/…/woocommerce-admin/…/number.js",
},

That’s what we have today. And AFAIK, that’s where DEWP functionality ends in terms of versions. But hopefully, we can add a bit of checking logic there.

Overlap

If our plugin uses

"@woocommerce/components": "^5.1.1",
"@woocommerce/number": ">=1.1.1 <1.2.0",
"fast-json-patch": "^3.0.0",

Then all the shared dependencies are there, so the bundle will include only fast-json-patch from gla/node_modules/, and the @woo… dependencies will be mapped as above.

Newer version

If our plugin uses

"@woocommerce/components": "^5.1.1",
"@woocommerce/number": "^2.0.0",
"fast-json-patch": "^3.0.0",

@woo…/components version matches, so will be removed from the bundle. But the …/number does not, so will be added to the bundle together with fast-json-patch, and the import map will be modified to look like this:

"imports": {
    "@woocommerce/components": "/wp-content/…/woocommerce-admin/…/components.js",
    "@woocommerce/currency": "/wp-content/…/woocommerce-admin/…/currency.js",
    "@woocommerce/number": "/wp-content/…/woocommerce-admin/…/number.js",
},
"scopes": {
    "/wp-content/plugins/gla/": {
        "@woocommerce/number": "/wp-content/…/gla/…/number.js",
    },
},

(So the imports originating from …/gla/ in GLA bundle will point to GLA-specific version)

Native import maps are resolved on the run-time, so they could use the WP/WC dependency maps from the currently running setup.
With Webpack, we need to do it on the build-time, so we would have to consider the ranges of granular dependency versions from the range of main dependencies. But the logic stays the same.

It solves 1, 2, 4, 5, 7, 8.
Requires action/changes in WordPress and DEWP repos.

Add runtime import map shim/resolver

This one could be the trickiest to implement, but if a given WordPress version knows its own granular dependency versions and receive the plugin's map, then theoretically it knows whether the versions match. If so it would do what it does today, and return the import from the WP bundle. If not could point back to the plugins bundle. So the "extended" bundle would be requested only if the currently running WP environment does not have a matching dependency.

However, I don't know enough on how WP handles all the scripts, to propose something more precise, or judge how doable it is.

It may solve all the problems, but would require a lot of changes across the stack.

Just spitballing some ideas:

Sounds like Webpack Module Federation could help here. The official documentation says

Many applications share a common components library which could be built as a container with each component exposed. Each application consumes components from the components library container. Changes to the components library can be separately deployed without the need to re-deploy all applications. The application automatically uses the up-to-date version of the components library.

If we replace application -> plugin and components -> main dependencies, it really sound similar to the problem exposed above.

Maybe we can replace DEWP with Webpack DLLs. Maybe we can publish NPM packages with the DLLs for specific WP versions (eg: @wordpress/wp-dll). This package will contain a pre-built DLL that plugin authors must include in their Webpack compilation. It will take care of extract the main dependencies.

This package could also provide a set of peerDependencies with the exact version of the expected libraries (i.e like your "Conceptually it’s gathering all package.jsons for respective WP, WC, DEWP, and plugin versions and putting them in a nice table.", but in JSON format).

So MyPlugin depends on @wordpress/wp-dll@5.8.1. Via peerDependencies, that forces my plugin to also depend on specific versions of other dependencies (eg: @woocommerce/components@5.0.0). This allows me to run tests with the exact versions that will be used in prod. It also includes a DLL config that I need to plug into my Webpack config, so it knows which packages to extract.

Although that would mean I need to maintain several "lines" of MyPlugin: one compatible with @woocommerce/components@5.0.0, another one compatible with @woocommerce/components@4.0.0... So maybe this is not as good as it sounded in my head when I started this comment :)

I also have one thing that I think causes a lot of pain points when it comes to working with the DEWP. And that is the inability to tree shake. If you use a singular function from lodash for example it adds the dependency lodash and therefore the entire library gets loaded. Which is a lot of overhead for a small simple function. And since the request is an external running something like the WebpackBundleAnalyzer won't know about it and therefore it can be hard to spot the actual cost of importing a package.

Regarding your comments, I actually like the idea of a runtime import map shim/resolver quite a lot. Core currently doesn't ship with multiple versions of the packages and I'm not sure whether it would be wise to do so. Of course, you could always bundle all your dependencies with your plugin but as you mentioned that leads to the same code being imported by multiple plugins and therefore a lot of overhead. So the idea of only loading a bundle if the version isn't a match is very intriguing. I also have no clue however how feasible it would actually be to implement something like it.