This sounds awesome. I would be happy to test for you.

Good to know @uglyeoin :)

@shandak could you please expand on the testing instructions to aid better testing?

@shandak could you please expand on the testing instructions to aid better testing?

Perhaps turning on debugging?

even turning debug on, testing instructions doesn't help too much .. or it's me ?

i'm only able to see a small (&random) less memory footprint on admin side ....

@shandak please provide more testing instructions ...

your pr's sounds good but quite hard to test

For test it better you can add strings below

static $count = 0;
\Joomla\CMS\Factory::getApplication()->enqueueMessage((++$count) . ' Loaded ' . get_class($this));

right after __construct() there https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Plugin/CMSPlugin.php#L65 to start counting(debugging) how many and which plugins were loaded with and without the fix during tested page loads.
For instance, for the demo raw Joomla, I have 18 loaded plugins

and with the applied fix I have 12 loaded plugins, which means only these 12 plugins actually need to be instantiated. And 6 plugins were loaded but was never used.

Of course with any other added extensions this amount is increasing quite fast. For instance, the difference in my latest project is 131 vs 66 loaded plugins.

BTW some of the events trigger when a page was already sent such onAfterRespond. So to get the full list of used plugins you need to save messages above to a file and check its rendered content right after the page has been loaded.
You can use the next code instead of mentioned above to check plugins list inside the logged file usedPlugins.txt

static $count = 0;

if ($count == 0)
{
	\Joomla\CMS\Filesystem\File::delete('usedPlugins.txt');
}

$message = (++$count) . ' Loaded ' . get_class($this);
\Joomla\CMS\Filesystem\File::append('usedPlugins.txt', $message . "\n");
\Joomla\CMS\Factory::getApplication()->enqueueMessage($message);

The plugin system has got many changes in j4 so we can't merge this in to j3 because its a new feature.

Could you please rebase this PR on J4?

This change is actually not going to fly even with the refactored event system in J4. Look at the two screenshots, it shows that the import order has changed and there are now system plugins being loaded and executed after other plugins. Any performance enhancement in the event dispatcher MUST retain the ordering of its listeners otherwise it is not going to be acceptable.

All events inside plugins(system or not) are being loaded exactly in the same place and in the same ordering as it must be. I made something like autoloading for all plugins so they will be initialised once when any of its events must be executed.
So that the key to improvement - if no one plugin's event was not being asked then the plugin will not be initialised at all.

Long and short, this is actually a major B/C break and cannot apply to J3 in any sane way because of how it alters the instantiation behavior of plugins. Just look at when the constructor of the system debug plugin is executed and that demonstrates the point.

Generally, it is not an event dispatcher's responsibility to instantiate a class. Rather, it should receive an instantiated object when you specify listeners for events (or you can get creative and decorate your listeners with some kind of factory that defers instantiation).

There might be a performance overhead by loading in unnecessary classes, but as things sit right now architecturally it is actually done correctly and this change introduces major side effects that cannot be overlooked.

Right, thanks for the explanation!
Then what about to change approach only for J4? We can add a new specific JEvent property, for example public $lateLoading = true by default. And set it false for debug plugin and for other things where it's necessary. I bet it needs quite rare. It also can be used for customer's plugins if they want to. When people start migration they will be aware about this feauture.

So for this PR I need to add one more check, for example by get_class_vars, mark plugins such debug as public $lateLoading = false and force load these plugins as usual.

JEvent and JEventDispatcher are gone in 4.0. So a lazy loading flag inside the plugin class isn't viable.

In 4.0, there is a Joomla\CMS\Event\LazyServiceEventListener class which is kind of like what you're aiming for. But, this class' design doesn't support a subscriber (as it's called inside the event API) and it doesn't support plugins because (as the name implies) it relies on pulling classes out of the DI container.

Since the Joomla\CMS\Plugin\CMSPlugin class is responsible for registering itself to the event dispatcher in 4.0 (because it is also handling all the bridging logic from the older JEventDispatcher class to the newer Joomla\Event\EventDispatcherInterface, architecturally you're kind of stuck now with needing to instantiate the plugin to get it into the dispatcher correctly. Or, there's going to need to be some re-thinking of how plugin classes are created to break this bit of class inheritance as there's really no major need for a plugin to inherit from a root class other than some convenience code (auto-register app and database as class member variables, helper method to load language files, params injected from the Joomla\CMS\Plugin\PluginHelper class) and then you can come up with a solution to be able to lazy load plugins, knowing the penalties that will incur (i.e. the late instantiation stuff).

Just shooting from the hip here, but the Joomla\CMS\Plugin\CMSPlugin class in 4.0 doesn't apply any legacy behaviors if the plugin implements Joomla\Event\SubscriberInterface. Maybe in the CMS you define a Joomla\CMS\Event\LazySubscriberInterface extending Joomla\Event\SubscriberInterface (and for all intents and purposes that can be an empty marker interface), and if a plugin implements that lazy interface then how it gets registered to the dispatcher is changed. I imagine at a minimum this would require some logic in the Joomla\CMS\Plugin\PluginHelper class to check if the plugin implements that interface and if so trigger the new code for "lazy" plugins otherwise do what it's doing now, and add a check in the Joomla\CMS\Plugin\CMSPlugin::registerListeners() method to not do anything if the plugin implements that interface.

I'm closing this PR. Thanks for your help. I'll try to play around J4 and the event's interfaces when I have time.