The error occurs under certain circumstances, but is not directly visible. I'm also not quite sure how to best describe the steps to reproduce. So let me just describe what I found:

When uninstalling library packages, installer script.php methods are not executed. The problem is that during uninstallation, script.php is looked for in a different location than where it was copied during installation.

During installation, script.php is copied to the the extension_root, i.e. the base directory of the library. See Joomla\CMS\Installer\Adapter\LibraryAdapter::finaliseInstall() (.../libraries/src/Installer/Adapter/LibraryAdapter.php, line 144). That line reads:

$path['dest'] = $this->parent->getPath('extension_root') . '/' . $this->manifest_script;

During uninstallation, script.php is looked for in in a manifests sub folder. See Joomla\CMS\Installer\Adapter\LibraryAdapter::setupUninstall() (.../libraries/src/Installer/Adapter/LibraryAdapter.php, line 376). That line reads:

$this->parent->setPath('source', JPATH_MANIFESTS . '/libraries/' . $manifest->libraryname);

The problem also occurs during a library upgrade, because it's uninstalled first as part of the process.

I managed to get the expected result by replacing the line with the setPath() in LibraryAdapter::setupUninstall() with the following:

$this->parent->setPath('source', $this->parent->getPath('extension_root'));

Not sure if it is the proper solution, but it does seem to solve the problem.