Steps to reproduce the issue

i do not have steps but it happens in cases of 2 error in the publish task of (J3.7.5)JModelAdmin or (J3.8+)AdminModel
JLIB_APPLICATION_ERROR_EDITSTATE_NOT_PERMITTED
JLIB_APPLICATION_ERROR_CHECKIN_USER_MISMATCH

[EDIT] short explanation
publish task , call model->publish , which return without setting error via setError
publish task look at getError, see none, and assumes success

Also faster to understand is to look at this code lines:
https://github.com/joomla/joomla-cms/blob/staging/libraries/src/MVC/Controller/AdminController.php#L211-L224

https://github.com/joomla/joomla-cms/blob/staging/libraries/src/MVC/Model/AdminModel.php#L1005-L1024

And also in then menus controller identical code as (J3.7.5)JControllerAdmin or (J3.8)AdminController
https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_menus/controllers/items.php#L224-L238

Detailed Explanation of the issue
The publish method of (J3.7.7)JModelAdmin or (J3.8)AdminModel

• for the errors mentioned above it will return false without setting an error by calling $this->setError()

Instead the error is added to the log file via JLog

Looking at the code it looks like

• the intention was to exclude the ID given the error and continue with changing the state of the rest of the record IDs, but ... then it execution continues returning false without calling setError()

Then back at the publish() control task of (J3.7.5)JControllerAdmin or (J3.8)AdminController, it does not check if model->publish() returned false !

• instead it checks if there were errors via $model->getError(), and since no error Messages were added it assumed that model->publish() was a success

Expected result

There model errors are show as failure by the publish() controller task
JLIB_APPLICATION_ERROR_EDITSTATE_NOT_PERMITTED
JLIB_APPLICATION_ERROR_CHECKIN_USER_MISMATCH

Actual result

They are shown as success

System information (as much as possible)

J3.7.5 / J3.8 staging

Additional comments

I think the intention of the code since the bad $PK is being unset,
was to allow the publish task at controller to decide if it will show the error or continue and call the model->publish task again, or something like this

An example of this happening with publish task of menus controller is here (note that menus controller has same code as (J3.7.5)JControllerAdmin or (J3.8)AdminController)

#15938 (comment)