I "think" this is because if you have two menus pointing to the same content the current joomla router will always chose the one with the lower itemid - I could be wrong though.

No it is clearly because ContentHelperRoute::getArticleRoute does not look at com_tags menu. It is explicit in the code :

$menus    = $app->getMenu('site');
$component  = JComponentHelper::getComponent('com_content');

$attributes = array('component_id');
$values     = array($component->id);

$items = $menus->getItems($attributes, $values);

foreach ($items as $item)
{
    if (isset($item->query) && isset($item->query['view']))

I'd borderline call it expected behavior or something that can't be fixed in 3.x. The component route helpers all only look for menu items for their own component and generally for a specific type (so getArticleRoute() is looking for a menu item route explicitly for a com_content article, it's not traversing the menu item tree), none support a notion that a parent menu item would come from another component or system menu item (alias for example).

No component supports routing with a tag ID, and likewise com_tags can't route to a content page in a component.

Could we at least consider that joomla not to erase/replace the Itemid of the page on which the article is shown ? Even if does not look to Itemid for other component, that would made the trick at least in this example.

Lets's say the itemId of 'CategoryMenu' is 100, and itemId of 'TagAMenu' is 200.

We are on the 'TagAMenu' page (with thus Itemid=200 in the url), and getArticleRoute() replaces set as all the articles Itemid to 100 ! There should be a test like this (sorry certainly not the right syntax)

$origItem=$request->get('Itemid');
if (($origItem) && (((int)$origItem)>0)) 
        {
            $link .= '&Itemid=' . $origItem;
        }

else if ($item = self::_findItem($needles))
        {
            $link .= '&Itemid=' . $item;
        }

I don't think the route helper needs that test personally, those functions are designed to do menu item lookups. If you really wanted to route that way you should be able to just route it in a layout override:

JRoute::_('index.php?option=com_content&view=article&id=' . $articleId . '&Itemid=' . $request->getUint('Itemid'));

The route helper is essentially giving you back that same string to put into JRoute, so you could save a few CPU cycles and just build it yourself.

You suggest (I'm a Joomla newbie) an override at the com_content.article.tmpl level ? Or you're speaking of another type of override ?
And could a plugin be a good solution ? This could be distributed to all the ones working the same way I do.

Closed as expected behaviour as outlined by @mbabker