I can't reproduce the issue

Hmm
You are right.
If we do not select again the image when editing the article, the path is not added.

This means we have to enter the full path to the image in the Default Value

I have been playing a bit further with this.
The default_value field is a textarea and is common to all custom fields.
Could not find an easy way to override the field type in this case.

The following patch could partly (see Note below) solve the issue, but only for the frontend display as it will not allow the preview of the default image when editing the article.

diff --git a/plugins/fields/media/tmpl/media.php b/plugins/fields/media/tmpl/media.php
index 1707e09..981093e 100644
--- a/plugins/fields/media/tmpl/media.php
+++ b/plugins/fields/media/tmpl/media.php
@@ -9,4 +9,6 @@
 defined('_JEXEC') or die;
 
+use Joomla\CMS\Component\ComponentHelper;
+
 if ($field->value == '')
 {
@@ -22,4 +24,8 @@
 
 $value  = (array) $field->value;
+
+$directory = $fieldParams->get('directory') ? $fieldParams->get('directory') . '/' : '';
+$imagePath = ComponentHelper::getParams('com_media')->get('image_path', 'images') . '/';
+
 $buffer = '';
 
@@ -31,4 +37,9 @@
 	}
 
+	if (!strpos($path, '/'))
+	{
+		$path = $imagePath . $directory . $path;
+	}
+
 	$buffer .= sprintf('<img src="%s"%s>',
 		htmlentities($path, ENT_COMPAT, 'UTF-8', true),

Note: I am using above the variable $imagePath.
For this plugin, it will always be images/ as the directory is defined in the xml.
This is an important limitation.
The plugin should be improved to let use the default "Path to Images Folder" defined in com_media configuration.

This means we have to enter the full path to the image in the Default Value

And in field type Imagelist we only have to enter the image name. That's at least inconsistent or zero UIX.

Back to field of type media:

Try this (two different directories). Now the setting Directory is completely useless:

Go into article. In field click Select or Clear and then Select. The media manager opens images/banners/ but not images/sampledata/fruitshop.

And in field type Imagelist we only have to enter the image name

Is that working really?
I always get The default value is invalid.

I always get The default value is invalid.

I forgot to mention pr #24643

Maybe also of interest in this context: #23568