Thank you for this PR

If I clear out the new option I can still see previews!
This is a known issue impacting all fields with a default value and we have an open bug report for it

Does the same change need to be made in com_templates as that has its own set of supported image types?

@brianteeman The short answer is ‘no’.

Here's the longer answer with the reasoning.

Regarding the ability to upload and list files, the Template Manager has its own media settings in its Options. To be able to upload, for example, SVG files you need to set “Valid Image Formats” to gif,bmp,jpg,jpeg,png,webp,svg. Doing that also lists the file in the file tree in the sidebar. This is already possible and nothing needs to change.

Regarding the ability to show files, the Template Manager shows you all files listed even in the “Valid Image Formats”. However, opening image files is limited to raster images so you can resize or crop them. We cannot add vector image support there. So, again there's nothing to do. You can see the file, that's about it.

If someone wanted to change something, that would be showing an inline preview of files which are neither raster images nor code files (like PDFs, SVGs and so on) for the sole purpose of having a preview and being able to rename or delete files. However, on one hand this is a completely different scope to this PR here, on the other hand I am not terribly familiar with the Template Manager's code and wouldn't want to risk touching it at this point in time. If a brave soul is wiling to do that I am willing to test it!

If a brave soul is wiling to do that I am willing to test it!

I will put it on my todo list when this is merged

@nikosdion it's nice that you're fixing this but the implementation is not ideal, read my comment on this here: #34634 (comment)

@dgrammatiko I have already considered what you suggested back in March and April 2020, long before you made your comment, when I was trying to add SVG support to my own sites (Joomla 3 and my Joomla 4 blog which at the time was using the J4 beta because I like living dangerously and strongly believe in dogfooding).

Your suggestion (and, to be fair, what I also had in mind at the time!) requires having default values for a subform field. This is currently impossible in Joomla. Your only choice is to add some really complicated code to read a JSON–encoded default value if and only if there is no other value in the form (which requires manually decoding the form source instead of reading the value of the field through the form object) and convert it to the array format expected by the subform field. This is very impractical to do in a generic component like com_config as it requires knowing which fields require this type of handling. It's also a hack.

Modifying the subform field to accept this kind of default values is a hard b/c break which cannot happen until Joomla 6.0 at the earliest. Even we wanted to do that, we'd need to provide an interim solution in 5.0 which supports both the old and the new subform default values which is impossible.

Even if we magically solved these problems we still have an issue. Upgrading a site to this new version would require reconfiguration of the media settings but there is no obvious way to tell the user this is needed beyond the often overlooked post-installation messages. Even if had a way to do that, users cannot see what their previous settings are, therefore they have no idea what to do. For us not to break their sites we would need an automated migration tool which runs during the upgrade which would have to reconcile what is most often than not competing and contradicting settings into something that makes sense.

Even if that was magically solved as well, we would be screwing third party developers who were told that the One True Joomla Way™ is to copy these media fields into their own extensions' Options. So not only do we break backwards compatibility hard (an extension written for 6.x would not run on 5.x and vice versa), we also require them to come up with their own migration code.

All of these problems have a very high potential of breaking Joomla and 3PD extensions, as well as existing sites. What for? To save 10' on a screen that you only use once when setting up the site? It's not worth it.

So, no, I am not doing it wrong, I am doing it the only way which is practical especially given the PLT's commitment to NOT breaking b/c hard in the upcoming major releases. I actually have experience managing mass–distributed software which is maintained over decades, you know…

This is currently impossible in Joomla.

That's false I'm doing that for years...

it's also a hack.

No it is NOT. Fields CAN have structured data (JSON), instead of plain text, bool, arrays. Actually that's what repeatable fields are also using

Modifying the subform field

You don't have to modify anything, the UI part needs some adjustments (js)

Upgrading a site to this new version would require reconfiguration

False: we keep the old fields as hidden and we sync them to any changes on the new ones, or vice versa

we would be screwing third party developers who were told that the One True Joomla Way™ is to copy these media fields into their own extensions' Options.

Again you can shadow the existing fields so for 3rd PD there's nothing broken till they catch up with the new code (eg v6)

Anyways, you can add the additional field I'm not objecting here. What I'm saying is that from an architectural point of view it's better to have ALL the data regarding each file type in a structure (OOP) otherwise each time that there's a need for something more you end up with another field and you know a gazillion of fields is what makes this CMS a nightmare...

OK, then, I am closing this PR so @dgrammatiko can show us his code and how he solves all the b/c issues. I am not wasting any more time with Joomla since Dimitris knows how to do everything better than everyone else.

In the meantime nobody can have previews of SVGs in Joomla because Dimitris says that any solution other than his is wrong. Everyone who is upset with that can tag @dgrammatiko and tell him what they think. I am done arguing with idiots.

Useful and necessary correction for everyone using SVGs in Joomla.
I totally agree with you! I tested this successfully.

I have tested this item ✅ successfully on 14fc317

Tested this successfully.

Please reopen...

I have tested this item ✅ successfully on 14fc317

Do we really need a new option? I do not see a need for a new option here as all elements which deliver a thumb_path variable should be rendered as preview in the media manager. So when I do an adapter which can deliver thumbnails for pdf's or videos, then the media manager should be able to display them, regardless of the file type.

The opposite is when a user adds a file type which doesn't have thumbnail support, then he/she gets confused why no thumbnails are shown.

Do I miss here something?

@laoneo In the PR I have technical notes. Let's read together the first thing I wrote there ;)

Why not reuse the “Legal Image Extensions (File Types)” option?

Well, for two reasons. The “Legal Image Extensions (File Types)” controls which file types are allowed to be uploaded through the Media Manager and selected in Media fields. In many cases site integrators need to allow uploading and/or selecting PDF files which are all too common in the business and non–profit world. Whether treating PDF files are media files and/or using the Media Manager as a generic file manager is a good solution is highly debatable, but given the limited file management tools the core Joomla! CMS provides it's a reasonable solution.

So here's the first reason. PDF files tend to be big and do not have thumbnails. If we let PDFs to be previewed when they are merely allowed to be uploaded the Media Manager would get bogged down. Having 100 PDFs at 2.5MiB each on average would mean that accessing the Media Manager page for that folder would require transferring 250MiB of unnecessary data. Groan.

The second reason is that previewing and allowing files to be uploaded are two distinct use cases. For example, if I know my clients are on a rural area, thus severely bandwidth limited (2Mbps ADSL or even 56Kbps dial-up is still a thing in remote areas), I would much rather prefer them to NOT see any image previews at all.

By the way, the example with the big PDFs is not out of my butt. I have several clients doing that. Yes, I know the Media Manager is not supposed to be a Downloads Manager but it's not like Joomla offers the latter, now, does it? It's hard to tell people they are doing it “wrong” when Joomla itself has not provided any way for them to do it “right”.

I need to check this with a large PDF file why this should be delivered in the list in the media manager.

I have tested this item ✅ successfully on 14fc317

RTC

Thank you very much for your work on this, Nicholas!

This is more than a bug fix, it is a big improvement! So I'm moving it to 4.3 :)

@fancyFranci Fair enough :)

So finally I had time to test your pr. I really do not understand why large PDF's have something to do with the issue here. PDF's are rendered as icons and not as preview images

So this leads me to the question why do we need an option? Even when I add pdf's as preview extension there is no preview image of it shown, which is obvious for PDF files. So this option leads to more confusion than it solves. I would expect that the thumb_path property is always be an image. There is no reason to have other file types as preview's.

If there is an issue with too big files as previews, then it is wrong to have an option in the media manager to turn off previews for all images. We should rather fix the local adapter to not send the full image as preview. Since #36930 the images are lazy loaded so it can even be accepted that full images can be delivered as previews, as only the amount of images are fetched which do fit into the viewport.

Just to be clear here I do like the change to be able to render svg's as previews, but not another option which is hidden in the media manager that turns off globally for all adapters preview images and can lead to an unexpected behavior. The availability of preview images should be controlled by the respective adapter who delivers the media data and not globally in the media manager. And this is controlled by the thumb_path property. I mean the local adapter does the check already that only files are delivered as preview images which are actually images here https://github.com/joomla/joomla-cms/blob/4.2-dev/plugins/filesystem/local/src/Adapter/LocalAdapter.php#L342.

@laoneo Add svg and pdf to the image file types before the PR. What happens?

Regarding thumbnails, the way Joomla is doing it is obviously wrong. It should be making and caching thumbnails. I have ran into this issue with my S3 filesystem adapter. Yesterday I was trying to come up with a solution, my notes are in akeeba/plg_filesystem_s3#1

PDF looks the same and SVG has no preview. Why are you asking?

That's why we made the media manager so extensible, that extension devs can write their own one with whatever functionality they need. I have caching and thumbnail support in DPMedia since the early beginning.

PDF looks the same and SVG has no preview. Why are you asking?

Okay, this confirms my suspicion: you have not added pdf to “Legal Image Extensions (File Types)”. This will make it easier for you to follow my thinking.

Now try to use a custom Field in an article of the Media type. You will see that you cannot select the PDF file. The only way this works is if you add it to the Legal Image Extensions (File Types) configuration option in com_media.

Before this here PR, this would result in the media manager showing the “no preview” (eye-slash) icon. All is good and well.

HOWEVER this also means that other image file types in that field don't get a preview either! This is what I am trying to fix with this PR.

This means that I have two options:

1. Reuse “Legal Image Extensions (File Types)” for deciding which files should have previews rendered. In this case, though, your browser now tries to load the entire PDF file. Add hundreds of PDF files and try to scroll to the file you are looking for and you'll see the problem I am talking about.
2. Introduce a new option so that defining which extensions are images (therefore selectable in a Media field, either form field or custom field) is decoupled from which file types can be previewed.
3. Be a dick and delegate that responsibility to the developers of the filesystem plugins with maddening UX issues, such as that some plugins will allow you to define this per plugin, or per source whereas others will have a hardcoded list.

“But, Nicholas, why would the user ever use the media field for PDF files instead of a URL field or pasting a link?!” I hear you ask. Several reasons, Allon. Several.

1. Joomla does not have a downloads manager nor does it have a download field type
2. People updating the content are non-technical end users. Imagine telling someone that “Go to the article text. Click on the image icon. Find the PDF file you want. Put your mouse over it. Click the three dots which magically appear in the top right. Click the icon that looks like chains; no, not this one, that one. Click on Get A Shareable Link. Click the icon that looks like an old-fashioned notepad that appeared next to it. Click on Cancel and again Cancel; yes, there are two buttons called Cancel, Todd. Now look up, there are tabs on the page. Click on the one called Related Documents. Click inside the field called Attachment. No, Todd, INSIDE. Yes. Now right click. Yes, Todd, right click is clicking with the right button of your mouse. Oh, sorry, you're left handed, yes, I mean the left button for you. Yes, it's still right click. No, it makes no sense. JUST DO WHAT I TELL YOU TO DO, TODD! Now click on Paste.” versus telling them “Click on the Related Documents. Find the field called Attachment. Click the select button. Find the PDF you want and click on it. Click on OK”. Which one you think is easier for Todd to understand, remember and follow?

And no, telling them to use a third party extension and custom field plugin to manage freaking PDFs which are the absolute norm of sharing information in non-profit and corporate settings is not an option. To the user this is Media. We either offer a Media Manager or not.

If the project is okay with either having Todd follow a byzantine process or his browser crashing when loading hundreds of PDFs I am happy to modify this PR and note that I am doing this under duress, so I bear no responsibility for the clusterfsck which will ensue.

That's why we made the media manager so extensible, that extension devs can write their own one with whatever functionality they need.

I vehemently disagree that creating and caching thumbnails for images is not something the core should implement. And I do NOT mean as a core filesystem plugin functionality, I mean as a core Media Manager feature which third party plugins can opt into.

Let's look at how WordPress manages media and why people have been saying for over a decade that its solution is superior. Each media file is its own database record. WordPress generates thumbnails automatically for uploaded media. WordPress displays thumbnails when you are selecting media.

Even if we do not want to create media records in the database (therefore giving us a Media FILES Manager, NOT a Media Manager) we can at the very least provide thumbnails, and load thumbnails instead of the massive image, thereby both previewing all image types equally and making the load on the server and the browser much lighter.

In the end of the day, did anyone take into account the users' needs when writing the new Media Manager (which at this point is already 5 years old)? Or did y'all go code happy trying to put lipstick on the same old pig, merely adding support for third party filesystems?

If you work with any professional setting which really needs a media manager you'll know that what they need for each image file is titles, descriptions, tags (keywords), thumbnails, copyright information, license information, who can use which image/file types (permissions) and so on. The Media Manager hits zero of the requirements which is why nobody likes it. It does not work well as a Media Manager, it does not work well as a File Manager, it's the worst of both worlds melded together in a barely usable heap.

Dammit, we've been having this discussion since Joomla 1.5. After 15 years we still have a media manager which does not manage media and does not even work well as a media files manager. At this point I don't know what else to say. I was just trying to add SVG support to these steaming pile of horse poop and all I get is “nah, we don't need this, it's already extensible, it's fine as it is”. Maybe I should give up and try finding a better media manager solution for Joomla. Any suggestions for alternatives welcome.

I am closing this PR for good. There is no point trying to make things work with Media Manager as it is. I will just revive my SVGMagic plugin which does in memory patching of the crappy Joomla core code to patch SVG support into the MediaHelper and call it a day.

its a bit odd for @laoneo to review this in this way as he has competing commercial solutions

my favourite file uploader offrs pdf previews https://github.com/Adri-Glez/filepond-plugin-pdf-preview

@brianteeman The thing is, even if a plugin does pass the undocumented thumb_url (only deduced by looking at the code and the data returned to the Media Manager) you still cannot preview image types the Media Manager's hardcoded list won't allow. So it's not a matter of anyone having a competing solution. If that was the case, I'd just write a bloody plugin for the local filesystem.

The caching etc was a sidebar. It is not related to the PR so it's within Allon's rights to tell me that he's already doing it (which I am already aware, Crystal has already showed me and I have seen what happens with large folders, thank you very much).

I personally don't see why the core would provide a subpar solution lacking fundamental features which would make it useful but at the same time I know what is involved with making thumbnails for the current Media FILES Manager so I see why it's not already in the core. So I would say it's not Allon acting maliciously, it's that the way the Media Manager works right now, thumbnails and their caching require more hands-on support which can only be provided by 3PDs and definitely not by the project.

So, as long as we have this subpar Media Manager concept we are stuck with a subpar experience. Quelle surprise…

Okay, this confirms my suspicion: you have not added pdf to “Legal Image Extensions (File Types)”. This will make it easier for you to follow my thinking.

Yes indeed, because a PDF is not an image as soon as I added it, the preview of the pdf shows up. So this mystery is solved.

I really ask you to make a step back and have a look on the media manager with eyes from a beginner. Why should the admin need to go to the media manager and add svg to the list of preview images? What is the criteria to enable svgs or not? How should the admin decide?

Yes indeed, because a PDF is not an image

unfortunately users dont agree

Why should the admin need to go to the media manager and add svg to the list of preview images

they should not need to but pd decided with jsst that svg would not be enabled by default

@laoneo Again, Not only am I looking it with the eyes of a beginner, not only have I actually held discussions with real world beginner, intermediate an advanced users but I also explained exactly why a user would a. use PDF as an image file format and b. why they need SVG support.

Okay, let me repeat myself for the last time. I am not going to respond to this PR anymore after this.

WHY DO YOU NEED TO ADD PDF AS AN IMAGE FILE FORMAT

Users need a way to add attachments to their articles.

Joomla has a kind-of file manager which kind-of displays previews for some files and allows you to select even less which is weirdly called "Media Manager" even though it does not do media management.

Media "Manager" allows by default to upload PDF files but does not allow by default to use them in Media fields. Weird.

The only way for this to happen is to use PDF as an image file. Even weirder, but it works.

The user wants to hammer nails. We gave them a screwdriver. Of course they will use the screwdriver as a hammer!

WHY ADD SVG TO THE IMAGE TYPES

It's 2022. Browsers support vector graphics in the form of SVG files since 1999.

People want to use SVGs for illustrations.

They are smaller, more flexible and can be styled with CSS — including states like hover and active or even dark mode — without requiring yet another big raster image file.

Smaller files mean faster sites.

Faster sites make visitors and search engines happy.

Therefore we need to use SVGs.

By default Joomla does not allow uploading of SVG files. You need to add that to the image file types option for the reason Brian mentioned.

By default even if you do that you cannot preview the SVG file.

If you cannot preview an image you cannot select it.

Joomla's media thing “helpfully” clips the filenames.

I can't see the filename (which might mean nothing anyway) and I can't see a preview. How am I gonna find what I am looking for?

Hence this PR.

WHY MEDIA MANAGER SUCKS AS A FILE MANAGER

The file managers in Linux (Dolphin, Nautilus, ...), Windows (Windows File Manager) and macOS (Finder, Path Finder, ...) do display previews of SVG files. Which is to say that Joomla's Media thing is far more useless than the generic file managers in Operating Systems which have remained largely unchanged for 10 to 20 years.

WHY MEDIA MANAGER SUCKS AS A MEDIA MANAGER

No thumbnail generation.

Slow, especially on large folders or over slow Internet connections.

No metadata management.

No tags.

No search.

Nothing the users ever wanted.

Because nobody ever asked the users how they want to use Joomla. Assumptions are made without any user research, without any user feedback.

WHY DID I EVEN BOTHER?

Beats me. I will stop trying to make Joomla's useless media manager work for users, even within the constraints it's operating in. It's futile. I guess the only way to fix the media manager is to write a new media manager and I don't have the time for it.

All the while we wonder why people move away for Joomla. Okay. Here's how the competition's Media Manager works. Note that the thumbnails were created when I uploaded the files by drag'n'drop.

Oh, and look, they can store metadata, too!

Yeah, PDFs are supported as well.

Look! The editor remembers the caption I set!

And apparently my alt text too. Imagine that!

It even does a simple gallery by selecting a bunch of media files.

This is why people leave Joomla. The competition lets them create rich content with a few clicks (the screenshots took me 3' to set up...). Meanwhile, we are still discussing whether we want to allow PDFs to be managed by the Media Manager, what use is to preview SVGs and pat ourselves in the back for adding remote filesystems when we have a Media Manager that makes it impossible to actually manage media.

Head. Meet desk.

@nikosdion The media field has a new attribute "types" in Joomla 4, which is not yet documented in our form fields documentation. See https://github.com/joomla/joomla-cms/blob/4.2-dev/libraries/src/Form/Field/MediaField.php#L109-L116 . Should that not allow to use PDFs in a medial field when PDS is among the allowed document extensions? I use the media field for video and audio, see e.g. here: https://github.com/richard67/mod_rfvideo/blob/main/src/mod_rfvideo.xml#L213-L220

My previous comment refers to this comment by @nikosdion

Media "Manager" allows by default to upload PDF files but does not allow by default to use them in Media fields. Weird.

The only way for this to happen is to use PDF as an image file.

When using types = „documents“ that should work.

Yes, that would work to select PDFs. Of course it's undocumented and doesn't change the fact that existing sites already have the wrong settings still.

In the end of the day, adding SVG support to Joomla is impossible because we either have to add it hardcoded in the MediaHelper which the production department doesn't want or we have to reuse the image file types which will screw up existing sites that had to work around problems in previous versions of Joomla.

It's okay. I have other tricks up my sleeve.

@nikosdion The media field has a new attribute "types" in Joomla 4, which is not yet documented in our form fields documentation. See https://github.com/joomla/joomla-cms/blob/4.2-dev/libraries/src/Form/Field/MediaField.php#L109-L116 . Should that not allow to use PDFs in a medial field when PDS is among the allowed document extensions? I use the media field for video and audio, see e.g. here: https://github.com/richard67/mod_rfvideo/blob/main/src/mod_rfvideo.xml#L213-L220

Meanwhile I've found the time and mood to add the missing documentation of that attribute here: https://docs.joomla.org/Media_form_field_type .

Is there any way we could have both this PR and #38805 working together? It would move the media manager forward with those major improvements from @nikosdion and @dgrammatiko.
Everyone agrees (and I should hope so) we need better previews and thumbnails for better performance and better user experience.
I see both PRs as good solutions and beat by miles the alternative.