? ? ? Success

User tests: Successful: Unsuccessful:

avatar jrseliga
jrseliga
20 Dec 2014

Summary

Allows articles and featured articles to be filtered by multiple access levels, authors, categories, and tags in Administrator.

Current Joomla! behavior

The article/featured articles search (filter) tools allow one choice per input, each acting as a logical AND.

New behavior

  • Each input still acts as the logical AND.
  • Each choice within a input act as the logical OR with all the other choices inside of that input.

Notes

  • The module "Article - Newsflash" has an option to select multiple categories, when for example two categories are selected the module pulls all of the articles from both categories. Since articles can only exist in one category this is the only logical behavior, not only for this module, but for any instance of a multiple select category field. As a result, it could be counter intuitive to implement other multiple select filters in a different fashion.
  • The content model already supported filtering based on multiple categories before this pull request was made (see condition starting at line 244). This pull request just enables it's use in the search filters.
  • Similarly it seemed intuitive to introduce virtually identical code blocks (behavior when multiple are selected) for Author, Access Levels, and Tags.

Testing

Administrator

Articles
  1. Create 3 articles:
    1. Article 1 - Category 1 - Tag 1 - Author 1 - Public
    2. Article 2 - Category 2 - Tag 1 & Tag 2 - Author 2 - Registered
    3. Article 3 - Category 2 - Tag 2 - Author 1 - Super User
  2. View Articles
    1. Use Search Tools
      • Just with a simple structure like this, there is a large number of potential test cases. Please refer to the New behavior section for an overview of the filtering logic
Featured Articles
  1. View Featured Articles
    • Repeat in Featured Articles Search Tools

Site

Currently the only core feature that resembles the use case for this is in the module "Articles - Newsflash". If similar "Search Tools" are developed for filtering a list of articles in the front-end, it could leverage this feature in the same way as the back-end.

Articles - Newsflash (module)
  1. Leave Categories option at "- All Categories -" only
  2. Select one tag up to n tags. Any article with any of the n tags should be displayed.
    • This can be limited by selecting multiple categories.

Acknowledgements

Thank you to @brianteeman and @nikosdion for taking the time to help with my first pull request!

avatar jrseliga jrseliga - open - 20 Dec 2014
avatar jissues-bot jissues-bot - change - 20 Dec 2014
Labels Added: ?
avatar jissues-bot jissues-bot - change - 20 Dec 2014
Labels Added: ?
avatar brianteeman brianteeman - change - 20 Dec 2014
Category Administration Tags
avatar brianteeman
brianteeman - comment - 20 Dec 2014

To test create 3 articles and 2 tags
Article 1 - tag 1
Article 2 - tag 1 AND tag 2
Article 3 - tag 2

Then use the search filter for tags to select the articles

Before the patch you can only select a single tag so a filter for tag 1 will return article 1 & 2
After the patch you can select multiple tags so so a filter for tag 1 will return article 1 & 2 AND
a so a filter for tag 1 & 2 will return article 2 & 3


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar brianteeman
brianteeman - comment - 20 Dec 2014

OOPS My test instructions may wrong

I assumed that searching for two tags would return all articles tagged with BOTH
Instead it selected articles tagged with EITHER.

If that is the intended behaviour then it works - but perhaps users would be expecting to only see items tagged with BOTH

AND

The patch removes the label Select Tags see screenshots whatever the expected behaviour of the filter this needs fixing

BEFORE
screen shot 2014-12-20 at 04 00 47

AFTER
screen shot 2014-12-20 at 04 00 51


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar brianteeman brianteeman - test_item - 20 Dec 2014 - Tested unsuccessfully
avatar nikosdion
nikosdion - comment - 20 Dec 2014

@test Successful

Regarding Brian's observations:

  • The search for multiple tags will return articles which belong to any of the tags. Looking for all tags would have a very adverse performance and memory impact.
  • The "Select an option" label can be changed, but it's tricky. See below.

To restore the "Select Tag" label in the field you need to make two changes:

In administrator/components/com_content/views/articles/tmpl/default.php add the line:

JHtml::_('formbehavior.chosen', '.advancedTags', null, array('placeholder_text_multiple' => JText::_('JOPTION_SELECT_TAG')));

ABOVE the JHtml::_('formbehavior.chosen', 'select'); line. The placement of the line matters!

In the filter_articles.xml file you need to add the line class="advancedTags" after multiple="true"

avatar brianteeman
brianteeman - comment - 20 Dec 2014

Fair enough on point one

Select an option does need to be resolved though or no one will know what
the option is they are selecting

On 20 December 2014 at 12:54, Nicholas K. Dionysopoulos <
notifications@github.com> wrote:

@test https://github.com/test Successful

Regarding Brian's observations:

  • The search for multiple tags will return articles which belong to any of the tags. Looking for all tags would have a very adverse performance and memory impact.
  • The "Select an option" label can be changed, but it's tricky. See below.

To restore the "Select Tag" label in the field you need to make two
changes:

In administrator/components/com_content/views/articles/tmpl/default.php
add the line:

JHtml::('formbehavior.chosen', '.advancedTags', null, array('placeholder_text_multiple' => JText::('JOPTION_SELECT_TAG')));

ABOVE the JHtml::_('formbehavior.chosen', 'select'); line. The
placement of the line matters!

In the filter_articles.xml file you need to add the line
class="advancedTags" after multiple="true"


Reply to this email directly or view it on GitHub
#5474 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar jrseliga
jrseliga - comment - 20 Dec 2014

Both vs Either

The intended behavior is for it to select articles tagged with either.

I used the core module "Articles - Newsflash" as the precedent for the decision. There is a field that allows for selecting multiple categories, when for example, two categories are selected it also displays articles in either. I assumed that would be consistent across all instances of multiple selections in core features, so I made the tags filter consistent.

Select Tag Label

There are two reasons I removed this. (In writing this, I saw the response by @nikosdion. Not sure if his solution addresses these, will test.)

  1. When a user picks a tag with the label there, that generic label persists when the page reloads. This is default behavior for when tag (and even category) form fields are set to allow for multiple (see Articles - Newsflash module's category option again to confirm).
    before

    If it's not there when a user picks a tag, they will see this after making a selection, which seemed more intuitive to me.
    after

  2. When the label is there it appears like any other tag in the field. When you expand Search Tools and see a tag labeled "Select Tag" and you click the "x" to remove that (with no other tags selected), in an effort to clear the Tag field and start adding your selection, the page reloads to match the change you just made. The problem being, that the page reloads with the same "Select Tag" option in the field again. As I mentioned in item 1, this is default behavior for form fields that can be set to allow multiples, so I didn't see a choice. The problem is kind of hard to describe, but testing it is simple.

    1. At line 78 of the XML (included in this pull request) append <option value="">JOPTION_SELECT_TAG</option>
    2. Administrator > Articles
    3. Be sure to "Clear" any filters in Search Tools
    4. Remove "Select Tag"
    5. Page reloads with "Select Tag" again.
avatar nikosdion
nikosdion - comment - 20 Dec 2014

Please try the code I kindly provided. It has NONE of the drawbacks you mentioned. As you will see if you do read my reply very carefully I did NOT add the tag. I completely sidestepped it by tricking Chosen into providing a different label when you use the special class name "advancedTags" on the field.

If you are interested in the underlying mechanics of my solution: the label is enforced on the field by Chosen. In default.php we already load Chosen, telling it to hijack all tags. However, since there are no options provided, the default JHtmlFormBehavior::chosen lines 68:71 kick in and apply the default label JGLOBAL_SELECT_SOME_OPTIONS. The solution to that is using a different instance of Chosen, with a different selector and a different label. That's why there are two parts in my solution.

FIRST: You need to apply a custom class name to the field. This is done in filter_articles.xml by adding the class= line.

SECOND: You need to add a custom Chosen instance against the advancedTags classname and with a custom label. This is what the line I told you to add in default.php does. However, you can't add it anywhere you please! The multiple Chosen instances run in the same order they are loaded. This is why I told you that it's extremely important to place the new line ABOVE the existing behavior.chosen line.

Please try the two super simple changes I gave you and try to understand why it all works when put together. About an hour ago I had no idea how to fix that issue either. I read the code, tried to understand it and work through the solution :)

avatar jrseliga
jrseliga - comment - 20 Dec 2014

As @nikosdion suggested, making those changes resolves the issue of the label.

Thanks for the explanation, as soon as I read your solution I understood what it would do. I'm just not sure I would have been able to figure it out myself, at least not as quickly as you did. Thanks!

As I mentioned in my tweet to you guys this was my first pull request, not only for Joomla, but ever. With that said, how do I handle making these changes? Do I create a separate pull request for /tmpl/default.php and reference this pull request? What do I do about updating the XML file in this pull request?

avatar jrseliga
jrseliga - comment - 20 Dec 2014

From what I can tell, I just need to make the appropriate changes and push them to the "multiple-tags-filter" branch and this pull request will be automatically updated?

avatar nikosdion
nikosdion - comment - 20 Dec 2014

I've spent eight years hacking around Joomla!, I'm sure it usually takes me less than most developers to figure out where to look for a solution :D That said, I'm not a magician. I'll tell you my secret sauce: start from the symptom and read the code upwards to the root cause. Since this is your first PR ever, please let me tell you how I figured out the solution.

When I saw the problem with the label I first looked at the XML file. It uses the "tag" field. I knew that a field of type "X" corresponds to the class "JFormFieldX" so I took a look at JFormFieldTag. I followed it up the class tree but there was nothing there for me to see. Hm...

Let's take a look at the page source. The field is hidden and replaced with a chzn-something one and the select field now has a class that's nowhere else to be found, "chzn-done". Where does that come from? (The rest of this step I actually skipped because I already knew where it would lead me – I had been here before!) Searching the Joomla! source for this string shows me that it's added by chosen.jquery.js and chosen.jquery.min.js. All right, let's look for "chosen.jquery" in the Joomla! source code. Oh, right, it comes from JHtmlFormbehavior::chosen! Let's read the code.

This is where I saw where the label comes from. A-ha! I need to change that. Where in my view do I load this form behavior? I searched administrator/components/com_content/views/articles for "behavior.chosen". Why "behavior.chosen"? Because I knew that JHtmlFoo::Bar is always loaded with a call to JHtml::_('foo.bar'). You could have eventually figured it out yourself by following the code back to JHtml itself. Anyway. I found that this comes in the default.php file. So my question was how do I change the label for just this one instance? Looking back at the JHtmlFormbehavior::chosen code I saw that I needed a. a new CSS selector and b. add some stuff to the $options array. A bit of trial and error and here I am looking to the causal observer as a magician who just pulled a rabbit out of his tall hat.

Regarding how to change the code in your PR, that's easier than you think. Your PR is simply a request to merge the changes you made to the multiple-tags-filter branch on YOUR repository to Joomla!'s staging branch. All you need to do is make more changes to your multiple-tags-filter branch. GitHub will magically update your PR.

Hey, now you know how to pull rabbits out of your hat too!

avatar jrseliga
jrseliga - comment - 20 Dec 2014

Thanks for taking the time to explain your thought process, understanding how goes much further than just being told what needs changed.

In the scheme of things I know this is minor but I used class="advancedMultiple" instead of class="advancedTags" as this solution isn't specific to just tags. For example it would be applicable to a category field since it can be set to allow multiple selections also.

avatar nikosdion
nikosdion - comment - 20 Dec 2014

Actually, you need one formbehavior.chosen call for each custom label. That's why I named the class advancedTags: the label "- Choose tag -" only applies to tags, not any other field like Categories.

avatar jrseliga
jrseliga - comment - 20 Dec 2014

Right, over thinking on my part. Fixed.

avatar jrseliga
jrseliga - comment - 20 Dec 2014

I'm not sure how the process plays out for this to be officially merged, but I realized that this also needs to be added to the featured view as well so that tag filtering is consistent in all of com_content. I will make those changes and push them shortly.

avatar jrseliga
jrseliga - comment - 20 Dec 2014

Featured Articles multiple tags filtering can now be tested. Also updated original comment with detailed instructions and explanation.

avatar jrseliga jrseliga - change - 20 Dec 2014
Title
Support multiple tags filtering in Administrator ContentModelArticles
Article/Featured Article multiple tags filtering in Administrator
avatar brianteeman
brianteeman - comment - 20 Dec 2014

Awesome stuff - thanks for your contribution its a great one for your first
(and hopefully not the last)

On 20 December 2014 at 16:28, jrseliga notifications@github.com wrote:

Featured Articles multiple tags filtering can now be tested. Also updated
original comment with detailed instructions and explanation.


Reply to this email directly or view it on GitHub
#5474 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar jrseliga
jrseliga - comment - 20 Dec 2014

The front-end model doesn't even allow filtering on a single tag, let alone multiple, I'll build that into this pull request before it's officially merged, should be done sometime tomorrow.

avatar ggppdk
ggppdk - comment - 20 Dec 2014

Quoting:
The search for multiple tags will return articles which belong to any of the tags. Looking for all tags would have a very adverse performance and memory impact.

-- I struggled with this a year ago, trying to have acceptable performance but then i gave up, if you have any suggestion on how to make a query require -ALL selected - tags instead of - ANY selected - in large web-site (thousands of tags and 10000+ items) it would be great


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar brianteeman
brianteeman - comment - 20 Dec 2014

@test all good now


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar brianteeman brianteeman - test_item - 20 Dec 2014 - Tested successfully
avatar sovainfo
sovainfo - comment - 21 Dec 2014

Suggest to join on type_id instead of type_alias. Should allow for using the index.

@ggppdk :

SELECT title, count(tag_id) as tags 
FROM `j336_content` 
JOIN `j336_contentitem_tag_map` ON type_id = 1 AND id = content_item_id  
WHERE tag_id IN (3,4) 
GROUP BY title 
HAVING tags = 2

What does your EXPLAIN say?

avatar ggppdk
ggppdk - comment - 21 Dec 2014

@sovainfo
thanks for reply !!,
this is similar to the query i am using, also tried placing the COUNT in the HAVING clause

  • Using your query to require ALL tags
    I get on the DB table: _contentitem_tag_map
    --> Using where; Using index; Using temporary; Using filesort

  • Query to require ANY tags (GROUP BY is removed and DISTINCT is used instead):
    I get on the DB table: _contentitem_tag_map
    --> Using where; Using index; Using temporary;

The difference is that the --require ALL tags-- uses filesort

This pull request should be performance safe, because it does not use GROUP BY/COUNT/HAVING to require ALL -selected- Tags that has the above mentioned performance problem

avatar sovainfo
sovainfo - comment - 21 Dec 2014

But it is still using type_alias instead of type_id !

avatar jrseliga jrseliga - change - 21 Dec 2014
The description was changed
Title
Support multiple tags filtering in Administrator ContentModelArticles
Article/Featured Article multiple tags filtering in Administrator
avatar jrseliga
jrseliga - comment - 21 Dec 2014

@brianteeman @nikosdion Pushed tag(s) filtering to the front-end model.

avatar jrseliga jrseliga - change - 21 Dec 2014
The description was changed
avatar jrseliga jrseliga - change - 21 Dec 2014
Title
Article/Featured Article multiple tags filtering in Administrator
Article/Featured Article multiple tags filtering in Administrator/Site
avatar jrseliga
jrseliga - comment - 21 Dec 2014

@nikosdion I think I've come up with a way to allow the model to filter articles that contain all of the tags instead of filtering for articles that contain any of the tags in a way that adresses your performance concerns.

I'd really appreciate your opinion on how this would scale to heavy content site's (thousands of tags and articles).

When running this query be sure to modify the WHERE clause to include tag_id's that exist in your database.

SELECT
    c.id,
    tagmap.tags
FROM
    dev1_content c
INNER JOIN
(
    SELECT
        citgm.content_item_id,
        GROUP_CONCAT(citgm.tag_id ORDER BY citgm.tag_id) AS tags
    FROM
        dev1_contentitem_tag_map AS citgm
    GROUP BY citgm.content_item_id
) tagmap
ON c.id  = tagmap.content_item_id
WHERE
    tagmap.tags like ('%2%4%')

Since tags are held in array, they could just be imploded with '%' and that string inserted into the WHERE clause.

avatar nikosdion
nikosdion - comment - 21 Dec 2014

Nope. I will tell you the fatal performance mistakes I can see in this query:

You are concatenating strings. This means going through the entire tags relations tables, loading it into memory and creating the concatenated column. Slow and to much memory usage.
You are then doing a group by which means that this ginormous computed table has to be read and filtered
Then you are inner joining it with the content table. This creates a new temporary table with a size of NxM where N is the number of rows of articles and M the number of rows in the double filtered table
Finally, you are using a LIKE query against a computed column. Not only there’s not a cat’s chance in hell of using an index, you are not even diong a string comparison: you are doing a computationally expensive LIKE search.

This query has every single problem which destroys performance and kills the db server on sites with more than a couple of thousand articles and a few dozen tags. I am afraid that its performance is far worse than what I had in mind :(

PS: Writing efficient database queries is not easy. And no, I won’t pretend it’s easy for me either. I’ve published at least one new major version of software because I rewrote the bulk of its db queries to make them more performant. It “only" took me a few weeks and more trial and error I am comfortable to admit in public...=

avatar jrseliga
jrseliga - comment - 22 Dec 2014

@nikosdion Might have found an elegant solution. Here is my thought process, then I'll outline a quick test case.

Right now filtering only on multiple tags as implemented in this Pull Request (any article that have any of the tags) requires joining on js_contentitem_tag_map, which inherently returns an article row for N tags it matches. Why not just take a count of each article? Only include rows where the count is equal to the number of tags (PHP count of the tag_ids array printed into the $query string).

As it currently exists in this Pull Request, the query for filtering any article that has any tag looks like this:

SELECT DISTINCT a.id, a.title, a.alias, a.checked_out, a.checked_out_time, a.catid, a.state, a.access, a.created, a.created_by, a.created_by_alias, a.ordering, a.featured, a.language, a.hits, a.publish_up, a.publish_down,l.title AS language_title,uc.name AS editor,ag.title AS access_level,c.title AS category_title,ua.name AS author_name

  FROM dev1_content AS a

  LEFT JOIN `dev1_languages` AS l
  ON l.lang_code = a.language

  LEFT JOIN dev1_users AS uc
  ON uc.id=a.checked_out

  LEFT JOIN dev1_viewlevels AS ag
  ON ag.id = a.access

  LEFT JOIN dev1_categories AS c
  ON c.id = a.catid

  LEFT JOIN dev1_users AS ua
  ON ua.id = a.created_by

  LEFT JOIN `dev1_contentitem_tag_map` AS `tagmap`
  ON `tagmap`.`content_item_id` = `a`.`id`
  AND `tagmap`.`type_alias` = 'com_content.article'

  WHERE (a.state = 0 OR a.state = 1)
  AND `tagmap`.`tag_id` IN (8,9)
  ORDER BY a.id DESC

Adding a GROUP BY and HAVING clause will show only the articles that have all of the tags in the filter.

SELECT DISTINCT a.id, a.title, a.alias, a.checked_out, a.checked_out_time, a.catid, a.state, a.access, a.created, a.created_by, a.created_by_alias, a.ordering, a.featured, a.language, a.hits, a.publish_up, a.publish_down,l.title AS language_title,uc.name AS editor,ag.title AS access_level,c.title AS category_title,ua.name AS author_name

  FROM dev1_content AS a

  LEFT JOIN `dev1_languages` AS l
  ON l.lang_code = a.language

  LEFT JOIN dev1_users AS uc
  ON uc.id=a.checked_out

  LEFT JOIN dev1_viewlevels AS ag
  ON ag.id = a.access

  LEFT JOIN dev1_categories AS c
  ON c.id = a.catid

  LEFT JOIN dev1_users AS ua
  ON ua.id = a.created_by

  LEFT JOIN `dev1_contentitem_tag_map` AS `tagmap`
  ON `tagmap`.`content_item_id` = `a`.`id`
  AND `tagmap`.`type_alias` = 'com_content.article'

  WHERE (a.state = 0 OR a.state = 1)
  AND `tagmap`.`tag_id` IN (8,9)
    GROUP BY a.id
    HAVING COUNT(tagmap.content_item_id) = 2
  ORDER BY a.id DESC

Test Case

  1. Create Articles
    • New Article 1 - New Tag 1 && New Tag 2
    • New Article 2 - New Tag 1
  2. Tags Component, note the ID of the two new tags just created
  3. Run the first SQL statement with those two tag_ids edit `tagmap`.`tag_id` IN (A,B)
    • Returns both articles
  4. Run the second SQL statement again using those two tag_ids (HAVING clause doesn't need to change unless you use more than two tags)
    • Returns only "New Article 1"

Benefits (in comparison to my first attempt)

  • Everything is already loaded into memory?
  • No longer concatenating strings
  • No additional joins
  • Not using a computational LIKE search
avatar ggppdk
ggppdk - comment - 22 Dec 2014

But that is what we said above, this query (GROUP BY/COUNT/HAVING) has very bad performance with a couple of thousand articles and a few dozens of tags, since it scales terribly, it uses:

Using where; Using index; Using temporary; Using filesort;

i think a workaround is to ADD:
ORDER BY NULL
to the query

Tested and it indeed removes the: Using filesort;

.... But also using
ORDER BY a.id DESC
removes the filesort !!
so your query seem good !!

avatar nikosdion
nikosdion - comment - 22 Dec 2014

Actually, this entire query as it already is in Joomla! is a counter-example of writing performance optimized queries. Optimizing it is above my pay grade: I am a PHP developer, not a DBA. But regarding the improvement we are discussing, the HAVING trick might be the lesser of all evils if we want to filter by multiple tags. Caveat: it should only be added when you have multiple tags. Now, would it be really bad for performance? Maybe.

Ideally we’d want a large dataset with tags to test this. I have written com_overload to create thousands of articles, I’ve not updated it with tags. So I can’t really test what would happen with tens of thousands of articles and a few hundred tags. This is a major pitfall. This query which appears to be just a little bit slower might be much worse than we can imagine in a large dataset because of that “Using temporary”. Just how many rows will that temporary table hold make all the difference. That’s why I am not sure if it has a major performance impact or not.=

avatar ggppdk
ggppdk - comment - 22 Dec 2014

Sorry my mistake, i am doing other things here (like everybody), i posted wrong feedback above,

Using
ORDER BY NULL
removes "Using filesort"

but
ORDER BY a.id DESC
does not remove "Using filesort"

avatar nikosdion
nikosdion - comment - 22 Dec 2014

You are looking at the wrong thing to remove ;) Using file sort merely tells you that MySQL is sorting on the primary key. In fact, since you are JOINing tables you will always see “Using temporary; Using filesort”. What we want to optimize is the size of the temporary table because that’s where your bottleneck is. In an ideal world your list would only be ID-sorted and non-searchable which would allow you to do two fast queries: first get the LIMIT x,y rows of all articles having all tags (query on one table, no JOINs, gives ) as your first WHERE clause. That’s super fast because the first table in that query (__content) is filtered on an index, then these rows are JOINed with the larger table resulting in a super small temporary table. In real life, we want the articles list to be sorted by any column and we want to apply a heck of a lot of filters. That’s why we’ve ended up with this monstrous, slow query.

For more context and background information please read http://www.percona.com/blog/2009/03/05/what-does-using-filesort-mean-in-mysql/ http://www.percona.com/blog/2009/03/05/what-does-using-filesort-mean-in-mysql/ and http://s.petrunia.net/blog/?p=24 http://s.petrunia.net/blog/?p=24 =

avatar jrseliga
jrseliga - comment - 22 Dec 2014

I don't want to get to far down the rabbit hole and lose sight of the original goal. I submitted this Pull Request with the intention that it would filter any article that contained any of the tags. Larger picture being that this is just one in a series of at least 3 Pull Requests that should add to the filtering options currently available.

To me, when you see them all together the only logical way to approach the dilemma of matching on any vs. matching on every is to filter any article that matches on any tag. I am strictly speaking from the user experience perspective, firstly because you'd then have to add an additional field asking which type of tag filtering. More importantly as you can see from this screenshot, my series of changes would allow you to select multiple categories, which brings me to the second point. An article can only exist in one category, thus the only way to bring this functionality into the search is to filter any article in any of the categories. Same is true for author, as there can only be one author per article.

multiple-search-filters-default

Thus the general foundation of the current articles model and filter fields kind of forces us to treat tags with the "any article with any tag" approach, which is what the current commits already do.

Tags were introduce to Joomla! as a means to connect items, and in part, to address the fact that articles can't be assigned to more than one category, a feature which was rather highly requested.

This series of changes would allow a user to filter a list of articles in a fashion similar to:
"Give me all of the articles from Science, Lifestyle, and Health, written by Joe Smith or Jane Doe, tagged as heart or diet "

Something that is currently not possible with the current model.

At this point we've detailed the performance concerns, but that really stems from a larger issue with the entire query. Of which re-engineering is more suited for someone of a more database driven background than I, and far outside the confines of this Pull Request. Should the model not support both "any article with any tag" and "any article with every tag"?

What I'm ultimately getting at is, shouldn't all potential functionality be built into the model, but only make certain parts of it "enabled" until a more performant means of gathering these advanced datasets is built?

My suggestion, allow the model to do both types of tag filtering. However, core functionality (back-end article filtering and the Articles Newsflash module) will only allow the more efficient "any article that has any of the tags".

@brianteeman & @nikosdion Thoughts?

avatar jrseliga jrseliga - change - 23 Dec 2014
Title
Article/Featured Article multiple tags filtering in Administrator
Article/Featured Article multiple tags filtering in Administrator/Site
avatar nikosdion
nikosdion - comment - 23 Dec 2014

I find the “has any of the selected tags” approach to be the one most reasonable. Searching for all selected tags can be useful in some very specific and uncommon use cases but the amount of energy we have to expend on properly designing it IMHO outweighs the potential benefits. So, my +1 vote on your patch remains. I consider it a correctly thought out and implemented feature.=

avatar brianteeman
brianteeman - comment - 23 Dec 2014

If it is to be "any of the selected tags" is it possible for there to be a
tooltip such as "Please select one or mag tags"

This would be then be consistent with the language and behaviour on other
multiple selects used in Joomla

On 23 December 2014 at 06:47, Nicholas K. Dionysopoulos <
notifications@github.com> wrote:

I find the “has any of the selected tags” approach to be the one most
reasonable. Searching for all selected tags can be useful in some very
specific and uncommon use cases but the amount of energy we have to expend
on properly designing it IMHO outweighs the potential benefits. So, my +1
vote on your patch remains. I consider it a correctly thought out and
implemented feature.=


Reply to this email directly or view it on GitHub
#5474 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar jrseliga
jrseliga - comment - 23 Dec 2014

@brianteeman I'm not opposed to that, but the only fields I can find that have tooltips are the ones with the label separate from the input.
field-with-tooltip

Search Tools

None of the other filter fields have a tooltip, as their labeled directly inside the field. (Search Tools and Clear button do).

search-tools-multiple-tags

Do you feel it's insufficient for this field to have the label "- Select Tag -" as it currently is?

Articles Newsflash

The field I added to this module already has a label separate from the input field and had a tooltip before I made any modifications.

avatar brianteeman
brianteeman - comment - 23 Dec 2014

OK fair enough ;)

On 23 December 2014 at 08:29, Justin Seliga notifications@github.com
wrote:

@brianteeman https://github.com/brianteeman I'm not opposed to that,
but the only fields I can find that have tooltips are the ones with the
label separate from the input.
[image: field-with-tooltip]
https://cloud.githubusercontent.com/assets/3277067/5535788/ad1bed76-8a53-11e4-9644-af8f744c6aa1.png
Search Tools

None of the other filter fields have a tooltip, as their labeled directly
inside the field. (Search Tools and Clear button do).

[image: search-tools-multiple-tags]
https://cloud.githubusercontent.com/assets/3277067/5535771/694c46fe-8a53-11e4-914f-a63e297556fe.png

Do you feel it's insufficient for this field to have the label "- Select
Tag -" as it currently is?
Articles Newsflash

The field I added to this module already has a label separate from the
input field and had a tooltip before I made any modifications.


Reply to this email directly or view it on GitHub
#5474 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar jrseliga
jrseliga - comment - 23 Dec 2014

Your worry about consistent behavior is still valid though. These components all have "Search Tools" that include tag filtering:

  • User Notes Categories
  • Category Manager: Articles
  • Category Manager: Banners
  • News Feed Manager: News Feeds
  • Category Manager: Newsfeeds
  • Web Links Manager: Web Links
  • Category Manager: Weblinks

The - Select Tag - field should probably behave exactly the same across all. I'll take care of that and update with test instructions.

I see that this pull request contains merge conflicts, is that something I need to resolve or is that handled by whoever merges these changes (if my changes were to be approved)?

avatar infograf768
infograf768 - comment - 23 Dec 2014

@nikosdion
please skype me.

avatar jrseliga
jrseliga - comment - 23 Dec 2014

Can now filter categories by multiple tags.

Handles:

  • User Notes Categories
  • Category Manager: Articles
  • Category Manager: Banners
  • Category Manager: Newsfeeds
  • Category Manager: Weblinks

@test same instruction set as outlined for Administrator Article filtering on tags

avatar jrseliga
jrseliga - comment - 24 Dec 2014

@nikosdion I've hit a bit of a wall, maybe you can point me in the right direction. com_weblinks and com_newsfeeds don't build filtering fields with XML like the components that have the "Search Tools".

Those two render their filter fields with JHtml::_('x.y'), the tags field specifically being JHTML::_('select.options')

My thinking is I need to use:

  • JHtml::_('behavior.multiselect');
  • JHtml::_('formbehavior.chosen', 'select');
  • or a combination of both.

I can't figure out how I could do so in /administrator/components/com_newsfeeds/view/newsfeeds/view.html.php where JHtml::addFilter() creates the tag field.

avatar nikosdion
nikosdion - comment - 27 Dec 2014

Hello Justin,

TL;DR version: do not bother with these two components at all.

The long explanation (go grab some popcorn now)

First of all, com_weblinks and com_newsfeeds are going to be removed from the main Joomla! distribution. Instead, they are going to be distributed as standalone components which can be installed through the “Install from Web” feature of Joomla!. They will follow their own development and release schedule.

The second problem is that they don’t use Search Tools but the bloody contraption called “Sidebar”. I hate it vehemently because it’s too inflexible. And let me explain this.

In NewsfeedsViewNewsfeeds line 168 we have:
JHtmlSidebar::addFilter(
JText::('JOPTION_SELECT_TAG'),
'filter_tag',
JHtml::
('select.options', JHtml::_('tag.options', true, true), 'value', 'text', $this->state->get('filter.tag'))
);
If you follow the code, the static addFilter method merely adds a list of options to the protected static $filters array of the JHtmlSidebar class. How is this thing rendered? By JHtmlSidebar::render, which uses the JLayouts API, namely the joomla.sidebars.submenu layout file (/layouts/joomla/sidebars/submenu.php). In lines 63 to 76 of that file we can see that the rendering is absolutely inflexible: it can only render a simple drop-down field. There is no way to render a multiselect.

Therefore, you can’t do what you want to do unless you rewrite the back-end of these two components to use the Search Tools code. But you can’t do that without making major changes to these components. These changes were never made because these two components were “marked for termination”. Ergo: do not bother with these two components at all.=

avatar jrseliga
jrseliga - comment - 19 Jan 2015

@brianteeman @nikosdion what steps do I need to complete for this to be approved?

avatar nikosdion
nikosdion - comment - 19 Jan 2015

For starters, this PR has merge conflicts. You need to resolve them, i.e. rebase your PR against the staging branch and fix all merge conflicts.

Afterwards, you need two successful tests. Make a call for help on social media. Usually 2-4 of your followers will have the time to test the PR.

Afterwards, a member of the Joomla! PLT (Production Leadership Team) or an assigned "open source gardener" (like Brian) will flag this as RTC (Ready To Commit) if it can be merged a.s.a.p. or "Needs review" if the PLT needs to discuss about it. I am not a member of the PLT so I do what you'll have to do: wait patiently and hope my PR makes it into the core :)

Speaking of which, I think the beta window closed last night or thereabouts. This means that our PRs which weren't already RTC at the time will have to wait. If they are major changes (like yours) or new features (like mine) will have to wait for Joomla! 3.5, hopefully in 5-7 months. I know that's a bit of a let down but I think you'd agree that Joomla! stability is above making individual developers happy – and I say that having been on the "oh, shoot, I missed the beta window again" train too many times to count...

avatar jrseliga jrseliga - change - 5 Mar 2015
Title
Article/Featured Article multiple tags filtering in Administrator/Site
Filter on multiple Access Levels, Authors, Categories, and Tags
avatar wilsonge wilsonge - change - 11 Mar 2015
Milestone Added:
avatar wilsonge wilsonge - change - 11 Mar 2015
Labels Added: ?
avatar wilsonge wilsonge - change - 11 Mar 2015
Labels Added: ?
avatar paternax
paternax - comment - 14 Mar 2015

Several combinations of selections were tested successfully. But deleting the filter entries in the search tool doesn't reset the filtering in the articles list. It has to use the clear button to reset the list.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar paternax paternax - test_item - 14 Mar 2015 - Tested unsuccessfully
avatar knoby2015 knoby2015 - test_item - 14 Mar 2015 - Tested successfully
avatar knoby2015
knoby2015 - comment - 14 Mar 2015

it´s ok but when i set all in normal, there will not switch to all articles. the filter is ok only the set of zeoro is not possible


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar FrankyDE
FrankyDE - comment - 14 Mar 2015

bug found :-(
the selection of the tags works great, but it does not "reload" when you deselect the tags
Example: Tag: Red: correct articles are displayed.

5474 - select_tagscorrect

When you delete the Tag Selection the article view is not refreshed ... normally then all articles should be displayed again.

5474 - select_tagscorrectnorefresch

Sorry, as I like the idea very much and I find it cool and pretty much straight forward.

By the way I do not know what the expected behavior would be, I expected:
tag1 AND tag2 AND tag3
but the selection appears to be
tag1 OR tag2 OR tag3

Any opinions on that?

avatar brianteeman
brianteeman - comment - 14 Mar 2015

@FrankyDE that is something that @phproberto and myself have been discussing - the ability to choose if you want it to be AND or OR


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar rmittl
rmittl - comment - 14 Mar 2015

The selecting tags works great and also the resfreshing. The refreshing doesn't works, if I deselect all tags.

I expected the the OR between the tags.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.
avatar rmittl rmittl - test_item - 14 Mar 2015 - Tested unsuccessfully
avatar jrseliga
jrseliga - comment - 14 Mar 2015

@FrankyDE there was a bit of discussion above about giving the choice between AND or OR. @nikosdion had some reservations on the idea due to how inefficient the query already is, and the implications it might have for large sites.

avatar jrseliga
jrseliga - comment - 14 Mar 2015

@all Perhaps I'm not understanding the problem when resetting a filter group. I made a simple demonstration of how it functions.

avatar ggppdk
ggppdk - comment - 29 Mar 2015

Some people above stated that
-- "deselecting all values"
from a multi-value filter does not "refresh" / list all articles

the reason is because a select-multiple when having no values behaves like a checkbox-set, aka -NO VALUE- is submited (?? by most / all browsers ??), so the filter maintains its session state

The solution is to detect that form was submited:
(*example code)

Client side
<input type="hidden" name="form_submited" value="1"/>

Server side:
$form_submited = JRequest::getVar('form_submited');
$FILTERNAME = $form_submited ?
JRequest::getVar('FILTERNAME') :
$app->getUserStateFromRequest( $option.'.'.$view.'.FILTERNAME', 'FILTERNAME', false, 'array' );
if ( $form_submited ) $app->setUserState( $option.'.'.$view.'.FILTERNAME', $FILTERNAME);

Please use above JInput instead of JRequest, etc, adapt to the actual code, as the above is just an example

-- the above solution is simple, if form was submitted then do not use session for getting the filter value, instead only use the posted data, and then set session data

avatar ggppdk
ggppdk - comment - 30 Mar 2015

Hello

i will not agree or disagree with this, possibly your suggestion is best

but mostly what i did is

  1. explain --why/what-- is happening, identifying the problem

  2. used a common (indepedent to joomla) suggested approach, of detecting if form was submitted and not using the session

-- i don't say it is a best choise, but i don't think it is not as harmful as you imply

about adding an extra option to the multiselect, i am not sure if this best, since

  • browsers
  • and chosen/select2 JS libs treat it differently than how single select "empty" value is treated

i think it will cause more work and also can you tell how to make this "none" value invisible without hacking choosen JS ? e.g. via CSS ,

or we will need some extra JS to do this
or does Joomla / choosen already have this feature ??

avatar jrseliga
jrseliga - comment - 30 Mar 2015

I've tested this across Firefox (Windows/Linux), Chrome (Windows/Linux), and IE 10 and am not experiencing any issues with deselecting. Perhaps this is a browser issue? IE 9 or older?

avatar nikosdion
nikosdion - comment - 30 Mar 2015

Please ignore my replies (I removed them now). A mail filter fail made me think they were about a completely different project. Receiving too much email can do that... Sorry!

avatar jrseliga
jrseliga - comment - 4 Jun 2015

Circling back to this, does anyone have more information on the environment their experiencing the issue with? Again I made a demonstration for more clarity on how it should be working.

avatar ggppdk
ggppdk - comment - 8 Jun 2015

Indeed,
you clearly show it to work, sorry i don't have time to test,
can someone else do it ?

avatar Bakual
Bakual - comment - 9 Jun 2015

This will also need some tests on PostgreSQL and MSSQL databases to make sure nothing breaks there.

avatar ggppdk
ggppdk - comment - 19 Jul 2015

I have tested firefox/chrome/ie11, seems to works properly, when clearing via clear button and by removing all selected options too.

Also article manager URLs seem to work properly

...&filter[access]=1

...&filter[access][]=1&filter[access][]=2

...&filter[somefiltername]=aaaa
'access' filter and other filters are cleared
thus #6916, which links from catergory manager towards the article manager seem to work properly

avatar zero-24 zero-24 - change - 26 Aug 2015
Status Pending Needs Review
avatar wilsonge wilsonge - change - 6 Nov 2015
Milestone Removed:
avatar wilsonge wilsonge - change - 7 May 2016
Status Needs Review Pending
avatar roland-d
roland-d - comment - 8 May 2016

@alikon can you take a look if this is OK for postgres and mssql? Thank you.

@jrseliga Can you please fix the merge conflicts so we can get it ready for 3.6? Thank you.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.

avatar brianteeman
brianteeman - comment - 5 Aug 2016

@jsreliga will you be updating this to fix the merge conflicts. If we dont get a reply then this will be closed in a few weeks.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5474.

avatar brianteeman brianteeman - change - 5 Aug 2016
Status Pending Information Required
avatar ggppdk
ggppdk - comment - 5 Aug 2016

@jrseliga

Circling back to this, does anyone have more information on the environment their experiencing the issue with? Again I made a demonstration for more clarity on how it should be working.

There were some bogus PHP versions (with suhosin ?),
that were not submiting select multiple as empty array, instead the array did not exist at all in the posted data

avatar roland-d
roland-d - comment - 5 Aug 2016

I have someone working on this, will know next Friday if it is finished or not.

avatar joomla-cms-bot joomla-cms-bot - change - 8 Aug 2016
Category Administration Tags Administration Components Language & Strings Front End Modules
avatar brianteeman
brianteeman - comment - 8 Aug 2016

Closed as we have a NEW pr #11517

avatar brianteeman brianteeman - close - 8 Aug 2016
avatar brianteeman brianteeman - change - 8 Aug 2016
Status Information Required Closed
Closed_Date 0000-00-00 00:00:00 2016-08-08 13:37:16
Closed_By brianteeman
avatar joomla-cms-bot joomla-cms-bot - change - 8 Aug 2016
Labels Added: ?

Add a Comment

Login with GitHub to post a comment