No Code Attached Yet bug
avatar kwellesleyedu
kwellesleyedu
1 Jul 2022

Steps to reproduce the issue

  1. Install Joomla 4.1.5 with all default settings.
  2. Create a category titled “Parent Category” with no parent.
  3. Create a child category titled “Child Category” with the parent set to “Parent Category”.
  4. Create a blank article titled “Article A” with the category set to “Parent Category”.
  5. Create a blank article titled “Article B” with the category set to “Child Category”.
  6. Navigate to the “Smart Search” component and create a new search filter with only both “Parent Category” and “Child Category” selected. Give it a title of “test filter”. The map count should show as “2”.
  7. Create a menu item with a type of “Smart Search” -> “Search”. Set it to the “Main Menu”, title it “Search”, and set the menu item to use a “search filter” of “test filter” (created in step 6).
  8. Now, on the front end, navigate to the search page created in the previous step. The url should be “index.php/search?f=1”. Notice the “f=1” which ensures we are using the “test filter” created in step 6.
  9. Search for “article”. The url should be “index.php/search?f=1&q=article”.

Expected result

Search page should show “Article A” and “Article B” since both contain the word “article” and both of their categories are selected in the search filter.

Actual result

Search page shows “No Results Found”.

System information (as much as possible)

Joomla 4.1.5
PHP 7.4

Additional comments

  1. If the search filter only contains root level categories and no child level categories, the filter works without issue. Following the example above, change "test filter" to only have "Parent Category" selected.
  2. If the search filter contains only a single child level category, the filter works without issue. Following the example above, change "test filter" to only have "Child Category" selected.
  3. If the search filter contains multiple child level categories, the filter does NOT work and no results show up regardless of what you search for.
  4. If the search filter contains a mix of root level categories and child level categories, the filter does NOT work and no results show up regardless of what you search for. (This is the example above)

It appears the issue may be due to either one or both of the following files:

/components/com_finder/src/Model/SearchModel.php

Potentially within populateState() and getListQuery() of SearchModel.php

/administrator/components/com_finder/src/Indexer/Query.php

Potentially within processStaticTaxonomy() for Query.php

The design of the taxonomy database table may be playing a role as well.

I imagine the majority of joomla sites have a decent mix of root level parent categories and child categories. As it is now, the search filters are completely broken for those that use them with child categories. I think most would appreciate it very much if this could be fixed as soon as possible.

Votes

# of Users Experiencing Issue
5/5
Average Importance Score
4.40

avatar kwellesleyedu kwellesleyedu - open - 1 Jul 2022
avatar joomla-cms-bot joomla-cms-bot - change - 1 Jul 2022
Labels Added: No Code Attached Yet
avatar joomla-cms-bot joomla-cms-bot - labeled - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 1 Jul 2022
The description was changed
avatar kwellesleyedu kwellesleyedu - edited - 1 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 11 Jul 2022
Title
[4.1] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
[4.1.5] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
avatar kwellesleyedu kwellesleyedu - edited - 11 Jul 2022
avatar kwellesleyedu kwellesleyedu - change - 11 Jul 2022
Title
[4.1.5] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
[BUG][4.1.5] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
avatar kwellesleyedu kwellesleyedu - edited - 11 Jul 2022
avatar pixelhexe
pixelhexe - comment - 18 Jul 2022

I have to confirm this as I just stumbled upon it.

The bug seems indeed to be in the file

/components/com_finder/src/Model/SearchModel.php

in the getListQuery() - more concrete the description says "We have to join the taxonomy map table for each group so that we can use AND clauses across groups. Within each group there can be an array of values that will use OR clauses."

But in the end there are only AND clauses in the HAVING SUM .... clauses of the query.

For my purposes (!!!) I could fix it by adding ", $glue = 'OR'" as follows in the HAVING clause:

// Iterate through each taxonomy group.
for ($i = 0, $c = count($groups); $i < $c; $i++)
{			
	$query->having('SUM(CASE WHEN t.node_id IN (' . implode(',', $groups[$i]) . ') THEN 1 ELSE 0 END) > 0');				
}

my bugfix is:

// Iterate through each taxonomy group.
for ($i = 0, $c = count($groups); $i < $c; $i++)
{			
	$query->having('SUM(CASE WHEN t.node_id IN (' . implode(',', $groups[$i]) . ') THEN 1 ELSE 0 END) > 0' , $glue = 'OR');				
}

My purpose is using a search filter to find articles inside several parent and child categories. With my bugfixing I can use this filter, without the fix there are no results as described from kwellesleyedu.

Maybe this would not be a general fix so somebody from the core developers should dive deeper. Thank you!

avatar kwellesleyedu
kwellesleyedu - comment - 18 Jul 2022

Indeed, the section of code you've described within getListQuery() inside /components/com_finder/src/Model/SearchModel.php is one potential location where a fix could be inserted.

We can dive deeper by looking at what exactly that section of code is iterating through. Namely, it is $this->searchquery->filters. Following the simple example in the first post of this issue thread, if we do some debugging and output this variable, we see the following:

array(2) {
    ["Category"]=> array(1) {
        ["Parent Category"]=> int(10)
    }
    ["Parent Category"]=> array(1) {
        ["Child Category"]=> int(11)
    }
}

As the description of that section of code goes (which you have also mentioned):

"We have to join the taxonomy map table for each group so that we can use AND clauses across groups. Within each group there can be an array of values that will use OR clauses."

Knowing this, clearly something is wrong. Due to the way the array is structured, the code interprets "Child Category" to be in its own group and uses an AND clause here which means the code is trying to find items within both "Parent Category" and "Child Category", which is impossible since an item can only be in a single category, hence the lack of results that show up.

So a fix could manifest itself within the code you mentioned to better interpret this array, but I'm not sure that that would be a robust solution. I think a better one may be to adjust how the array is structured so that the existing code works. (for example, in this very specific example, "Child Category" should be grouped under "Category" along with "Parent Category")

Within SearchModel.php, we then see that $this->searchquery->filters is first set within the populateState() function.

If we continue diving deeper, we then see that populateState() is setting $this->searchquery by using /administrator/components/com_finder/src/Indexer/Query.php. Specifically, the constructor of Query.php is using the processStaticTaxonomy() function to set $this->filters. In other words, $this->searchquery->filters used by SearchModel.php seems to ultimately be set by processStaticTaxonomy() within Query.php.

The way that the function processStaticTaxonomy() works is intertwined with the design of the taxonomy table in the database.

So there are many steps within this process where code could be adjusted to produce a fix, but I'm not sure how deep the Joomla team wants to go with changes. Presumably, the change should be a general fix that works in all scenarios.

(Although I'm somewhat able to decipher what is going on here, I'm not really familiar with the Joomla codebase and how everything is interconnected, so I'm not comfortable proposing an explicit list of changes.)

Anyway, can someone add the "bug" label (and any other relevant ones) to this issue? I see it hasn't been added on yet.

avatar Quy Quy - change - 18 Jul 2022
Labels Added: bug
avatar Quy Quy - labeled - 18 Jul 2022
avatar crimle
crimle - comment - 20 Jul 2022

Same issue here. I tried the solution provided by pixelhexe and it works for me! I would really appreciate it if this bugfix is included in the core.

avatar pixelhexe
pixelhexe - comment - 20 Jul 2022

Thanks @crimle for your feedback - good to hear that it works for you! But be careful - my fix is not working in general.

For example it is working if you need to do a search for articles throughout several categories - parent or child or both. So I only select these categories in the filter - and nothing else, not "type", not "author". Otherwise the filter does not work any more (because it will find ALL articles if selected "article" as type e.g. due to the OR in the query).

So somebody has to do the fine grading of the Joomla programming work as @kwellesleyedu correctly said ...

avatar crimle
crimle - comment - 21 Jul 2022

In the filter I am using

Type = Categories and Articles
Author = none
Category = those related to the sport in question

For my needs it works and I am happy.

Kind regards
Chris

avatar pixelhexe
pixelhexe - comment - 22 Jul 2022

Unfortunatley I have to ask a follow up question to @crimle: did you try to search for an author's name and would you confirm that the finder does NOT provide articles created or modified by this author? As in your filter you also don't want to look for author's name, right?

Because on my site there seems to be a second bug in the finder. First I thougt it might be related with my bugfix above but it isn't.

Here is a five year old bugfix which still seems to work - at least for me:

https://joomla.stackexchange.com/questions/19506/smart-search-exclude-search-articles-by-author-name

Thank you once again.

avatar crimle
crimle - comment - 25 Jul 2022

Using Smart Search (without Filter)
Searching for author's name lists any article that has ever been created by this user. Which is completely pointless.

I opened «Content Maps» and disabled the author in question. This has no effect to the search results. Any article that has ever been created by this user are still listed.

You may also disable the whole parent item «Author». There is still no effect to the search results.

Using Smart Search (with a user defined filter)
In this filter I did NOT select the author.
Searching for author's name does NOT list articles that have been created by this user. This is working as expected.

Using Smart Search (with a user defined filter) after applying the bugfix you mentionned above
Searching for author's name lists any article that has ever been created by this user. Which is NOT working as expected.

So, sadly enough, as much as I appreciate your bugfix contribution, I must admit that it works partially only. But maybe this error would not exist if the basic issue with Smart Search (Content Maps > Disabled Authors) was solved.

Kind regards
Chris

avatar pixelhexe
pixelhexe - comment - 25 Jul 2022

Thank you, Chris, for coming back with this detailed testing!
So it just reaffirms "my fix is not working in general"...

For my special needs it is working with or without setting a filter (which selects some categories, nothing else), but only with adding the mentioned 2 bugfixes in core files. This is of course not the way it should be. :(

For those with German language knowledge - I found the author's name issue also described in the german forum

  • without any solution as well.

So let's hope that there will come a smart developer for fixing the smart search of Joomla 4.
Apart from these problems the smart search is SO much better than the old search extension... I really appreciate all the programming work for this.

avatar EJBJane
EJBJane - comment - 2 Aug 2022

Hi there,
Can I confirm that this Forum post is related to the same problem?
https://forum.joomla.org/viewtopic.php?f=808&t=995243
You think I can use the fix above too?
Thanks,
Liz


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38201.

avatar kwellesleyedu
kwellesleyedu - comment - 2 Aug 2022

Yes, it appears you are describing the same problem.

The code change brought up by @pixelhexe is not a general fix and only works in a very specific situation so you need to be extremely careful if you want to try it out.

I think a more involved design change may be needed for a fully robust solution. (as opposed to just changing some of the outer most code)

That said, I think we're all just waiting on the Joomla team to address this issue. No idea when they'll get around to this.

avatar EJBJane
EJBJane - comment - 2 Aug 2022

Thanks for your quick reply, I will keep following this post then.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38201.

avatar EJBJane
EJBJane - comment - 2 Aug 2022
avatar kwellesleyedu
kwellesleyedu - comment - 2 Aug 2022

Indeed, I imagine this issue to be affecting a lot of people. It's perfectly natural to organize categories into different levels and is something I presume most partake in to some degree.

avatar kwellesleyedu
kwellesleyedu - comment - 16 Nov 2022

Just following up on this. (3 months later now)

I've just tested out the latest Joomla 4.2.5 with PHP on 8.1 and the issue described here still exists. The filters feature of the smart search component is still very broken.

avatar kwellesleyedu kwellesleyedu - change - 16 Nov 2022
Title
[BUG][4.1.5] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
[BUG] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
avatar kwellesleyedu kwellesleyedu - edited - 16 Nov 2022
avatar kwellesleyedu kwellesleyedu - change - 16 Nov 2022
Title
[BUG] Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
[BUG] J4 Smart Search (com_finder) search filters broken when selecting multiple categories AND including a child level category
avatar kwellesleyedu kwellesleyedu - edited - 16 Nov 2022
avatar jonbq
jonbq - comment - 8 Dec 2022

Having the same issue. Can't believe this has been left for 6 months - anything larger than a small website means search filters are basically unusable. Nesting categories is so common.

Can someone from the Joomla team comment on this?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38201.

avatar Hackwar
Hackwar - comment - 10 Feb 2023

This here should fix that: #39751

Since we have a PR, I'm closing this one.

avatar Hackwar Hackwar - close - 10 Feb 2023
avatar Hackwar Hackwar - change - 10 Feb 2023
Status New Closed
Closed_Date 0000-00-00 00:00:00 2023-02-10 12:38:38
Closed_By Hackwar

Add a Comment

Login with GitHub to post a comment