? ? ? Pending

User tests: Successful: Unsuccessful:

avatar Hackwar
Hackwar
23 May 2018

This PR extends the work of #20391 and while it can be tested the way it is, #20391 should be merged first and later this here.

This PR adds a new option to the configuration of Finder/Smart Search to select the default stemmer to use for content items marked as "All" or monolingual sites. You can select "None", a specific language or the default language for the frontend, that is set in the language manager.

All the code for this feature is in FinderIndexerHelper::stem().

How to test

  1. Install 4.0 with sample data
  2. Run indexer in the backend and see that content set to "All" in the DB in #__finder_terms is not stemmed.
  3. Go to component configuration and set the default stemmer to your prefered setting.
  4. Clear Index and start Index again, noticing that content set to "All" is now stemmed, too.
avatar Hackwar Hackwar - open - 23 May 2018
avatar Hackwar Hackwar - change - 23 May 2018
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 23 May 2018
Category Administration com_finder Language & Strings Front End
avatar infograf768
infograf768 - comment - 24 May 2018

As repeated multiple times in #20391, we do not approve adding the stemmers in the language packs, which makes this PR here unacceptable.

avatar brianteeman
brianteeman - comment - 24 May 2018

Please change we to I - you do not speak for everyone

avatar infograf768
infograf768 - comment - 24 May 2018

I am not alone as you know well but let's have a joke: Never heard of "We the King" ?

avatar brianteeman
brianteeman - comment - 24 May 2018

I believe you mean the "Royal We" https://en.wikipedia.org/wiki/Royal_we

avatar infograf768
infograf768 - comment - 24 May 2018

Specially to this part though as I switched recently to a North Indian status :

in Hindustani and other North Indian languages, the majestic plural is a common way for elder speakers to refer to themselves when addressing those younger than them.

avatar Hackwar Hackwar - change - 12 Jun 2018
Labels Added: ? ?
avatar joomla-cms-bot joomla-cms-bot - change - 16 Jun 2018
Category Administration com_finder Language & Strings Front End Unit Tests Repository Administration com_finder Language & Strings Front End
avatar Hackwar Hackwar - change - 16 Jun 2018
Labels Added: ?
avatar Hackwar
Hackwar - comment - 16 Jun 2018

I've now extended this PR to set the right language for tokenisation. Still waiting for the stemmer PR to be merged. That would clean up this one here a lot...

avatar joomla-cms-bot joomla-cms-bot - change - 20 Jun 2018
Category Administration com_finder Language & Strings Front End Unit Tests Repository Administration com_finder Language & Strings Front End
avatar Hackwar Hackwar - change - 20 Jun 2018
Labels Removed: ?
avatar Hackwar
Hackwar - comment - 20 Jun 2018

Ok, so merging the stemmer PR in here did not clean up the number of commits... ?‍♂️ But it did clean up the change-delta, so it should be pretty obvious what I'm doing here. This is ready to test and to commit. ?

avatar carlitorweb carlitorweb - test_item - 20 Jun 2018 - Tested successfully
avatar carlitorweb
carlitorweb - comment - 20 Jun 2018

I have tested this item successfully on fb32ca0


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20562.

avatar infograf768
infograf768 - comment - 21 Jun 2018

COM_FINDER_CONFIG_LANGUAGE_DEFAULT_DEFAULT_LANGUAGE="Default installed language"

In fact it is "Default Site Language" if I do not mistake.

Also (unhappily), the description does not display (grrr)
String exists:
COM_FINDER_CONFIG_LANGUAGE_DEFAULT_DESC="Set the language to be used for non-multilingual sites or content marked as \"All\". (Depends on available languages.)"
but it is not added in the field and, if I add it, it does not display.
The label is quite insufficient to understand what this parameter is about...
(If solved, I would anyway take out (Depends on available languages.) as it is quite obvious.

In any case, the field type should NOT be contentlanguage as these can be deleted on a monolanguage site, but simply language.

avatar infograf768 infograf768 - test_item - 21 Jun 2018 - Tested unsuccessfully
avatar infograf768
infograf768 - comment - 21 Jun 2018

I have tested this item ? unsuccessfully on fb32ca0

Just because of the field type. Otherwise it works fine afaics


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20562.

avatar Hackwar
Hackwar - comment - 21 Jun 2018

Changed and fixed.

avatar infograf768 infograf768 - test_item - 21 Jun 2018 - Tested successfully
avatar infograf768
infograf768 - comment - 21 Jun 2018

I have tested this item successfully on fb32ca0

OK for me. People fighting against Tips in 4.0 should really think better.

Such a parameter does need explanations and not through a possible doc in a unique language. The description of the field should display here.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20562.

avatar mbabker
mbabker - comment - 21 Jun 2018

The fight isn’t against tips, full stop. The fight is against tips by
default on all the things and tips in an inaccessible manner. I don’t
think it’s too much to ask to let people figure out a solution to that
instead of retaining the status quo because it’s easy…

On Thu, Jun 21, 2018 at 8:17 AM infograf768 notifications@github.com
wrote:

I have tested this item successfully on fb32ca0
fb32ca0

OK for me. People fighting against Tips in 4.0 should really think better.

Such a parameter does need explanations and not through a possible doc in
a unique language. The description of the field should display here.

This comment was created with the J!Tracker Application
https://github.com/joomla/jissues at
issues.joomla.org/tracker/joomla-cms/20562.


You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
#20562 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAWfoW6JF9F4iPRd7WO0m2Qlpbw8PTVuks5t-5z-gaJpZM4ULTfZ
.

--

  • Michael Please pardon any errors, this message was sent from my iPhone.
avatar dgrammatiko
dgrammatiko - comment - 21 Jun 2018

@infograf768 wasn't the answer from @mbabker here: joomla-projects/gsoc17_helpscreens_on_jdocs#7 (comment) explanatory enough? Why are you keep crying over the freaking tooltips?

avatar infograf768
infograf768 - comment - 22 Jun 2018

Why are you keep crying over the freaking tooltips?

Cool, my friend... Maybe just because nothing is done about it for the moment (that I have seen) and you proposed Note as a solution...

avatar infograf768
infograf768 - comment - 22 Jun 2018

@Hackwar
I may have found an issue here and it is weird.
I have set the stemmer to use French on a monolanguage site. In my site is a phrase containing the word française. I can't get the suggestion. I checked that it is among the terms in db and it is.
If I set the stemmer to use English, I do get the suggestion OK, although the word is not highlighted in the result.

Stemmer set to english
screen shot 2018-06-22 at 10 31 17
result
screen shot 2018-06-22 at 10 37 39

Did not test this yet on a multingual site.

avatar carlitorweb
carlitorweb - comment - 22 Jun 2018

@infograf768 is not from this PR. As I comment to @Hackwar on #20384 (comment) i think is some wrong JS.

avatar Hackwar
Hackwar - comment - 22 Jun 2018

There is an issue with the stemmer setting. I'm going to fix that this evening. That it doesn't highlight the words is an issue with JS indeed. There are several issues with the JS and right now I would be happy if someone else could have a look and find solutions to those. My JS is not so strong...

avatar carlitorweb
carlitorweb - comment - 22 Jun 2018

Will be better open an issue, with all the JS problem you have found. In this way, others can have a look

avatar infograf768
infograf768 - comment - 22 Jun 2018

@carlitorweb
the highlighting as well as the loss of focus on the field are indeed js related, but not the fact that a suggestion is found with English stemmer and not French stemmer. :)

avatar carlitorweb
carlitorweb - comment - 22 Jun 2018

@infograf768 that is true, sorry.

avatar infograf768
infograf768 - comment - 26 Jun 2018

@Hackwar
No change versus #20562 (comment)

avatar Hackwar
Hackwar - comment - 26 Jun 2018

Ok, so I found a few issues in the code. It works right for me now. But please make sure that you re-index before testing.

avatar Hackwar
Hackwar - comment - 26 Jun 2018

What I mean is, that you have to recreate the index after you applied this patch.

avatar infograf768
infograf768 - comment - 27 Jun 2018

OK, this time it works better. I obtain suggestions. ?

Non ascii terms (forêt, française) are not highlighted in the results as we already know. Therefore I looked a bit further.

I dumped $this->query->highlight and this is what I get.
screen shot 2018-06-27 at 08 21 56

In this specific case, I solved the issue by modifying the highlighter method in /libraries/cms/html/behavior.php

I commented line 489 to 492

		//foreach ($terms as $i => $term)
		//{
		//	$terms[$i] = JFilterOutput::stringJSSafe($term);
		//}

screen shot 2018-06-27 at 08 43 00

Therefore it looks like using JFilterOutput::stringJSSafe on non ascii creates this issue.
This is only used in the highlighter method in core.
That method loads the legacy/highligter.js. Could it be that that js does not interprete UTF8 correctly?

This issue is ALSO present in 3.x for com_finder. I guess it was never solved because the component is not much used.
The problem is NOT present when using com_search on 3.x.

@dgrammatiko

avatar Hackwar
Hackwar - comment - 27 Jun 2018

Ok, so @infograf768 you have a successfull test, right? The JS issues are not related to this PR and they are a separate issue. (Which we indeed do have to fix for 4.0)

avatar infograf768 infograf768 - test_item - 27 Jun 2018 - Tested successfully
avatar infograf768
infograf768 - comment - 27 Jun 2018

I have tested this item successfully on fb32ca0


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20562.

avatar infograf768 infograf768 - alter_testresult - 27 Jun 2018 - carlitorweb: Not tested
avatar infograf768
infograf768 - comment - 27 Jun 2018

Needs another tester to set RTC.

avatar carlitorweb carlitorweb - test_item - 27 Jun 2018 - Tested successfully
avatar carlitorweb
carlitorweb - comment - 27 Jun 2018

I have tested this item successfully on fb32ca0


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20562.

avatar franz-wohlkoenig franz-wohlkoenig - change - 27 Jun 2018
Status Pending Ready to Commit
avatar joomla-cms-bot joomla-cms-bot - edited - 27 Jun 2018
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 27 Jun 2018

Ready to Commit after two successful tests.

avatar infograf768
infograf768 - comment - 29 Jun 2018

@Hackwar
Any way to show the name="language_default" field only when multilang is disabled?

avatar Hackwar
Hackwar - comment - 29 Jun 2018

That field is especially necessary for non-multilang sites. So... No.Am 29.06.2018 16:14 schrieb infograf768 notifications@github.com:@Hackwar
Any way to show the name="language_default" field only when multilang is disabled?

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

avatar infograf768
infograf768 - comment - 29 Jun 2018

That field is especially necessary for non-multilang sites.

Sure that is why I asked if we could hide it when multilang is enabled. Could not find a way.

avatar Hackwar
Hackwar - comment - 29 Jun 2018

You also need it for multilang for those texts that are marked *.Am 29.06.2018 16:34 schrieb infograf768 notifications@github.com:
That field is especially necessary for non-multilang sites.

Sure that is why I asked if we could hide it when multilang is enabled. Could not find a way.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

avatar infograf768
infograf768 - comment - 29 Jun 2018

You also need it for multilang for those texts that are marked *

Correct. Thanks.

avatar wilsonge wilsonge - change - 1 Jul 2018
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2018-07-01 13:54:02
Closed_By wilsonge
Labels Added: ?
avatar wilsonge wilsonge - close - 1 Jul 2018
avatar wilsonge wilsonge - merge - 1 Jul 2018
avatar wilsonge
wilsonge - comment - 1 Jul 2018

Looks good to me! Thanks!

Add a Comment

Login with GitHub to post a comment