[#13377] - Add new Terms view
- Closed
- 21 Apr 2019
- Medium
- Build: staging
- # 13377
- Diff
- chrisdavenport:ss-terms-view
- Success continuous-integration/drone the build was successful Details
- Success continuous-integration/travis-ci/pr The Travis CI build passed Details
- Failure JTracker/HumanTestResults Human Test Results: 0 Successful 2 Failed. Details
- Error continuous-integration/jenkins/pr-merge This commit cannot be built Details
User tests: Successful: Unsuccessful:
Summary of Changes
For some time now I have found it very helpful to have a Terms view in Smart Search, especially when trying to debug new Smart Search plugins. I had a bit of time to spare during the holiday break so I polished it up and I'm presenting it here for possible inclusion as a new feature in Joomla 3.7.
As well as adding the Terms view, this PR also adds a Terms column to the Indexed Content view which shows the number of terms indexed for each content item and is also a link to the Terms view filtered by that content item.
This PR adds a terms_count column to the #__finder_links table to keep track of the number of terms indexed for each content item. Because this causes an error in recent MySQL versions, I have also updated the default date for the datetime columns so the error isn't thrown.
I have added update files for Postgres and SQL Azure, but I have no idea if they are correct or not.
Testing Instructions
- Apply this PR.
- Go to Extensions -> Manage -> Database and click Fix.
- Go to Components -> Smart Search. Rebuild the index by clicking Clear Index, then Index. Alternatively, use php cli/finder_indexer.php --purge to rebuild the index from the command line.
- Notice that the Indexed Content view now has an extra Terms column which shows the number of terms indexed for each content item. The numbers are also clickable links that will take you to the Terms view.
- Take a look around the Terms view, clicking on stuff. Hopefully, it will work as expected.
Documentation Changes Required
If this PR is accepted, I'll add an appropriate help screen. The PR adds a help key of JHELP_COMPONENTS_FINDER_MANAGE_TERMS which for en-GB will attempt to load the help screen at https://docs.joomla.org/Help37:Components_Finder_Manage_Terms
| Status | New | ⇒ | Pending |
joomla-cms-bot
- | Category | ⇒ | SQL Administration com_admin Postgresql MS SQL com_finder Language & Strings |
I have tested this item
Done Test Instructions until click on "Index": got "An Error Has Occurred - Unknown column 'terms_count' in 'field list' SQL=UPDATE #__finder_links SET md5sum = 'b08e86fd430bdd09a730c3d35ff80162' , terms_count = 64 WHERE link_id = '1'
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13377.
@franz-wohlkoenig Did you "Go to Extensions -> Manage -> Database and click Fix."?
@chrisdavenport Yes, like your Instruction said. Tried 2 Times, same Result.
Only one Weblink is indexed, but the new "Terms"-Menue on left Side is filled.
I get an error message, when I was trying to build the index. I pressed "fix" the database before.
I have tested this item
Sorry my test end with an error, you can see here
getestet on #jc17de
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13377.
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13377.
Yeah, for some reason the SQL statements are not being executed by the database fixer. I don't know why.
Yeah, for some reason the SQL statements are not being executed by the database fixer. I don't know why.
example
ALTER TABLE `#__finder_links`
+ CHANGE `indexdate` `indexdate` DATETIME NOT NULL DEFAULT '1000-01-01 00:00:00',If i remember corrently,
the above is currently not possible to be checked by the database fixer,
thus, it cannot know if it has already been executed, so it is ignored
@ggppdk That would explain it. So what's the right way to add a column to a table?
Adding column is detectable by database fixer,
i did not mean the ADD column part of the SQL upgrade script
e.g. this will be detected by DB fixer
ALTER TABLE `#__finder_links` ADD COLUMN `terms_count` INT NOT NULL DEFAULT '0' AFTER `object`;what is not implemented / not detectable by DB fixer (again if i remember correctly):
is modifying column properties, (like e.g. changing default column value and setting NOT NULL flag)
Maybe it is acceptable to just run the SQL update script on upgrade,
since this is not first time that exact case has been encountered,
If you search the SQL upgrade files you will find similar cases that will not work with manual update files and then using the DB fixer
@chrisdavenport Any interest in taking this forward still? It's been untouched in nearly 6 months.
my comment above was a little misleading i meant the column attributes
- NULL
- DEFAULT
Can not be detected by schema checker,
... only if you also add a different TYPE, then they will run (because TYPE is checked thus it will trigger the query to run)
About your column problem with adding new column
just use: ADD COLUMN
instead of: ADD
- the schema checker only supports 'ADD COLUMN' , it does not support 'ADD'
Also instead of multiple 'CHANGE' inside same ALTER, this is also not supported by schema checker,
- you need to use 1 'ALTER TABLE' per 'CHANGE'
- and a PR to add support for detecting modifications in NULL and DEFAULT column attributes,
i can make this last one for Mysql (/Maria DB) and someone can help to convert it to other DBs
Can we still get this implemented? I would prefer to add this to Joomla 4.0 though.
Should still work. Just needs the database update issue fixed.
Should still work. Just needs the database update issue fixed.
My PR for checking NULL and DEFAULT has been merged
@chrisdavenport Are you going to update this PR or should I? I would create the new PR for 4.0-dev though.
@chrisdavenport, thank you for your work on this. I've taken this PR and adapted it to 4.0-dev, but I run into an issue: It feels wrong to me. With the work that I did recently on removing the fake sharding, debugging a finder plugin becomes quite a bit easier and also a column and a filter for the view is dropped. That however means that the new view in the backend displays almost 100% the same as if you would simply look into your database admin tool of choice for the #__finder_terms table. This feature also aims basically exclusively towards developers of finder plugins and that is a really nieche target demographic to warant to add this feature.
I understand that this view would be usefull for developers of finder plugins with the confusing sharding that we have right now, but with that being hopefully removed with #20185, the usefullness of this view is dramatically reduced.
How do you think about this? Should I still follow through with your PR and create one for 4.0-dev or should we rather drop this?
| Category | SQL Administration com_admin Postgresql MS SQL com_finder Language & Strings | ⇒ | Administration com_admin com_finder MS SQL Postgresql SQL |
| Title |
|
||||||
| Category | SQL Administration com_admin Postgresql MS SQL com_finder | ⇒ | SQL Administration com_admin Postgresql MS SQL com_finder Language & Strings |
Can we close this one? Staging doesn't accept new features anymore and for 4.0 this change is not necessary anymore, as I described above. For almst a year no response was given to my comment.
closed as stated above.
| Status | Pending | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2019-04-21 15:39:20 |
| Closed_By | ⇒ | franz-wohlkoenig | |
| Labels |
Removed:
J3 Issue
|
||
I have tested this item? unsuccessfully on 41b9ae6
Done Test Instructions until click on "Index": got "An Error Has Occurred - Unknown column 'terms_count' in 'field list' SQL=UPDATE
#__finder_linksSETmd5sum= 'b08e86fd430bdd09a730c3d35ff80162' ,terms_count= 64 WHERElink_id= '1'This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13377.