What happened?

Smart Search is diacritic-sensitive, so a search term typed without accents never matches the indexed accented word. On a Greek site, searching βιβλιοθηκη returns 0 results while βιβλιοθήκη returns 10 — the same word, and the one users on phones are most likely to type. This is not limited to Greek; it affects every language that uses diacritics.

The cause is in the schema, not the query logic. #__finder_terms, #__finder_terms_common, #__finder_tokens and #__finder_tokens_aggregate declare term/stem as utf8mb4_bin:

https://github.com/joomla/joomla-cms/blob/6.2-dev/installation/sql/mysql/extensions.sql#L376-L379

CREATE TABLE IF NOT EXISTS `#__finder_terms` (
  `term` varchar(75) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `stem` varchar(75) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',

Same declaration at lines 402, 596 and 617-618, and unchanged on 5.4-dev, 6.1-dev and 6.2-dev.

utf8mb4_bin compares byte-for-byte, so ή and η can never be equal — in =, in LIKE 'x%' (Term Matching "Begin with"), or in LIKE '%x%' ("Fuzzy"). There is no configuration option that changes this.

The usual justification for a binary collation — preserving case — does not apply here. Indexer\Language::tokenise() already lowercases every token in PHP before it is stored:

https://github.com/joomla/joomla-cms/blob/6.2-dev/administrator/components/com_finder/src/Indexer/Language.php#L148

$input = StringHelper::strtolower($input);

I confirmed this empirically: SELECT COUNT(*) FROM #__finder_terms WHERE BINARY term <> BINARY LOWER(term) returns 0 on a real index. So in practice utf8mb4_bin's only remaining effect on matching is diacritic-sensitivity.

Version

6.1.2 (declaration also present on 5.4-dev, 6.1-dev, 6.2-dev)

Expected result

Searching a word without its diacritics finds the accented word, as users reasonably expect and as most search engines behave.

Actual result

0 results. The user has to reproduce the exact accents to find anything — and with Term Matching on "Begin with", partial input fails as soon as it reaches the first accented character (βιβλιοθ → 10 results, βιβλιοθη → 1, βιβλιοθή → 10).

System Information

• Joomla 6.1.2, PHP 8.3.32, MariaDB 10.6.19
• Site language el-GR, Term Matching = "Begin with", stemmer snowball

Tested on the server, same MariaDB instance:

Additional Comments

Switching those columns to utf8mb4_unicode_ci fixes it — I've applied it to a production site and search now behaves as expected. All four tables must move together, since the indexer joins tokens ↔ tokens_aggregate ↔ terms ↔ terms_common and a mismatch raises Illegal mix of collations mid-index. (soundex can stay utf8mb4_bin — ASCII codes, only ever compared with itself.)

I want to be upfront about the two real costs, which may well be why this hasn't been changed:

1. It cannot be a plain ALTER on an existing site. idx_term_language(term, language) is UNIQUE, and terms that differ only by diacritics collide once accents stop being significant. On a small site (~230 indexed items) 135 term groups collided — αγαπη/αγάπη, αγιου/αγίου, σύλλογός/συλλογος/σύλλογος. The index has to be purged first and rebuilt, so any update SQL would need to be paired with a forced reindex. #__finder_terms_common is not derived data and would need its own de-duplication (one collision there: πως/πώς).

2. It merges words that differ only by diacritics. In Greek, άλλα ("other") and αλλά ("but") become one term; likewise χρόνια/χρονιά. Recall improves, precision drops slightly. For most sites that is the right trade, but it is a behaviour change, not a pure bug fix.

Given that, some options, in rough order of how disruptive they are:

• Change the collation for new installs only, leaving existing sites alone unless they opt in.
• Expose it as a Smart Search option (like the existing Term Matching setting) so a site can choose accent-sensitive or accent-insensitive.
• Keep the schema and handle folding in the indexer/query layer instead, normalising diacritics into stem — heavier, but keeps term exact and would let both behaviours coexist.
• At minimum, document it, since today it looks like a bug to any site owner in a language with diacritics and there is nothing in the UI to suggest a cause.

Happy to open a PR for whichever direction maintainers prefer.