• I read the Generative AI policy and my contribution is either not created with the help of AI or is compatible with the policy and GNU/GPL 2 or later.

Summary of Changes

Smart Search's "Relevance" ordering sums the raw weights stored in
#__finder_links_terms. That weight is calculated at index time from the term
frequency, the length of the term and the multiplier of the context the term
appeared in. It carries no information about how rare the term is, so a match
on a very common word counts for exactly as much as a match on a highly
distinctive one. In a multi-word query the common words therefore drown out the
words that actually identify what the user is looking for.

This PR scales every match by the inverse document frequency (IDF) of its term:

LN(1 + totalIndexedLinks / ti.links)

ti.links is the per-term link counter the indexer already maintains in
#__finder_terms, so no schema change and no reindex is required — the data
this needs is already there on every existing site.

Two small notes:

• LN() is used rather than LOG(), because LOG() is base 10 on PostgreSQL
  and base e on MySQL. LN() is the natural logarithm on both.
• The * 1.0 in the ratio is load bearing: both operands are integers and
  PostgreSQL would otherwise do integer division and truncate the ratio to 0 or 1.

Also included:

• A new Cypress system test, tests/System/integration/site/components/com_finder/Relevance.cy.js,
  which builds a small corpus of articles with a controlled term distribution and
  asserts the resulting ordering.
• tests/System/integration/site/components/com_finder/Search.js is renamed to
  Search.cy.js. The Cypress specPattern in cypress.config.dist.mjs only
  matches *.cy.{js,jsx,ts,tsx}, so that existing Smart Search test has never
  actually been executed.

While implementing and testing this, it turned out that there are additional bugs in the system where the counter in the terms table is not correctly updated upon deletion. However this is outside of the scope of this PR and will be fixed in another PR. There are also other changes planned for Smart Search which affect indexing results, but again, those are outside of the scope of this PR and will have to wait a bit.

Testing Instructions

1. Create a set of articles so that one word is common across the site and
   another is rare. For example:
   • ~12 articles that each contain the word zorbex once.
   • One article "Decoy" containing zorbex five times and quilon once.
   • One article "Target" containing zorbex once and quilon four times.
2. On the site, search for zorbex quilon and set the ordering to "Relevance".
3. Then search for just zorbex and check the ordering again.

Alternatively, run the new system test:
npx cypress run --spec tests/System/integration/site/components/com_finder/Relevance.cy.js

Actual result BEFORE applying this Pull Request

Searching for zorbex quilon puts "Decoy" above "Target". It holds six matching
words in total against the target's five, and because every match is worth the
same regardless of how common its term is, the higher raw count wins — even
though all but one of the decoy's matches are on the word that appears all over
the site and tells you nothing.

Expected result AFTER applying this Pull Request

Searching for zorbex quilon puts "Target" above "Decoy". Its matches are
concentrated in the rare term, which is the one that distinguishes it from the
rest of the corpus.

Searching for zorbex alone is unchanged: a single-term query applies one
uniform factor to every match, so plain frequency ordering still applies and
"Decoy" is still first.

Link to documentations

Please select:

• Documentation link for guide.joomla.org:

• No documentation changes for guide.joomla.org needed

• Pull Request link for manual.joomla.org:

• No documentation changes for manual.joomla.org needed