Joomla! Issue Tracker | Joomla! CMS #22042 - [4.0] Finder: Replacing iconv() with mb_convert

Fixed in Code Base
19 Sep 2018
Medium
Build: 4.0-dev
# 22042
Diff
Hackwar:patch-31

Pending

User tests: Successful: Unsuccessful:

Hackwar
7 Sep 2018

The original issue is this here: https://forum.joomla.org/viewtopic.php?t=860554#p3503601
Looking through the documentation, iconv() indeed has issues: https://secure.php.net/manual/en/function.iconv.php

The more reliable solution as it seems is this code.

Testing Instructions

Apply this patch and run the indexer of Smart Search in the backend. See that it still works.

be3db30 7 Sep 2018

Replacing iconv() with mb_convert_encoding()

Hackwar - open - 7 Sep 2018

Hackwar - change - 7 Sep 2018

Status

New

⇒

Pending

joomla-cms-bot - change - 7 Sep 2018

Similar solution is at:

joomla-cms/libraries/src/Filter/InputFilter.php

Lines 796 to 814 in eb68e01

    
           	protected function remove($source) 
        
           	{ 
        
           		// Check for invalid UTF-8 byte sequence 
        
           		if (!preg_match('//u', $source)) 
        
           		{ 
        
           			// String contains invalid byte sequence, remove it 
        
           			$source = htmlspecialchars_decode(htmlspecialchars($source, ENT_IGNORE, 'UTF-8')); 
        
           		} 
        
           		// Iteration provides nested tag protection 
        
           		do 
        
           		{ 
        
           			$temp = $source; 
        
           			$source = $this->_cleanTags($source); 
        
           		} 
        
           		while ($temp !== $source); 
        
           		return $source; 
        
           	}

laoneo - comment - 10 Sep 2018

The PHP mb extension is not always installed on hosts. So either way I would add a check or fallback to iconv if it is not available.

mbabker - comment - 10 Sep 2018

symfony/polyfill-mbstring covers this if the extension isn't installed and it is already a transient dependency of 4.0 so this is fine as is.

laoneo - comment - 11 Sep 2018

laoneo - close - 19 Sep 2018

laoneo - merge - 19 Sep 2018

laoneo - change - 19 Sep 2018

Status	Pending	⇒	Fixed in Code Base
Closed_Date	0000-00-00 00:00:00	⇒	2018-09-19 08:13:41
Closed_By		⇒	laoneo
Labels	Added: ?

laoneo - comment - 19 Sep 2018

Thanks

Add a Comment

Older
Newer

Joomla! Issue Tracker - CMS

[#22042] - [4.0] Finder: Replacing iconv() with mb_convert_encoding()

Testing Instructions

Add a Comment

	protected function remove($source)
	{
	// Check for invalid UTF-8 byte sequence
	if (!preg_match('//u', $source))
	{
	// String contains invalid byte sequence, remove it
	$source = htmlspecialchars_decode(htmlspecialchars($source, ENT_IGNORE, 'UTF-8'));
	}

	// Iteration provides nested tag protection
	do
	{
	$temp = $source;
	$source = $this->_cleanTags($source);
	}
	while ($temp !== $source);

	return $source;
	}