? ? Pending

User tests: Successful: Unsuccessful:

avatar Hackwar
Hackwar
18 Apr 2020

This converts several of the core mails to the mail template system.

List of mails send in Joomla

  • Testmail in global configuration
  • Contact form in com_contact
  • com_contact API controller to send a mail
  • Joomla Updatenotification
  • Massmailer mail
  • Mail to the new user when creating a new user in the backend
  • Password reset for users in frontend
  • Username reminder for users in frontend
  • Notification mail from com_actionlogs (administrator\components\com_actionlogs\src\Model\ActionlogModel.php line 174)
  • com_messages notification (administrator\components\com_messages\src\Model\MessageModel.php line 435)
  • com_privacy send export to user (administrator\components\com_privacy\src\Model\ExportModel.php line 240)
  • com_privacy notification about a new request (administrator\components\com_privacy\src\Model\RequestModel.php line 362)
  • com_privacy creating a new request (components\com_privacy\src\Model\RequestModel.php line 188/190 ATTENTION!! HERE IS A BUG! The mail is send twice!)
  • reminder of expiring privacy consent (plugins\system\privacyconsent\privacyconsent.php line 604)
  • User registration (components\com_users\src\Model\RegistrationModel.php line 198, 254, 650, 711)

If someone else wants to take over the remaining ones, be my guest.

Testinstructions

Basically, you have to trigger sending the mails in the cases marked as done above and see if they still arrive identically to the way they did before. Then create a template and see if your modifications are reflected.

Comments

I had to extend the MailTemplate class with a few modifications and I guess this wont be the last time this is necessary.

Votes

# of Users Experiencing Issue
0/1
Average Importance Score
5.00

avatar Hackwar Hackwar - open - 18 Apr 2020
avatar Hackwar Hackwar - change - 18 Apr 2020
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 18 Apr 2020
Category SQL Administration com_admin Postgresql com_users Language & Strings Front End com_contact Installation Libraries Plugins
avatar Hackwar Hackwar - change - 18 Apr 2020
The description was changed
avatar Hackwar Hackwar - edited - 18 Apr 2020
avatar Hackwar Hackwar - change - 18 Apr 2020
Labels Added: ? ?
avatar toivo
toivo - comment - 19 Apr 2020

Results from selected initial tests to see if the marked emails work as before:
Mail: "Testmail in global configuration"
Result: works as before

Mail: "Mail to the new user when creating a new user in the backend”
Result: failure
"Warning
Empty message body
Email could not be sent"

Mail: "Password reset for users in frontend"
Result: failure
"Warning
Empty message body
Notice
Reminder failed: Failed sending email."

My test environment is Windows 10, Wampserver 3.20, MySQL 5.7.23 and PHP 7.4.2.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 19 Apr 2020

Changed the Mail Templates – Options: Per Template Mail Settings No --> Yes
The result from requesting username reminder and password reset emails is error "0 Call to a member function get() on null"


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 19 Apr 2020

params of com_mails: {"mail_style":"plaintext","alternative_mailconfig":"1","copy_mails":"0","attachment_folder":""}
Call stack

Function Location

1 () JROOT\libraries\src\Mail\MailTemplate.php:198
2 Joomla\CMS\Mail\MailTemplate->send() JROOT\components\com_users\src\Model\RemindModel.php:191
3 Joomla\Component\Users\Site\Model\RemindModel->processRemindRequest() JROOT\components\com_users\src\Controller\RemindController.php:42
4 Joomla\Component\Users\Site\Controller\RemindController->remind() JROOT\libraries\src\MVC\Controller\BaseController.php:729
5 Joomla\CMS\MVC\Controller\BaseController->execute() JROOT\libraries\src\Dispatcher\ComponentDispatcher.php:146
6 Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() JROOT\libraries\src\Component\ComponentHelper.php:384
7 Joomla\CMS\Component\ComponentHelper::renderComponent() JROOT\libraries\src\Application\SiteApplication.php:196
8 Joomla\CMS\Application\SiteApplication->dispatch() JROOT\libraries\src\Application\SiteApplication.php:235
9 Joomla\CMS\Application\SiteApplication->doExecute() JROOT\libraries\src\Application\CMSApplication.php:231
10 Joomla\CMS\Application\CMSApplication->execute() JROOT\includes\app.php:63
11 require_once() JROOT\index.php:36


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar Hackwar
Hackwar - comment - 20 Apr 2020

Have you applied the database changes as well?

avatar toivo
toivo - comment - 20 Apr 2020

Which database changes? I tested #28722 with J4 Patch Tester RC2 and the latest J4 Nightly Build. Yesterday I noticed that the latest Nightly Build was in fact from April 14, which may explain the issue. Asked about it in #27666 and richard67 was going to inform the CMS maintenance team. April 14 is still the latest Nightly Build.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar Quy
Quy - comment - 20 Apr 2020

You can also try the download package in this PR.
Scroll to the bottom.
Click on Show all checks.
In the Download row, click Details.

https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31378/

avatar toivo
toivo - comment - 21 Apr 2020

@Quy Sorry, I tried but your instructions are not detailed enough. What am I supposed to download from https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31378/ ? Links to both .zip files are broken.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 21 Apr 2020

@toivo When testing a Pull Request (PR) it is always good to look first which files it changes. You can see that for this PR here on GitHub: https://github.com/joomla/joomla-cms/pull/28722/files. If a PR adds a new update sql script or changes an existing one, it means the PR also contains database changes. In case of this PR here these are the files 4.0.0-2020-04-16.sql added by this PR. To apply such database changes you can use PhpMyAdmin or whatever and run the SQL from the file for your database type (mysql I guess) in that tool. You have to replace the "#__" with your actual database prefix in table names before executing the SQL. You should know that there is no way tro roll back these database changes, i.e. you should do such tests only on a testing environment which you later can throw away or reset to a standard installation, never do that on a real live site. The Patchtester can't apply and revert database changes.

I hope these explanations help you a bit to understand the previous comment about database changes of this PR.

avatar toivo
toivo - comment - 21 Apr 2020

@richard67, that was helpful, thanks. I will apply the two .sql files against the database created by the Nightly Build from April 14 and run the J4 Patch Tester RC2. - Any news when the Nightly Builds might continue with new versions?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 21 Apr 2020

@toivo They are still working on it as far as I know. Regarding SQL, don't forget to replace the "#__" with the table prefix (including the onderscore at the end) before running the statements.

avatar toivo
toivo - comment - 21 Apr 2020

@Richar67 Just installed the latest Nightly Build from April 14, executed com_admin/sql/updates/mysql/4.0.0-2020-04-16.sql and applied PR #28722 using J4 Patch Tester RC2, then found that the language files are out of date because com_mails is displaying raw language strings, for example com_contact_MAIL_mail_TITLE. I would like to test this PR but where are the updates to the language files?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar richard67
richard67 - comment - 21 Apr 2020

@toivo I can't see any language sting with name like com_contact_MAIL_mail_TITLE (case insensitive) is being used. So either there is something wrong with your testing environment, or the name of that language string is different.

avatar richard67
richard67 - comment - 21 Apr 2020

@Hackwar It seems there is something wrong with the update sql script. First of all there is a warning or error (depending on mysql strict mode settings) telling that columns htmlbody and attachments don't have a default value. I think you should include these columns into your insert statement and insert an emptry string for them. This should be done for both the mysql and the postgresql file, like it is already done in the supports.sql for the corresponding database type.

In addition, @toivo is right: There are strange language sting names instead of real texts shown in the mail templates list view after having run the update SQL. Here just an example with 2 of them, but it's true for all the new records.

j4_pr-28722_wrong-texts-in-list-view

avatar toivo
toivo - comment - 26 Apr 2020

Localhost: Windows 10, Wampserver 3.2.1, Apache 2.4.1, MySQL 8.0.19, PHP 7.4.5
J4 version: Nightly Build of April 26 - https://developer.joomla.org/nightlies/Joomla_4.0.0-beta1-dev-Development-Full_Package.zip

Part 1 "Test if emails still arrive identically to the way they did before"
• Testmail in global configuration - OK
• Mail to the new user when creating a new user in the backend - OK
• Password reset for users in frontend - OK
• Username reminder for users in frontend - OK
• Contact form in com_contact - OK

• not tested: com_contact API controller to send a mail
• not tested: Joomla Updatenotification

• Massmailer mail - error - details below:

An error has occurred.
0 Too few arguments to function Joomla\CMS\Mail\MailTemplate::__construct(), 1 passed in C:\www\joomla4\administrator\components\com_users\src\Model\MailModel.php on line 179 and at least 2 expected
Call stack

Function Location

1 () JROOT\libraries\src\Mail\MailTemplate.php:90
2 Joomla\CMS\Mail\MailTemplate->__construct() JROOT\administrator\components\com_users\src\Model\MailModel.php:179
3 Joomla\Component\Users\Administrator\Model\MailModel->send() JROOT\administrator\components\com_users\src\Controller\MailController.php:44
4 Joomla\Component\Users\Administrator\Controller\MailController->send() JROOT\libraries\src\MVC\Controller\BaseController.php:729
5 Joomla\CMS\MVC\Controller\BaseController->execute() JROOT\libraries\src\Dispatcher\ComponentDispatcher.php:146
6 Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() JROOT\libraries\src\Component\ComponentHelper.php:384
7 Joomla\CMS\Component\ComponentHelper::renderComponent() JROOT\libraries\src\Application\AdministratorApplication.php:131
8 Joomla\CMS\Application\AdministratorApplication->dispatch() JROOT\libraries\src\Application\AdministratorApplication.php:174
9 Joomla\CMS\Application\AdministratorApplication->doExecute() JROOT\libraries\src\Application\CMSApplication.php:231
10 Joomla\CMS\Application\CMSApplication->execute() JROOT\administrator\includes\app.php:63
11 require_once() JROOT\administrator\index.php:36

MySQL reported two warnings when the SQL update file 4.0.0-2020-04-16.sql was executed:
1364 Field 'htmlbody' doesn't have a default value
1364 Field 'attachments' doesn't have a default value


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar richard67
richard67 - comment - 26 Apr 2020

MySQL reported two warnings when the SQL update file 4.0.0-2020-04-16.sql was executed:
1364 Field 'htmlbody' doesn't have a default value
1364 Field 'attachments' doesn't have a default value

I've reported this already with my comment above.

avatar richard67
richard67 - comment - 26 Apr 2020

@toivo Thanks for reporting back. Please wait with further tests until the necessary corrections have been made in this Pull Request (PR). I'll give you a notice when the PR can be tested again.

avatar richard67
richard67 - comment - 26 Apr 2020

@Hackwar For the SQL error mentioned in the comments above for updating see Hackwar#40.

avatar richard67
richard67 - comment - 26 Apr 2020

@Hackwar For the missing language strings and the replacement of dots by underscores in language string names for titles and descriptions see Hackwar#41. Please review the descriptions since I am not sure if they are 100% right.

avatar richard67
richard67 - comment - 26 Apr 2020

@toivo If you want to repeat your tests: You can download a full installation package "Joomla_4.0.0-beta1-dev+pr.28722-Development-Full_Package.zip" which includes the changes from this Pull Request (PR) from this page here: https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31563/. Unpack it into an empty folder on your webserver and make a new installation with this package. After that, you don't need to install patchtester and apply this PR, it is already applied with the installation. Then you can test mailing functions.

If you want to test if the update sql scripts of this PR work, you can install a new, clean 3.10 nightly build. Then you can update this 3.10 to a 4.0-dev with this PR applied by using either the update package "Joomla_4.0.0-beta1-dev+pr.28722-Development-Update_Package.zip" downloaded from the page linked above and the "Upload & Update" tab of the "Joomla Update" component, or you use the more comfortable way, setting the update channel to "Custom URL" in the "Joomla Update" component's option, entering the update URL "https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31563/pr_list.xml" listed at the bottom of the page mentioned above and using the "Live Update" tab of the "Joomla Update" component.

Of course you can also apply the patch of this PR with patchtester on a clean 4.0 nightly build and then execute the SQL statements from the update sql script, replacing the "#___" by the table prefix, if you feel more comfortable with this, but then the 4.0 needs to be a clean fresh nightly or current 4.0-dev branch where no other patches are applied.

avatar toivo
toivo - comment - 27 Apr 2020

richard67 thanks, I use today's nightly build, ran the same tests as last time, part 1 and the results were the same - error in Mass Mail.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 28 Apr 2020

Today's Nightly Build and initial testing of Part 2 "Create a new template and see if your modifications are reflected"

• Testmail in global configuration - OK with plain text, HTML template not saved and reports "Empty message body. Email could not be sent"
• Mail to the new user when creating a new user in the backend - OK with plain text, HTML template not saved and reports "Empty message body. Email could not be sent"

The available tags are listed in lower case. When a tag is clicked, it is added in CAPS. If a tag is typed in and saved in lower case, the tag is not transformed. Suggest that tags are presented in CAPS. If a tag is added in lower case, it should be converted to CAPS when the form is saved. There may also be other options to make the behaviour consistent.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar Hackwar
Hackwar - comment - 28 Apr 2020

I fixed the issues that you had with the massmail feature.

avatar toivo
toivo - comment - 29 Apr 2020

@Hackwar thanks, both the default mail and the mail sent after updating the Mass Mail template are now working.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 29 Apr 2020

Questions how to test the remaining two core mails included in this PR:

the com_contact API controller to send a mail

  • is there any documentation how to use the con_contact API?

Joomla Updatenotification

avatar richard67
richard67 - comment - 29 Apr 2020

how could an earlier version be enforced so that the update notification gets triggered?

@toivo You can patch the version of your installation to something lower than what it really is.

Just change the EXTRA_VERSION here

https://github.com/joomla/joomla-cms/blob/4.0-dev/libraries/src/Version.php#L64

e.g. to "alpha12".

Then in the Joomla Update component's options, change the update channel to "Custom URL", minimum stability to "Development" and as URL enter the custom URL for the 4.0 nightly builds:
https://update.joomla.org/core/nightlies/next_major_list.xml.

Then save the options, use the button to search for updates, and then an update should be found.

avatar toivo
toivo - comment - 2 May 2020

@richard67 thank you, that sounded simple but unfortunately it did not work.

The plugin updatenotification.php seems to look up the version information from the row files_joomla in the extensions table, the id of which is 208, at least right now. After the version is changed to '4.0.0-alpha12' and the Unix time lastrun in the params column of the plugin is changed to an earlier time, the System Dashboard page highlights the availability of 4.0.0-beta1-dev in yellow and under 'Update Checks' the plg_quickicon_joomlaupdate box is red, but the updatenotification plugin does not send any emails.
The lastrun Unix time in the params column of the plugin in the extensions table is updated every time. I guess comparing the code between updatenotification.php and plg_quickicon_joomlaupdate might reveal the reason why emails are not sent. -- I even uncommented the following line in updatenotification.php to force the update before realising that the lastrun parameter was the key: // define('PLG_SYSTEM_UPDATENOTIFICATION_DEBUG', 1);


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar toivo
toivo - comment - 2 May 2020

@richard67 ... and, if you are wondering, my J4 test site sends emails perfectly all right, including the test mail and for example when a new user is created from the back end, therefore SMTP mail is not the root cause.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 4 May 2020

@Hackwar & @richard67 - in addition to the above problem with getting Updatenotification to send an email, here is a question about com_contact API controller to send a mail. Managed to work out how to use the API from [#27021] - [4.0] Token authentication for the API application, then listed contacts using GET in Postman but attempt to POST to create a contact generates the same error, whether the fields Name and Category are included in the request or not.

Request:
http://joomla4/api/index.php/v1/contact?type=contacts&name=Joe Bloggs&alias=joe-bloggs&catid=8&Name=My Name&Category=My Contact Category

Result:
{
"errors": [
{
"title": "Field required: Name\nField required: Category"
}
]
}
Otherwise the results PR#28722 look good and the testing could be finalised quickly.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 4 May 2020

@toivo Sorry, can't help with the API questions. I hope someone else can.

avatar toivo
toivo - comment - 4 May 2020

@richard67 No problems. URL encoding the name field did not get rid of the error. Here is the POST: http://joomla4/api/index.php/v1/contact?name=John%20Smith&alias=john-smith&catid=8
... and this is the result:
{
"errors": [
{
"title": "Field required: Name\nField required: Category"
}
]
}
Hopefully adding a few debug Log statements to the com_contact API code will reveal what is going on.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 4 May 2020

@Hackwar - here is the trace:
CRITICAL 127.0.0.1 error Uncaught Throwable of type Tobscure\JsonApi\Exception\InvalidParameterException thrown with message "Field required: Name
Field required: Category". Stack trace: #0 C:\www\joomla4\api\components\com_contact\src\Controller\ContactController.php(83): Joomla\CMS\MVC\Controller\ApiController->save(NULL)
#1 C:\www\joomla4\libraries\src\MVC\Controller\ApiController.php(342): Joomla\Component\Contact\Api\Controller\ContactController->save()
#2 C:\www\joomla4\libraries\src\MVC\Controller\BaseController.php(729): Joomla\CMS\MVC\Controller\ApiController->add()
#3 C:\www\joomla4\libraries\src\Dispatcher\ApiDispatcher.php(59): Joomla\CMS\MVC\Controller\BaseController->execute('add')
#4 C:\www\joomla4\libraries\src\Component\ComponentHelper.php(384): Joomla\CMS\Dispatcher\ApiDispatcher->dispatch()
#5 C:\www\joomla4\libraries\src\Application\ApiApplication.php(333): Joomla\CMS\Component\ComponentHelper::renderComponent('com_contact')
#6 C:\www\joomla4\libraries\src\Application\ApiApplication.php(110): Joomla\CMS\Application\ApiApplication->dispatch()
#7 C:\www\joomla4\libraries\src\Application\CMSApplication.php(231): Joomla\CMS\Application\ApiApplication->doExecute()
#8 C:\www\joomla4\api\includes\app.php(60): Joomla\CMS\Application\CMSApplication->execute()
#9 C:\www\joomla4\api\index.php(35): require_once('C:\www\joomla4\...')
#10 {main}

Looking at the save() function in Api\Controller\ContactController, line 74 looks odd:
!isset($data['com_fields']) && $data['com_fields'] = [];


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar toivo
toivo - comment - 4 May 2020

@Hackwar - this is an interesting exercise. Used Postman to post the following request http://joomla4/api/index.php/v1/contact?name=John%20Smith&alias=john-smith&catid=8&language=en-GB (had to add language because "Save failed with the following error: Field 'language' doesn't have a default value") and the result was {"errors":[{"title":"Field required: Name\nField required: Category"}]}.

Now the good news: a CLI script using cURL and JSON request added a new contact all right, but the contact was unpublished. But no email was triggered. When is the com_contact API controller expected to send an email? I thought that a new contact would have triggered that.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar alikon
alikon - comment - 4 May 2020
avatar toivo
toivo - comment - 4 May 2020

@alikon thanks, that useful document did not come up in my searches earlier but I managed to find the right input parameters for adding a Contact by trial and error. I might update the doc and add the details of the Token authentication method that was mentioned briefly in the spec https://docs.joomla.org/Joomla_Api_Specification, which method is now working perfectly

Now the key question is whether a POST request should allow only JSON format, which is working, or is it also expected to accept URL variables, which format does not seem to work in my test: name=John%20Smith&alias=john-smith&catid=8&language=en-GB


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar Hackwar
Hackwar - comment - 6 May 2020

The email of the com_contact contact API is similar to the contact form one, so it is a mail that you can send to the contact.

avatar toivo
toivo - comment - 6 May 2020

@Hackwar thank you, just tested the contact form through API and it worked all right. Now the only email template I still have to work out how to trigger and then test is the Joomla Update notification. Looks promising.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 8 May 2020

@Hackwar @richard67 after the tweaks mentioned at https://issues.joomla.org/tracker/joomla-cms/28722#event-525740 managed to test successfully the Joomla Updatenotification mail template before and after modifying the mail template, but one more tweak and one bug fix were required, the details are below. Can I now mark my test as successful?

File: plugins/system/updatenotification/updatenotification.php after PR#28722 was applied
line 176
// return; // $updates was always empty
line 269

  •   		$mailer = new MailTemplate('joomla.updatenotification', $jLanguage->getTag());
    
  •   		$mailer = new MailTemplate('plg_system_updatenotification.mail', $jLanguage->getTag()); // template_id<hr /><sub>This comment was created with the <a href="https://github.com/joomla/jissues">J!Tracker Application</a> at <a href="https://issues.joomla.org/tracker/joomla-cms/28722">issues.joomla.org/tracker/joomla-cms/28722</a>.</sub>
    
avatar richard67
richard67 - comment - 8 May 2020

@toivo What do you mean with "but one more tweak and one bug fix were required"? The tweak is a hack needed for testing, similar to the version number hack. But the bug fix? Is it a known bug, i.e. do we have an issue or a PR for that?

avatar toivo
toivo - comment - 8 May 2020

@richard67 line 269 in updatenotification.php has incorrect template_id and I cannot find updatenotification.php from the reported issues. I would be happy to raise it as a bug, but can it be done separately from PR#28722? I know very little about forks etc. Please advise.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar toivo
toivo - comment - 9 May 2020

@richard67 the actual tweak I had to do is actually on line 164. Forced the minimum stability for findUpdates() from 'stable' to 'development' but even then findUpdates() $results was empty and therefore had to comment out return from line 164.

	// Get any available updates
	$updater = Updater::getInstance();
	$results = $updater->findUpdates([$eid], $cache_timeout, 0);  // added 0 = 'development' to replace default 'stable'

	// If there are no updates our job is done. We need BOTH this check AND the one below.
	if (!$results)
	{
		// return;
	}

This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar toivo toivo - test_item - 9 May 2020 - Tested successfully
avatar toivo
toivo - comment - 9 May 2020

I have tested this item successfully on 9517235


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 21 May 2020

@toivo Would be cool if you could test again with the latest changes. Thanks in advance.

avatar richard67
richard67 - comment - 22 May 2020

@toivo Sorry, please wait a bit with testing. I don't want to waste your time. I am testing it myself and find some smaller issues, which might be corrected, and then you would have to test again, so better wait a bit to save one testing.

avatar toivo
toivo - comment - 22 May 2020

@Richar67 Ok, I will wait. I had tested the other mails before the patch, except Joomla Updatenotification, then was struggling with com_contact API - the API does not work if 'Use URL Rewriting' is selected in Global Configuration, which looks like a bug. Also my CLI script that posts JSON data to the com_contact form and used to work now produces HTTP 500 error "Runtimeexception: Display email form disabled".


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 24 May 2020

@toivo

I had tested the other mails before the patch, except Joomla Updatenotification

I think you haven't tested all cases, otherwise you would have seen that there is a problem with replacing the language strings when toggling the "No" button to "Yes" in the template for new user creation in backend. I have provided a correction for that, and @Hackwar just has merged it. More details see at the bottom of this comment.

then was struggling with com_contact API - the API does not work if 'Use URL Rewriting' is selected in Global Configuration, which looks like a bug.

Can you reproduce this on a new installation of latest 4.0 nightly build without this PR here applied? If so, please create a new issue for it.

Also my CLI script that posts JSON data to the com_contact form and used to work now produces HTTP 500 error "Runtimeexception: Display email form disabled".

Same as above: Does it work when using latest 4.0 nightly without this PR, and it only doesn't work when this PR here is applied? Or doesn't it work in both cases? If the latter, then you should open a new issue.

Now regarding the tests: For each mail template it needs to test 3 cases.

  1. No mail template created for language en-GB: Everything should work as well as without this PR.
  2. Mail template created for language en-GB, but left with the default settings, i.e. the toggle buttons for subject and body both are set to "No", and the fields for subject and body show the language string names: Same as 1, all should work in the same way as without this PR.
  3. Edit the mail template and set the toggle buttons to "Yes": The language string names should be replaced by the content of the strings, which may contain tags like "{SITENAME}" or similar. If you toggle the buttons back, the language string names are shown again. Finally toggle the buttons again to "Yes". Then you can modify these texts if you want. Then save. Then test that all works as well as before, but the modified texts are used if you have modified them.

When testing with other languages than en-GB, the language strings might not contain the replacement tags like e.g. "{SITENAME}". The reason is that we don't really have J4 language packs yet. But beside that it should work the same way, and you can use these tags in subject and body.

avatar richard67
richard67 - comment - 24 May 2020

@toivo Please check my previous comment and test again. Thanks a lot for your efforts up to now, and thanks in advance for further tests. I know it's time consuming.

avatar richard67
richard67 - comment - 24 May 2020

There is one issue left for which I don't know yet if it happens also without this PR: When using mass mail there is a check box for HTML mode, but this seems to be ignored. Mail is always sent as text with content type text.

avatar richard67
richard67 - comment - 24 May 2020

I've tested everything with success except of the API, for this I don't have the environment or I need advice, because my local testing environment has not mailing set up, and the environment I used here is on my domain, where mailing works, but it is protected with an htaccess password.

avatar toivo
toivo - comment - 24 May 2020

@richard67 thanks for the detailed instructions, I will tackle the test tomorrow. My localhost websites use Gmail account with Application Specific Password, following the tutorial https://docs.joomla.org/How_do_I_use_Gmail_as_my_mail_server%3F (which I helped to keep uptodate). Let me know if you would like to use a PHP CLI script to post the API request to the Contact form.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 24 May 2020

@toivo Is it right that we are dealing here with what is described in the API docs as follows:

Submit Contact Form

curl -X POST -H "Content-Type: application/json" /api/index.php/v1/contact/form/{contact_id} -d

{
    "contact_email": "email@mail.com",
    "contact_message": "some text",
    "contact_name": "name",
    "contact_subject": "subject"
}

Is that the right thing?

avatar toivo
toivo - comment - 24 May 2020

@richard67 that is the right POST request. Here is the PHP CLI script I am using as a text file:
post_json_contact_form.php.txt


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.
avatar richard67
richard67 - comment - 24 May 2020

@toivo I've managed to set up local mailing and managed to create a new contact and other stuff via API using curl on command line. Only for the contact form handled by this PR I get an internal server error 500. Same when using your CLI script (adapted to my needs). I have to check if it works without this PR, but maybe you are faster.

avatar richard67
richard67 - comment - 24 May 2020

@toivo I get the internal server error also without this PR applied.

avatar toivo
toivo - comment - 25 May 2020

As long as the API plugins are enabled, the user profile is saved once and the API token is copied and pasted to the script, it will work, as long as 'Use URL Rewriting' has NOT been selected in Global Configuration. Just tested this again in today's Nightly Build.

The issue is discussed in #28394, which patch allows the API to work when 'Use URL Rewriting' has been selected. Hopefully #28394 is soon fixed so that the API will work in both cases, whether 'Use URL Rewriting' is selected or not.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 25 May 2020

@toivo I have not enabled URL rewriting in global config because I know of the error you've mentioned. But as I said, I get here an internal server error also without it.

avatar toivo
toivo - comment - 25 May 2020

@richard67 ok, I also get the internal server error after applying the patch, the error is "Display email form disabled". Here is the trace:

2020-05-25T06:00:28+00:00 CRITICAL 127.0.0.1 error Uncaught Throwable of type RuntimeException thrown with message "Display email form disabled.". Stack trace: #0 C:\www\joomla4\libraries\src\MVC\Controller\BaseController.php(729): Joomla\Component\Contact\Api\Controller\ContactController->submitForm()
#1 C:\www\joomla4\libraries\src\Dispatcher\ApiDispatcher.php(59): Joomla\CMS\MVC\Controller\BaseController->execute('submitform')
#2 C:\www\joomla4\libraries\src\Component\ComponentHelper.php(384): Joomla\CMS\Dispatcher\ApiDispatcher->dispatch()
#3 C:\www\joomla4\libraries\src\Application\ApiApplication.php(333): Joomla\CMS\Component\ComponentHelper::renderComponent('com_contact')
#4 C:\www\joomla4\libraries\src\Application\ApiApplication.php(110): Joomla\CMS\Application\ApiApplication->dispatch()
#5 C:\www\joomla4\libraries\src\Application\CMSApplication.php(231): Joomla\CMS\Application\ApiApplication->doExecute()
#6 C:\www\joomla4\api\includes\app.php(54): Joomla\CMS\Application\CMSApplication->execute()
#7 C:\www\joomla4\api\index.php(35): require_once('C:\www\joomla4\...')


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 25 May 2020

@toivo As I wrote I get my error also without the patch applied. But with your trace you reminded me to switch on debugging in global configuration, thanks for that ;-) The error message you get would mean that you have switched off the display of the contact form (email) in the settings of that contact or in global settings, and the contact inherits that. Could you check that?

avatar toivo
toivo - comment - 25 May 2020

@richard67 I do not think I turned off anything in the global options for contacts. This is still the first stage of testing, when no template has been created after the patch has been applied.

I am also struggling with the Updatenotification. The changes that made it run and send the email are not working in this Nightly Build.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.

avatar richard67
richard67 - comment - 25 May 2020

Other thing I've noticed: In Mass Mail there is a check box for HTML mode. If checked, the content type of the mail is "text/html", otherwise it is "text/plain", when this PR here is not applied. When this PR is applied, the content type is always "text/plain", regardless if the check box for HTML mode is checked or not. @Hackwar Any idea? Is that expected because no html mail with mail templates? Or is it a bug?

avatar richard67
richard67 - comment - 25 May 2020

Beside the HTML mode scheck box of the Mass Mail form being ignored, there are other, already known issues with HTML mode and mail templates, see #28524 and #27619 .

@Hackwar But the general question remains what to do with the check box options in the Mass Mail form. Maybe they should be removed from this form if these options have to be configured in the mail template?

avatar richard67
richard67 - comment - 25 May 2020

@toivo Regarding the update notification email I suggest you do not use any hack. Use a clean 4.0-dev or latest J4 nightly build. Then in the Joomla Update Component's option, change update channel to "Custom URL", set minimum stability to "Development" and enter following custom URL: https://test5.richard-fath.de/j4-nightlies-update-url-patched-for-test.xml, then save.

This should cause an update to 4.0.0-beta2-dev be found (which is fake because it just links to the nightly of tonight). The go to the Extension Installer's options and set the cache time to zero and minimum stability to "Development" and save. Then you should get some email, if everything else is set up right.

Please test if this works for you and report back the result.

avatar richard67
richard67 - comment - 25 May 2020

@toivo Maybe the internal server error in my case comes from my mail using smtp, but in the API's ContactController.php there seems the PHP mailer to be used in any case. So that could be another issue not related to this PR. Update: Forget about it. I see with debug on the same as you reported, the RuntimeException: Display email form disabled. in \/joomla-cms-4.0-dev\/api\/components\/com_contact\/src\/Controller\/ContactController.php:125. It seems to come from here:

if (!$contactParams->get('show_email_form'))
{
throw new \RuntimeException(Text::_('JLIB_APPLICATION_ERROR_DISPLAY_EMAIL_FORM'));
}

This part was not touched by this PR, so I think it's really unrelated.

avatar toivo
toivo - comment - 25 May 2020

@richard67 The configuration of the workaround to get the Updatenotification email without any tweaks worked all right, but my Super User received four (4) identical emails and they keep coming because of the cache time. I can now apply the patch and continue testing, unless some changes are going to be made overnight.

avatar richard67
richard67 - comment - 25 May 2020

@toivo I found the problem: I have to explicitely switch the "Show email form" to "Yes" for the particular contact, then it works.
j4-api-show-contact-form

The reason is that the code I've linked in my previous comment checks only the options of the particular contact, and only if Yes, not if any inherited setting.

That's definitely an issue not related to this PR.

But when I got it work, the email contained the language sting constants and nothing else. I have to check if this happens also without this PR.

avatar toivo
toivo - comment - 25 May 2020

@richard67 Ok, setting the option 'Contact Form' to 'Show' makes sense. The option 'Session Check' is probably not going to be used in the API.

avatar richard67
richard67 - comment - 25 May 2020

@toivo When I send the email now with the API to the contact, I get the language constants in the email when having no template for en-GB or when having a template with all buttons to "No" so it uses the language constants. Only when switching the buttons to "Yes" I get the translated text. This could be related to this PR. I will continue to investigate and provide a patch for @Hackwar like before if I find the reason and a valid fix.

Update: If I add the language strings COM_CONTACT_COPYSUBJECT_OF, COM_CONTACT_COPYTEXT_OF, COM_CONTACT_ENQUIRY_SUBJECT and COM_CONTACT_ENQUIRY_TEXT to file api/language/en-GB/joomla.ini, the emails contain the translated texts. But I don't think that's the way it should be solved, or is it? How is the api supposed to load language files? The one it should load here is language/en-GB/com_contact.ini, but it isn't loaded so it needs that hack to add the strings to api/language/en-GB/joomla.ini.

avatar richard67
richard67 - comment - 25 May 2020

Update: If I add a Factory::getLanguage()->load('com_contact', JPATH_SITE); to function _sendEmail of file api/components/com_contact/src/Controller/ContactController.php, the strings get translated in any case. @Hackwar What would be the roight place for that? Just at the top, or just after $app = $this->app;, or elsewhere?

avatar wilsonge
wilsonge - comment - 25 May 2020

_sendEmail of file api/components/com_contact/src/Controller/ContactController.php

Sounds an ok fix to me. Honestly moving that sendEmail part to a helper function shared by API and site rather than having duplicates in the controller of api and site might be best to avoid the duplication (and that would also have to load the lang file)

avatar richard67
richard67 - comment - 25 May 2020

Honestly moving that sendEmail part to a helper function shared by API and site rather than having duplicates in the controller of api and site might be best to avoid the duplication (and that would also have to load the lang file)

Agree ... but there is anyway potential regarding future PRs for improvement of the Mail Templates component, e.g. regarding HTML mode. Most of the issues are already covered by issues #28524 and #27619 .

But this PR should be ok now, I think. Will do a complete test again later.

P.S. And also potential of API improvements and simplifications, of course.

avatar wilsonge wilsonge - change - 25 May 2020
Status Pending Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2020-05-25 18:34:52
Closed_By wilsonge
avatar wilsonge wilsonge - close - 25 May 2020
avatar wilsonge wilsonge - merge - 25 May 2020
avatar wilsonge
wilsonge - comment - 25 May 2020

Merging this to get it through in time for beta this weekend. @richard67 if you still have time to report the results of your tests would be good

avatar richard67
richard67 - comment - 25 May 2020

@wilsonge @Hackwar Test were all ok with text emails. I've tested for each template without a template created for en-GB, then with template but default using language strings, and finally with own content. HTML templates don't work yet, as stated in issue #28524 , and so the "HTML mode" of the Mass Mail doesn't work, it always sends text mails (content type "text/plain"). Another issue #27619 also discusses the HTML templates.

avatar richard67
richard67 - comment - 25 May 2020

@wilsonge @Hackwar So it needs a decision about the check box for HTML mode in the Mass Mail form: Make it work or remove it from UI.

avatar richard67
richard67 - comment - 25 May 2020

@wilsonge @Hackwar I've created a new issue so the remaining mails to be done will not be forgotten: #29215 .

avatar infograf768
infograf768 - comment - 26 May 2020

Hmm
+COM_USERS_MAIL_REMINDER_DESC="This mail is sent to a user when by the \"Forgot your username?\" link e.g. in a login form."
when by ?

avatar richard67
richard67 - comment - 26 May 2020

@infograf768 Clearly a mistake, should be "by" only, without the "when". Could you make a PR?

avatar infograf768
infograf768 - comment - 26 May 2020

will do as well as another issue concerning com_mails

avatar richard67
richard67 - comment - 26 May 2020

Which other issue? If it is the mass mail html mode not working: I wanted to open an issue for it later.

avatar infograf768
infograf768 - comment - 26 May 2020

see #29224

avatar richard67
richard67 - comment - 26 May 2020

For issue #28524 with HTML body see my new PR #29229 .

Add a Comment

Login with GitHub to post a comment