User tests: Successful: Unsuccessful:
The default behaviour of the contacts component is to use the site's
name and email address as the sender of all contact email that gets
sent via com_contact forms with a reply-to set to the actual sender. If
we leave it like this, then antispam filters cannot be trained to
distinguish spam from regular email. The proposed change is to use the
actual sender's name and email address in all the email's headers (from
& reply-to).
Issue raised here: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=33849
Status | New | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2014-06-11 21:29:57 |
Labels |
Added:
?
?
|
Status | Closed | ⇒ | New |
Without this change, our site's contact form (without using any antispam system like reCaptcha) would receive spam messages with the site itself being the sender (it's the settings in either the component or the site). That means we could not flag incoming spam messages as spam, because we would essentially be marking ourselves as spammers.
If we change this on the other hand (which is what we've been doing for more than a year), then we can successfully report spammers into Gmail or other similar systems.
Are you sure you're not confusing what I wrote? We're talking about the "sender" that is being set for all outgoing email from contact forms in your Joomla site. This is bad when you, the site owner, know where you're waiting email from (your site's contact form).
Let me repeat, I'm talking about com_contact, not the other email sending functions in Joomla. So new accounts created will correctly have the site email address as the sender and so any other related functions...
Additionally, if this were to be implemented the your way (according to what you say), it should use the recipient info set in each com_contact form created and not the site's name and email address...
But again, the existing logic is wrong. When you receive an email from your site's contact form, that email should contain the actual sender's details and not your site's details. This worked as it should in Joomla 1.5. I assume it was brought into J2.5+ by accident...
I think what Beat means is that the sender of the email has to match the domain of the server sending the email. Otherwise the receiving email servers will reject the message as spam (because it looks faked to them).
In the end, with this change you would indeed be able to combat spam better in your client software, but it would be pointless because your email server would reject all messages already and you don't get any messages anymore.
When you receive an email from your site's contact form, that email should contain the actual sender's details and not your site's details.
The actual sender of the email is your server. Not the user who fills out the form. That's the point
Why would the receiving email server (aka YOUR server) reject the email? Did Joomla 1.0 & 1.5 work wrong for almost a decade? This is nuts.
The form should reflect the actual process: A person emails you, you see their details. Period.
If all the email you receive comes from you (your site) you can neither flag the email received as spam (if it's indeed spam) and therefore you cannot train your antispam filters, nor can you work right with email in plain tasks like sorting your emails by the bloody sender.
The actual sender is the person sending you the email, it's not your server. Your server is just a proxy in this process. If spam email is being sent with your name into a Gmail address, you will soon have a problem.
Like I said, did Joomla 1.5 and before worked wrong? @beat
Like I said, did Joomla 1.5 and before worked wrong?
Spam raised a lot since the days of 1.0 and 1.5 and server improved their spam detecting techniques. What worked back then, doesn't necessary still work today.
This doesn't prove your point though. And of course it works still, we operate some very high traffic websites under J1.5 with no problem at all. The problem exists for J2.5+.
What about an OPTION/TOGGLE in configuration? Just allow to switch this behavior, so anyone can use what suits best for their site. I think both ways makes sense, and are personal preference:
@fevangelou I do understand your point of view, but unfortunately the standards have gone the opposite way since 1.0 and 1.5 times, and that's why Joomla has been adapted to respect the new standards.
Following IETF standards are now key for sending deliverable emails :
1. SPF: http://www.ietf.org/rfc/rfc4408.txt and http://www.ietf.org/rfc/rfc6652.txt
2. DKIM: http://www.ietf.org/rfc/rfc6376.txt
3. DMARC: http://www.dmarc.org/ , and proposed IETF standard: https://datatracker.ietf.org/doc/draft-kucherawy-dmarc-base/
Any serious emailer (and that includes Joomla's contact form) should respect IETF standards.
At Joomlapolis.com, we are sending out well over a million emails per month from all our mail servers and as such we are classified as a massive non-spamming email sender, but for that we had to go through make sure that all our infrastructure perfectly respects all of the above standards (and many more mass email-hosters requirements) to achieve a decent deliverability.
Check at how any mass sender does it (FB, Twitter, Google, Yahoo, and so on): They all implement SPF and DKIM and have DMARC DNS-records too, thus emails sent with their "From" domain but from non-authorized servers end up in spamboxes or even being completely refused.
There are many "good email server practices" white-papers on the web, and they all say similar things: Don't try to send "From" a non-authorized server (in addition of configuring your SPF, DKIM and DMARC records, which all major email services providers did. As those alone cover over 50% of email addresses worldwide, it's well embraced standards.
TL;DR: Times where any mail server could fake being allowed to send emails for any other email-domain are gone. And actually, if you think about it, it's a good thing to be able to verify the sending server's authorization to send that mail for the "From" domain. That alone combats a good part of spams.
Best Regards,
Beat
@pepperstreet : You can still reply to the sender directly, as he is in the "Reply-To" header line. And all email clients reply to "Reply-to" and not to "From" field, if "Reply-To" is present.
TL;DR : Joomla 2.5 and 3.x send correct email headers, and allow reply to the contact-form sender directly. No need for change for that.
Moreover @beat all the protocols you refer too are down to DNS. What does that have to do with the freaking sender's name & email?
Fixing this resulted in reduced spam for us. Leaving it ad it was increases spam. What is it that you don't understand?
The fact that you send a gazillion email newsletters (for which I've been told delivery to Gmail accounts is always in the spam folder) doesn't prove your point.
I'm talking about contact forms, you talk about newsletters. Totally different things.
Furthermore, the reason big sites use services like Mandrill or SendGrid as their SMTP is because these services have special agreements with ISPs all around the world to ensure proper delivery of emails sent via these services. Every big site does that. Otherwise they would just follow the protocols and specs and have 0 emails delivered...
Apples and oranges.
Hi Fotis (btw i hate those impersonal @fevangelou that github makes us use),
It's really not personal, and i'm fully on your side that it doesn't look natural to mere humans. And yes, I'm often frustrated too to discover that a great or simple idea i had doesn't withstand some established standards.
And plesae don't try personal attacks like above. We met at conferences, and we both know that both of us are not "bad" people. Thanks!
Regarding the RFCs, it's more than just DNS, even if the implementation-side for the mailserver-administrator is at DNS level too.
TL;DR: All well-implemented mailservers with decent anti-spam are nowadays looking up the DNS server of the "From" email address domain for SPF, DKIM and DMARC records. Then they check the email headers and originating IP addresses to be compliant with those authorative email-related DNS-records for the emailing "From" address. Thus, they are really relevant for this case.
Regarding the other questionings from your reply:
Both contact-forms emails and any other emails (not only newsletters btw) are emails, so they have most in common.
Now your own issue might be that your anti-spam software is configured to not look at spam-nature "whitelisted" "From"-addresses, and that you should just remove that "From" address of your server from your anti-spam white-list ?
Just trying to help Joomla and you, nothing else. Really.
With warm and sincerely friendly Regards,
Beat
Our email platform just so happens to be Google Apps. With Joomla as is we get filled with spam. With Joomla and the simple one liner fix, spam is close to zero because we can train Gmail to distinguish spam senders.
Postfix works in both cases sending the emails.
Google Apps (Gmail) is the receiver.
What do you make out of that?
Did you try another contact form component? Do they different?
Since I don't see common sense prevailing here, we'll probably built our own (as we did with K2)...
Please don't attack people here, it's not needed at all.
Before writing your own component, you could also have a look at http://extensions.joomla.org/extensions/contacts-and-feedback/contact-forms.
I think if you would do it as an option where the user is able to change the behavior, then it could be discussed. But as it is now there is no chance this gets in. I'm going to close this PR.
Feel free to open a new one if you decide to do it as a parameter.
Status | New | ⇒ | Closed |
Closed_Date | 2014-06-11 21:29:57 | ⇒ | 2014-06-12 19:33:15 |
Title |
|
Fotis got a point here. We have over 150 demos on our server,
visitors are using contact forms to test the demos , we cant block those emails since the sender is us. In my opinion the email sender should be the email of the person WHO is SENDING email trough MY email form, not my website email.
Example: Joomla install email and default site email.
and my contact form receiver email is me@mydomain.com
Visitor sending email: my_email_IS_SENDER@otherdomain.com
http://prntscr.com/3sc885
So sender should be my_email_IS_SENDER@otherdomain.com
Agree to do this as parameter.
Yes , me to , param would be great
Imho, this should be a global switch for com_contacts. If the mods agree, I can move on and implement this simple yet important feature.
But what will the default state be? I'd obviously recommend the new state.
Default state , new state ,
+1 for Fotis suggestion
new state
Default has to be current behaviour, otherwise you change something for existing sites which we don't accept.
I would add the parameter to the global settings in the form tab, plus also to the menu item and contact params.
Maybe one could even do it so you can have the choices to use site email
(default), contact email
or visitor email
as from address?
Yep, you can do so in the install.sql file if you want. However it would mean that the default values in the code and xml forms and the one set during installation would be different. That's usually not nice.
And honestly, I'm not sure if I would agree with that default setting myself. Beat almost certainly will not.
"Default has to be current behaviour, otherwise you change something for existing sites which we don't accept."
@Bakual No offense, but Joomla has a grand history in breaking things, important things. This worked differently in Joomla 1.0 & 1.5 and was abruptly changed in 1.6+. Lots of things break inbetween releases for Joomla. So many we'll need another Github repo to count them.
@dgt41 The PHP code can set a default value even if the setting is not changed anywhere in the forms, so there is no need to edit the SQL file (there is no such value in there anyway)...
@fevangelou right now there's no such value, but, I guess, if you implement this with a param somewhere (com_contacts) then there will be a value in the database. I just mention this because there was another similar situation with the prepare content option of custom html module...
No offense, but Joomla has a grand history in breaking things, important things.
That's why the development strategy changed this year, with a big emphasis on not breaking stuff. Let's not break it intentionally.
The PHP code can set a default value even if the setting is not changed anywhere in the forms, so there is no need to edit the SQL file (there is no such value in there anyway)...
The default value in the PHP code has to match the one in the XML files. Otherwise the behavior suddenly changes after the user saves the options (without actually changing something).
That's why you would have to use SQL files to either set the value during the update or the installation, depending on how you want to change the default values. It's a tricky thing to do and I would not recommend it if not really needed.
+1 Default should be former behaviour, always.
Can the PR be updated with the proposed setting so it can be tested? Thanks.
Actually, I still consider the current approach bad. Just try replying to a message you receive from a contact page to your Gmail account. Gmail will falsely identify you (the site's email = the main email sender according to Joomla) and the reply you write will (naturally) be sent to yourself. You need to manually delete the preselected response email address and copy/past the actual email address of the person who used your site's contact form to get in touch with you.
Actually, I still consider the current approach bad.
Quoting myself from earlier in this discussion:
I think if you would do it as an option where the user is able to change the behavior, then it could be discussed. ... Feel free to open a new one if you decide to do it as a parameter.
and
Default has to be current behaviour, otherwise you change something for existing sites which we don't accept.
This issue was discussed many years ago on one of the mailing lists. Sorry can't find the link just now.
The issue only occurs iirc with gmail and ONLY when the site email (set in configuration.php) and the contact email are the same. If you set the site email to something like site@example.com and the contact email as name@example.com then the problem does not occur. (thats what I've been doing for years with no problem)
@fevangelou wrote:
Actually, I still consider the current approach bad. Just try replying to a message you receive from a contact page to your Gmail account. Gmail will falsely identify you....
Hi Fotis,
As you correctly state, it is a bug in Gmail, which is not respecting the Internet Email specification RFC5322.
Did you submit a bug-report to Google ?
Here is a howto describing how to report Gmail bugs to Google:
http://email.about.com/od/gmailtips/qt/How-To-Report-A-Gmail-Bug.htm
There is also a public Gmail product forum here that is also a second bugs-report enty for google:
https://productforums.google.com/forum/#!categories/gmail
Thanks to the current settings I wake up to 300,000 bulk emails daily ( I’d attach a screenshot if I could) Not to mention server burning through storage and all other sites being impacted... when is this going to get adopted already? 4yrs and counting
Those running newsletters are the only ones benefiting now and they can make a manual change when needed - or maybe have a toggle?
Idk. What’s there now just doesn’t work.
Seriously
Hi Fotis,
Thanks for the idea, but I think that change would be an issue as it would not implement correctly the IETF SMTP RFPs for SPF and DKIM.
And as over 50% of email addresses today (e.g. gmail, yahoo, microsoft) are protected by SPF and DKIM records, and sending a mail "From" that address on a (non-SMTP-sending authorized by SPF and DKIM records SMTP server of the) Joomla site would end up being rejected by all mail-providers checking for SPF and DKIM validity of sending SMTP server.
So Joomla, as is, is implementing it correctly, with "From" being the valid email of the site (which must be (but also can be) configured correctly) and the "Reply-to" being the enquirer.
Doing such a change would be defeating all anti-spam email measures being put in place.
Imho, you sure don't wan't your site's contact form to land into a spam folder. Anti-spam measures should be taken at the contact form (e.g. captcha & co), not at the email server for contact form.