J3 Issue ? ?
avatar mbabker
mbabker
2 May 2018

I'm about to ask a very loaded and very blunt question, primarily directed toward project leadership but all feedback is welcome. Is there interest (as in should this be a roadmap item for a short term release schedule) in having a "proper" tool suite and API in core to help facilitate privacy (GDPR) related items? In essence this means having some high level APIs in the Joomla\CMS\Privacy namespace and a com_privacy at a minimum with some baseline functionality and a way for site owners to interface with said API.

Why does core need this functionality? For the most part, core itself works with a very limited set of data that might fall under the scope of privacy related laws such as GDPR, however it acts as the framework/facilitator for thousands of extensions to perform actions which do fall under that scope, and I do believe that we as that framework owe it to our userbase to at a minimum do the due diligence to either provide a set of tools to facilitate these types of activities in a consistent manner or to tell users that the core project does not feel it is its responsibility to offer these tools and they should be provided by the ecosystem.

Now, here is my offer if this is indeed something the project will accept and support with resources (that means I need you, casual reader, to assist with this effort; I will not do the entire project myself). Internally a discussion started about having a 3.9 release with privacy/GDPR related functionality added to core based on the many PRs that have already been opened and I offered to coordinate that release; that offer still stands. As it relates to this effort, I will offer to team lead, project manage, or whatever you want to call it, to steer the efforts in a coordinated manner so that we can have a releasable product in a reasonable timeframe (clearly we're already a few months too late to starting this work, so anyone expecting a finished product before May 25 has unrealistic expectations unless you're going to pay me full time to work on this, just being blunt about it).

In return, as I hinted at above, I need commitments from interested parties in working on this tool suite. Coders to help write the code, testers to help make sure we aren't building an unusable mess, writers to help document all the things, etc. etc. I will not take this project on by myself, I already do way too much with too little time and making my time offer here is stretching my available resources for other project tasks.

If agreed upon in principle, we can have a space set up relatively quickly and I will outline specifics of what I would consider the MVP for this effort to be at that time.

avatar mbabker mbabker - open - 2 May 2018
avatar joomla-cms-bot joomla-cms-bot - change - 2 May 2018
Labels Added: ? ?
avatar joomla-cms-bot joomla-cms-bot - labeled - 2 May 2018
avatar joomla-cms-bot joomla-cms-bot - labeled - 2 May 2018
avatar brianteeman
brianteeman - comment - 2 May 2018

i agree in principle - obviously - but i will absolutely not waste another minute of my time on this if it is going to face continual pushback

avatar nibra
nibra - comment - 2 May 2018

+1
Count me in.

avatar mbabker
mbabker - comment - 2 May 2018

i will absolutely not waste another minute of my time on this if it is going to face continual pushback

Me either. That's why before I put in any effort beyond this item I am basically looking for a "we support this feature in principle" response; I'm not wasting my own or anyone else's time on a feature that isn't going to ship this year and isn't supported by leadership. At this point all I'm asking for is the support and the commitment from other teams and individuals to support this effort as it progresses (dev and getting a release milestone accomplished), as I've made perfectly clear here I'm putting my hat in the ring to do a lot of the heavy lifting as it relates to coordination and release efforts.

avatar Sandra97
Sandra97 - comment - 2 May 2018

I support this idea. I'll do what I'm able to do to help.

avatar parthlawate
parthlawate - comment - 2 May 2018

As I've said on Glip I support this wholeheartedly.. and would also go ahead and contribute developer time from the Tekdi and Techjoomla team.

avatar brianteeman
brianteeman - comment - 2 May 2018

it doesnt matter what we say. all that matters is that those whose voice matters agree. so far they have either been against adding data privacy code into joomla or have been completely awol.

presumably it is not a big job to do as we already have all the code created for joomla.org

avatar ot2sen
ot2sen - comment - 2 May 2018

Will test and provide feedback.
Actually think it would be a good idea of finding a bag of gold coins to speed this up would be well spent money by the project.

avatar brianteeman
brianteeman - comment - 2 May 2018

@ot2sen the bag of coins was spent on the code created for joomla.org

avatar mbabker
mbabker - comment - 2 May 2018

Someone in the compliance team has also pointed out the .org code as being a solution that can be built on. If they (or anyone who has already created similar code) are willing to share (the compliance team seems to be on board with the sharing piece) then that does help us move things along. Being so late in the game it's going to be helpful if we can consume/use existing code, but that's a detail I'd like to focus on after getting project approval instead of now.

avatar jeckodevelopment
jeckodevelopment - comment - 2 May 2018

Count me in. I'll do what I'm able to do to help.

avatar tonypartridge
tonypartridge - comment - 2 May 2018

+1 and @GeraintEdwards might be able to help too.

avatar aDaneInSpain
aDaneInSpain - comment - 2 May 2018

I am very much for this. Especially if it includes some kind of way for 3rd party developers to highlight/manage their personal data as outlined in my issue.

avatar mbabker
mbabker - comment - 2 May 2018

Something like that is already on my mind. It HAS to be pluggable/extensible.

avatar jomres
jomres - comment - 2 May 2018

I'm currently implementing the "download your personal info" side of things for Jomres. It would be really nice to receive a call from J telling Jomres that A the user's requested a download or B the user's requested anonymisation. Right now it has to be done within Jomres and then potentially either I'll have to anonymise the J core users table, or just ignore it and hope that the site admins direct users to another tool for that. Working the other way around would be much more helpful, so receive from J > Anon my data (of which, you can imagine, a booking engine has quite a lot) > wave cheerio to the guest.

avatar coolbung
coolbung - comment - 2 May 2018

We're in. In fact we'd begun some design brainstorms around the architecture of this. A wiki of the brainstorm is available here - https://github.com/techjoomla/user-deboarding/wiki

Some high level items :

  • The core provides an API to store named consents
  • If a user decides to revoke a consent the core triggers an event
  • Each extension implements a method to delete/anonymise data after user deletion.
  • Each extension may implement an API to generate an indexed export (this facilitates a google takeout like service)
avatar davies401
davies401 - comment - 2 May 2018

100% Yes please. It would make more sense than having to find a GDPR component for each site

avatar JoomliC
JoomliC - comment - 2 May 2018

+1
Count me in ! (test + code + 3rd party)

Thanks @brianteeman and helpers for inspiring work done here: #20051

avatar dam-man
dam-man - comment - 2 May 2018

@coolbung what needs to be done. As there no code attached yet in the repo. Does this become part of the J core?

avatar Webdongle
Webdongle - comment - 2 May 2018

@mbabker

Internally a discussion started about having a 3.9 release with privacy/GDPR related functionality added to core

Great idea hope it works

avatar Ruud68
Ruud68 - comment - 2 May 2018

This is important for Joomla so count me in; testing, troubleshooting, development work.

avatar crystalenka
crystalenka - comment - 2 May 2018

I don’t have a lot of time to commit to this but would be happy to provide UX-oriented feedback when asked.

avatar srseale
srseale - comment - 2 May 2018

I have been looking for some way to contribute for some time. This sounds like a good thing. I can write documentation, test and I am even willing to help with the code. I'm a newbie at this, totally unfamiliar with Github and this process but I'm willing to learn...
Count me in.

avatar mbabker
mbabker - comment - 3 May 2018

After talking this over with @wilsonge and based on his last comment in #20140 we are moving forward with this.

I've set up a repo at https://github.com/joomla-projects/privacy-framework which has the high level items identified, most of which I would consider needing to be completed to have a MVP for merging back to core. Hopefully what's there so far makes some sense, if not feel free to ask questions.

PRs like #20051 or #20173 are probably good candidates to move into that repo so we can make sure they fit into the overall goals and workflow that's being created.

avatar franz-wohlkoenig franz-wohlkoenig - change - 3 May 2018
Status New Discussion
avatar franz-wohlkoenig franz-wohlkoenig - change - 3 May 2018
Category Feature Request
avatar WilkoRietveld
WilkoRietveld - comment - 3 May 2018

Great initiative, I am available for testing!

avatar rameshelamathi
rameshelamathi - comment - 3 May 2018

+1
Good move. Totally agreed. Count me in

avatar sakiss
sakiss - comment - 5 May 2018

Where can i find specs for that project?

avatar schultz-it-solutions
schultz-it-solutions - comment - 7 May 2018

I am ready to give some (limited) manpower resources into this project as well!

avatar mbabker
mbabker - comment - 13 May 2018

So for those with some time to help, we've got some PRs that could really use some testing and feedback (especially on the UI/UX aspect of things where I've coded the functionality, lets just say that's a little rough)

avatar brianteeman
brianteeman - comment - 14 May 2018

whatever happened to using the code generated by the compliance team?

avatar mbabker
mbabker - comment - 14 May 2018

Was informed what they're building is very specifically targeted at joomla.org requirements and not suitable for mass distro.

avatar ot2sen
ot2sen - comment - 14 May 2018

@mbabker How best to test the PRs? Will the patch tester work in a branch or I need to dust off how to manually add them? Would like to help testing tomorrow, just a bit of brain fog after a day at work ;)

avatar mbabker
mbabker - comment - 14 May 2018

Probably best downloading the ZIP that GitHub builds and installing from that. Trying to use patch tester against a clean install from this repo's staging branch is going to give you a broken environment.

avatar schultz-it-solutions
schultz-it-solutions - comment - 14 May 2018

ot2sen: today I just figured it out the way mbabker suggests, and was successful...

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 15 May 2018

@schultz-it-solutions you figured out how to test but haven't test sucecessfully, right?

avatar f-hamel
f-hamel - comment - 15 May 2018

Maybe we also need to anonymise the ip-address in case of logging.
Like in /administrator/logs/error.php. Where the clientip of failed logins is stored,

avatar schultz-it-solutions
schultz-it-solutions - comment - 15 May 2018

@franz-wohlkoenig: yes of course "only how to"... sorry, that was ambiguous

avatar brianteeman
brianteeman - comment - 15 May 2018

See discussion elsewhere on this tracker about why you don't need to nor
should we anonynise IP addresses

On Tue, 15 May 2018, 07:09 Ruediger Schultz, notifications@github.com
wrote:

@franz-wohlkoenig https://github.com/franz-wohlkoenig: yes of course
"only how to"... sorry, that was ambiguous


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#20281 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPH8SZw0GpOnY8J5lOTmlQVBxkNU7vRks5tynE0gaJpZM4TvaqT
.

avatar f-hamel
f-hamel - comment - 15 May 2018

@brianteeman Maybe I misunderstand you.

But I think we really should discuss about this here.
Because the European Court of Justice decided, in C582/14 that IP addresses are personal data because they can be related with an natural person.
Further ecj said collecting and using personal data relating to a user are only allowed
"so far as that the collection and use of that data are necessary to facilitate and charge for the specific use of those services by that user"

So where is a need to anonynise ip addresses and maybe the Privacy Tool Suite is the right place for this.

So if I'm wrong with that, it's ok. This should only be a hint.

avatar brianteeman
brianteeman - comment - 15 May 2018

No you misunderstand that court decision. As explained in detail here https://www.whitecase.com/publications/alert/court-confirms-ip-addresses-are-personal-data-some-cases the court said that it "can be personal data" in some circumstances.

The CJEU decided that a dynamic IP address will be personal data in the hands of a website operator if:

  1. there is another party (such as an ISP) that can link the dynamic IP address to the identity of an individual; and
  2. the website operator has a "legal means" of obtaining access to the information held by the ISP in order to identify the individual.

On the facts, if the BRD has the legal power to compel the relevant ISP to disclose sufficient information to identify Mr Breyer, then Mr Breyer's IP address will be personal data in the hands of the BRD.

As a regular site owner does not have that legal power then it is not personal data

avatar Webdongle
Webdongle - comment - 15 May 2018

As a regular site owner does not have that legal power then it is not personal data

In addition the Host server would still have the records ... so a regular site owner would not be destroying evidence.

avatar f-hamel
f-hamel - comment - 15 May 2018

Thanks for sharing this @brianteeman
I guess we have a grey area here, like we have many of them in GDPR.
And there are lots of debates going on about if ip addresses are personal data.
IMO they are.
Maybe in the close future we will get clear decision by the European court.

I see there is an issue joomla-projects/privacy-framework#16.
Log rotating withe clearing them would be great solution.

In addition the Host server would still have the records ... so a regular site owner would not be destroying evidence.

@Webdongle As I'm working for a German hosing provider, I can tell you we have to anonymize IP addresses for the access.log, and we will do so. Also we will rotate apache logs away after a few days.
Just because German law and the implement of the GDPR in Germany.
So we no evidence if someone need them, e.g. in case of an attack. So we have big controversy debate about the retention of data and GDPR.

Maybe we get a log rotate to Joomla, which would help to solve the problem that ip addresses are stored a long period and with this unnecessarily.

But I think this discussion will go to far here.
So for me it's no more a part of this issue.

avatar tonypartridge
tonypartridge - comment - 15 May 2018

@brianteeman already is looking at log rotation.

But an IP is only personal data when an IP is associated with personal data. IP’s can be changed, can be multiple so it’s not that persons data as they do not own it 9.9/10 times.

On 15 May 2018, 17:08 +0300, Felix Hamel notifications@github.com, wrote:

Thanks for sharing this @brianteeman
I guess we have a grey area here, like we have many of them in GDPR.
And there are lots of debates going on about if ip addresses are personal data.
IMO they are.
Maybe in the close future we will get clear decision by the European court.
I see there is an issue joomla-projects/privacy-framework#16.
Log rotating withe clearing them would be great solution.
@Webdongle As I'm working for the German hosing provider, I can tell you we have to anonymize IP addresses for the access.log, an we will do so. Also we will rotate apache logs away after a few days.
Just because German law and the implement of the GDPR in Germany.
So we no evidence if someone need them, e.g. in case of an attack. So we have big controversy debate about the retention of data and GDPR.
Maybe we get a log rotate to Joomla, which would help to solve the problem that ip addresses are stored a long period and with this unnecessarily.
But I think this discussion will go to far here.
So for me it's no more a part of this issue.

You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

avatar mbabker
mbabker - comment - 15 May 2018

As pointed out in the discussions on the privacy-framework repo, there are legitimate use cases for processing IP addresses that would trump any personal data consent or "right to be forgotten" type of requests, so Joomla can't just blindly anonymize them. What we can do is provide tools for things such as log rotation to ensure logs are deleted in a timely manner, and review core's logging to ensure when IP addresses are logged there is a valid reason to do so. And to me that is the extent that our efforts need to go to, anything further creates too many problems.

avatar schultz-it-solutions
schultz-it-solutions - comment - 15 May 2018

Agree with mbabker 100%.

avatar tonypartridge
tonypartridge - comment - 15 May 2018

Agree too!

On 15 May 2018, 17:39 +0300, Ruediger Schultz notifications@github.com, wrote:

Agree with mbabker 100%.

You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

avatar f-hamel
f-hamel - comment - 15 May 2018

So with what @mbabker wrote I agree too.
Saving ip addresses for legitimate use cases in short period, is fine.
A start writing my first post in this issue did not saw the discussion in the privacy-framework repo.

avatar Wildfigmedia
Wildfigmedia - comment - 25 May 2018

Happy to help with testing and reporting

avatar gwsdesk
gwsdesk - comment - 26 May 2018

With GDPR we need a consent option on all forms on a website. This includes the Joomla contact-us forms for instance notwithstanding registration etc. We should not start the entire discussion on GDPR here again (who is right and who is wrong) That makes no sense. We need basic consent/denial/etc options implemented and we are late (no offense) since this is known for a long time. So we need to do urgently something. If you use RSforms/Breezingforms etc you can easy add a consent and your privacy policy (Joomla content) can be adjusted.... You also can use a plugin that asks for a consent/denial/info many are available

However I believe we need to address this globally as Michael mentioned? Do I miss something?

Leo 8)


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/20281.
avatar brianteeman
brianteeman - comment - 26 May 2018

yes you have missed that this work is under way

avatar gwsdesk
gwsdesk - comment - 26 May 2018

Tnx Brian much appreciated. We have a link to "work in progress"?

avatar mbabker
mbabker - comment - 26 May 2018

The same https://github.com/joomla-projects/privacy-framework repo that's been linked to a few times in this issue over the last few weeks.

avatar brianteeman brianteeman - change - 28 May 2018
Labels Added: J3 Issue
avatar brianteeman brianteeman - labeled - 28 May 2018
avatar studio42
studio42 - comment - 31 May 2018

Does now a core solution exist ?
I need it for acy mailing, chronoform, Joomla forms, user connect, Virtuemart.... in same website
I dont want add a plugin for each case, because i dont want that a user have to confirm in each form but only one time if the user is connected for eg.
And this is only for 1 website, i have some website using more forms.
What is the current solution/plugin/component that can handel all case ?
Regards

avatar mbabker
mbabker - comment - 31 May 2018

It is still a work in progress as indicated by the activity on the https://github.com/joomla-projects/privacy-framework repo.

avatar Quy
Quy - comment - 20 Jun 2018

See PR #20800

avatar brianteeman brianteeman - change - 20 Jun 2018
Status Discussion Closed
Closed_Date 0000-00-00 00:00:00 2018-06-20 02:26:15
Closed_By brianteeman
avatar brianteeman brianteeman - close - 20 Jun 2018
avatar brianteeman
brianteeman - comment - 5 Jul 2018

A lot of people here wanted to see the privacy component in Joomla. We worked hard to create it - now its your turn. You volunteered to help test - without those tests it will not be released. Find out how here #20800

avatar brianteeman
brianteeman - comment - 30 Jul 2018

Seriously? There were loads of people that said they wanted this feature and stood up to be testers and yet hardly any of them have bothered. Maybe we should just drop this. It's a major new feature and if its not widely tested then I would be very reluctant to add it.

avatar Wildfigmedia
Wildfigmedia - comment - 31 Jul 2018

Sorry guys, Been on some much needed leave. On this now

avatar Ruud68
Ruud68 - comment - 19 Sep 2018

Hi, not sure if this is the correct thread to start a discussion, if not please point to the correct place :)

I am currently testing 3.9-beta2 (nightly) on my acceptance environment: Complements on everybody involved, looks great!

One thing I noticed is that all privacy related components (com_privacy, com_actionlogs) do not have ACL permissions implemented but are (hard) limited to super users only.

Now I am maintaining some Joomla sites for 'larger' organizations. There is a strict segregation of duties as to what a super user can do and a manager / administrator: this is mainly based on technical capabilities (install / update components, etc.)

Furthermore these organizations most likely have an employee in their mids who is responsible for privacy related things (e.g. a DPO: data protection officer).

Now because the current privacy implementation is based on the rule: super user only, this requires a 'mix' of a functional and technical maintenance... This is to limiting.

What I would expect is that com_privacy, com_actionlogs has an ACL permission implementation so I (as hired technical super user) can 'delegate' the functional privacy role to the person responsible for this in the organization without giving this user super user rights....

Does this make sense?

avatar mbabker
mbabker - comment - 19 Sep 2018

It was purposefully decided to restrict these components to super users only. In the case of a privacy request, when exporting data the person processing the request inherently must have (or gains through an information leak) read access to all of the data that plugins are responding with. When removing data, the person processing the request must inherently have (or gains by bypassing ACL) what is in essence a combination of core.edit and core.delete permissions. And for action logs, someone could see a user performing actions in a component they may not even be aware is installed because ACL restricts them from even seeing it.

If we made it so non super users were able to access these components, inherently we are either dealing with some form of ACL violation in other parts of the system or the processed data must be ACL restricted to that user's role, resulting in an incomplete action log or data export.

avatar micker
micker - comment - 19 Sep 2018

i am not sure that only for super user ... in RGPD, DPO is a person who need to check this, it's not webmaster of site, its a security responsability... i think we need to allow this ability to a specific user or group

avatar mbabker
mbabker - comment - 19 Sep 2018

If someone can spec something out in a way that does not come across as an ACL bypass security issue we'll review it. It is not that we necessarily want to keep these areas super user only, but when you consider what the components are designed to do, you also must keep in mind that the DPO person essentially has to have a combination of core.manage, core.edit, core.edit.state, and core.delete permissions to adequately handle requests and that through the privacy component the person processing a delete request will more than likely bypass any "regular" ACL checks for the edit and delete permissions that would be done on a normal save or delete action.

At the end of it all, we still need to be mindful of the site's ACL restrictions and that what the last comments here are proposing would in effect act as a ACL bypass through the privacy component interface; it is entirely possible through what you both are requesting that a DPO would be given read or edit access to an item they would not normally otherwise have and I'm not sure that is an acceptable risk for core.

Add a Comment

Login with GitHub to post a comment