Simplenews and the EU General Data Protection Regulation (GDPR)

We have been building a new module to integrate GDPR requirements with Drupal’s Simplenews module in a bid to make Simplenews subscriptions compliant. In this post I’ll outline some of the achievements so far - and also some of the remaining challenges ahead...

Even though the European Union’s General Data Protection Regulation (GDPR – more info at was approved on 27 April 2016 [1], it seems few people in the Drupal community (including us) took much notice until April 2018, with the regulation coming into force on 25 May 2018. Unsurprisingly, we have been a bit busy ever since.

Some of the key changes brought in by the regulation are:

  • Consent for the processing of data must be explicitly given, and the purpose for which the data is being used must be clearly stated.
  • You must be able to prove that consent has been given, and what a person consented to.
  • Each person has a right to access the data held about them, meaning you will need to be able to provide them with a copy of the data you hold.
  • The regulation also establishes the “right to be forgotten”, so you must be able to also delete any data you hold about someone.

There are additional requirements in the case of data breaches and also about privacy by design.

This regulation has some impact on almost every website, for example user accounts (login names and email addresses constitute personal data), or – the topic of this blog post – newsletter subscriber data.

For Drupal, development of a module facilitating GDPR compliance began in January 2017 ( The GDPR module – in its Drupal 8 version – includes several components helping a web developer to comply with the GDPR:

  • A GDPR checklist module, which assists web developers to comply with the GDPR (but doesn’t actually do it for you).
  • A GDPR Consent module, which helps you track consent based on user accounts and consent agreements.
  • Drush commands which help to anonymise data when exporting the database.

Other submodules are still planned – such as GDPR Tasks.

While this module provides some tools and infrastructure for GDPR compliance, it is really focused on Drupal user accounts.

As we use Simplenews on several websites we developed, we started thinking about this when an issue about GDPR compliance of Simplenews was opened on (a similar issue has been opened for Mailchimp, which is also often used, but as Mailchimp does not save subscriber data in Drupal, this needs to be handled differently).

When we started to look into how to address the requirements of GDPR for Simplenews subscribers, we thought that using the GDPR module might be a good way to go, enabling us to make use of the existing functions provided by this module. The functionality we were interested in was:

  • The management of consent agreements.
  • The ConsentResolver infrastructure (which in GDPR Consent is limited to Drupal users)

As a first step we implemented a ConsentUserResolver for the simplenews_subscriber entity. That was reasonably simple, but has an important limitation: you can only use it if all your Simplenews subscribers are also Drupal users, which in most cases is simply not true.

That meant more work. The good news is that Simplenews subscribers are Drupal entities, as are users. So we adapted the code of the ConsentUserResolver (a plugin manager, plugin, field, widget, etc) to create a ConsentSubscriberResolver which allows the tracking of the agreements (and any revision of an agreement) a subscriber has consented to.

Some problems had to be solved along the way. For example, when someone fills in the subscription form, the subscriber entity does not yet exist. That’s created later, and updated when a subscriber confirms their subscription. In the present implementation, we update the subscriber entity and relate it to the consent agreement when a subscription is confirmed. This also means we won’t have consent agreements for subscriptions that have never been confirmed (bot subscriptions) – which is probably a good thing.

The work is presently available as a Sandbox project at The aim is to get it into the main Simplenews module as a submodule, but a bit more work is required to achieve this, such as adding unit tests.

1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data,

Is the internet sustainable? Guess what? No it isn’t.

How many lightbulbs?

We often think of “the internet” as a “virtual world”, somewhere in “the cloud”. Want to know something? Quickly use your search engine of choice to find out. Or browse Wikipedia. Need something? Just shop online, and it will be delivered to your doorstep (it should be obvious that this last step – delivering an item – clearly has environmental implications, and many cities are now struggling with the number of delivery vans). Video streaming, computer gaming, and everything else – all massive money making uses of “the internet” and all supposedly making our lives easier and more fun.

In addition to influencing and changing consumer behaviour, “big data” is the buzz phrase for all kinds of enterprises. Big data in the cloud, conveniently processed and delivered in consumable chunks for all sorts of purposes. All happening in a “virtual reality” and sold, conceptually, as a benefit of the real world we live in.

What gets lost in this rosy image is that “the internet” depends on a very material infrastructure, which is not without energetic and environmental impact.

According to Google’s own figures, the average Google search uses about 0.0003 kWh of energy, equivalent to about 0.2 grams of CO2. That does not sound much. It is the equivalent of turning on a 60W lightbulb for 18 seconds. But then – according to Google more than 2 trillion searches (1,000,000,000,000) are performed every year. So your lightbulb could be on for more than a million years! What sounds incredibly small initially turns into something of considerable impact given the trillions of times it is being performed.

Climate Change News reported in December 2017: “Global computing power demand from internet-connected devices, high resolution video streaming, emails, surveillance cameras and a new generation of smart TVs is increasing 20% a year, consuming roughly 3-5% of the world’s electricity in 2015, says Swedish researcher Anders Andrae.

In an update to a 2016 peer-reviewed study, Andrae found that without dramatic increases in efficiency, the ICT industry could use 20% of all electricity and emit up to 5.5% of the world’s carbon emissions by 2025. This would be more than any country except the US, China and India.

Others estimated at even higher energy consumption. According to a report by Greenpeace, “The energy footprint of the IT sector is already estimated to consume approximately 7% of global electricity. With an anticipated threefold increase in global internet traffic by 2020, the internet’s energy footprint is expected to rise further, fuelled both by our individual consumption of data and by the spread of the digital age to more of the world’s population, from 3 billion to over 4 billion globally.

The internet boom and “big data” is not just something happening “in the cloud”. It is having a real impact on our real environment. A good example is Ireland, a country popular with multinational companies looking for a place to build new data centres. Plans by Amazon, which already has several data centres in and around Dublin, to build a new 20,739 sq m (223,000 sq ft) data centre near Dublin raised concerns about its potential energy use: if fully realised to potential this one data centre alone would use 4.4% of Irelands projected electricity consumption by 2026. And Amazon is not alone. Google, Facebook, Apple and Microsoft have all announced significant data centre investments for Ireland. According to projections by EirGrid, Ireland’s grid operator, 15% of all electricity demand in Ireland will come from data centres by 2026.

Even though new energy production capacity in Ireland is 100% renewable, the increase in energy demand will mean that Ireland misses it’s EU renewable target of 16% of total energy needs from renewable sources by 2020, leading to an annual fine of 75 million €.

Crypto currencies and blockchain – energy consumption going through the roof

If all this was not bad enough, the present craze for crypto currencies – and more importantly, the future promotion of the underlying blockchain technology – will increase energy consumption even more. According to Digicomonist, one single Bitcoin transaction requires 427kWh of electricity and produces 209.28 kg of CO2. A typical European home uses about 3600kWh of electricity annually. This means your one Bitcoin transaction (to buy what and of what value?) uses as much electricity as the average European household in 43 days! A Nepalese household could be powered for more than 1 year (487 days) by the energy used in a single Bitcoin transaction.

The problem is not Bitcoin specifically (Ethereum, another popular crypto currency, also has a high energy use for each transaction, though considerably lower than Bitcoin), but the way Blockchain, the underlying technology, works. Mike Orcutt explains this in a recent piece in MIT Technology Review: "What brings cryptocurrencies like Bitcoin and Ethereum to life is the way all the computers in their networks agree, over and over, that what a blockchain says is true. To do this, they use an algorithm called a consensus mechanism. You’ve probably heard it called “mining.” And that’s the process guzzling energy, and huge amounts of it. Even though there are efforts under way to develop other, less energy guzzling, “consensus mechanisms” for blockchain, this is by design a tricky issue, leading Orcutt to conclude, a bit cynically: “The reality is that we are probably stuck with energy-guzzling cryptocurrencies, at least for a while. In the meantime, maybe true believers would be wise to invest their digital coins in renewable electricity sources.

However, the problem is not just that we are stuck with energy-guzzling cryptocurrencies, but that we are stuck with energy-guzzling blockchains, the new kid on the block for “big data”, evident at this years World Economic Forum in Davos, where several large shops have been converted into blockchain "spaces", offering back-to-back sessions with utopian titles such as "Blockchain beyond Earth" and "Redefining human value", according to a BBC news report

According to a White Paper published by the World Economic Forum in June last year, “Blockchain, or distributed ledger technology, could soon give rise to a new era of the Internet even more disruptive and transformative than the current one. Blockchain's ability to generate unprecedented opportunities to create and trade value in society will lead to a generational shift in the Internet's evolution, from an Internet of Information to a new generation Internet of Value.” This should get us worried – also for many other reasons, as I pointed out in an article I co-wrote in my “activist” capacity.

While it might sound less of a problem, given that many of the main IT companies are aiming for 100% renewable energy, this is not really the solution. As the example of Ireland shows, you can only build so much renewable generating capacity. And while the sun and wind might be almost unlimited resources, the minerals and materials used to construct renewable energy infrastructure are not. There comes a point – if we are not already past it – where we simply need to make a choice between building renewable to satisfy basic human needs, or building renewable to satisfy the energy demand of Blockchains or Google searches.

José Halloy wrote recently in Ecopolitica: All this allows to conclude that, in terms of energy and materials, the present information technologies are not sustainable in the long term. The tensions over the consumption of energy and materials will manifest during this century. (…)

The scientific literature recognises that the important limits will be reached during the XXI century and this question is closely related to climate change and energy transition. Taking into account the time needed for scientific and technological development, this means that the necessary research to develop alternative solutions have to be carried out at the earliest possible time, and intensively.”

Next time you think about quickly searching the web for something during a conversation, think twice.

ISPConfig3 with Mail Quarantine, Roundcube and Amacube plugin

Netuxo recently migrated its mail hosting from an aging server using postfixadmin to a new server which forms part of our wider ISPConfig3 based hosting infrastructure. The default ISPConfig3 installation uses amavisd-new, all administered via the ISPConfig3 interface, which in our case is on a different server (a master-slave setup with one master server and several slave servers). We also wanted to provide the option to quarantine spam and virus emails, as several clients had expressed their fear of losing that important email because it could just get bounced (and they wouldn't know where this important email would come from, so whitelisting wasn't an option). It took us a while to figure out a good way of doing this, and this blog post describes what we did.

Our setup

Our setup is as follows:

  • A ISPConfig3 master server on server A. This server also provides webmail services using Roundcube.
  • A mailserver (server B), with ISPConfig3 as slave to the master on server A.
  • The ISPConfig3 Roundcube plugins, which allow users to control many aspects of their email account (such as vacation reply, forwarders, spam policy) from Roundcube.
  • A badly hacked version of the Amacube Roundcube plugin to access the mail quarantine. We did not want users to change their amavis settings using this plugin, as this is dealt with via the ISPConfig3 plugins.
  • For us as system admins a version of Mailzu to access the site or server quarantine.

Installation and configuration

We started with the default ISPConfig3 installation on a Debian Stretch server, adding server B as a slave to server A during installation. Server B only provides mail services, and in theory does not even need a web server, but this does not matter here. After setting up server B as mail server, we migrated all mail domains, accounts, aliases, forwarders etc from postfixadmin (this might be the topic for another blog post), so that they exist in ISPConfig3's databases on both, the master and slave servers.

The next step was the installation of the ISPConfig3 Roundcube plugins. The installation is well documented on the github page, so we do not cover it here. The set of plugins make use of the ISPConfig3 API interface and access the master server A. In our case, and following the installation instructions, this worked well out of the box. ISPConfig3 takes then care of replicating the settings to the slave server(s).

Mail Quarantine

We mostly followed the manual by Mathieu at to set up mail quarantine. In our case we opted for creating the additional tables for mail quarantine in the Amavis/ISPConfig3 database, mostly because Amacube later gave us a lot of trouble trying to use two different databases. There is no problem with having these additional tables in the ISPConfig3 database on only the slave server B, as ISPConfig3 will just ignore them. Important is that these tables need to be on the server actually running amavisd-new, NOT on the master server (unless the master server happens to be your mail server).


Amacube is where it gets complicated, and where we had to start hacking. For a start, Amacube is designed to access the amavis database directly, and in its original form allows a user to manipulate values in the amavis database. This is not what we wanted here, for two reasons:

  • As in our case we have a master->slave setup via ISPConfig3, manipulating the database on one server (server B) would get ISPConfig3 confused. Amacube does not do it the ISPConfig3 way - it just is not designed to do so (and there is nothing wrong with that).
  • Most of that side of the functionality of Amacube is provided by the set of ISPConfig3 Roundcube plugins, so there was no need for Amacube to provide that functionality.

What we were after was Amacube quarantine management, nothing more.

In a first step, we commented out the functionality related to the mail spam filter settings in Amacube. It probably would have been cleaner to do so via an additional config variable which would allow to disable this functionality, but in a first step we went for just commenting it out.

Then we got to the point we were interested in: Managing the Mail Quarantine. Amacube needs access (can be readonly) to the amavis/ISPConfig3 database on server B, which is the mail server. We created a MySQL user which can access this database remotely (remember, we have Roundcube installed on server A). In a first step, we had to adapt the query in Amacube quite a bit to make it work - some field names were different, some fields were in different tables, and some tables have different names in ISPConfig3 compared to Amavis. These fixes got us as far as showing the quarantined emails for exactly the logged in user. So far, so good.

In a second step we added some more to the query to:

  • Get all quarantined email for any alias of that user's email address, or any straight forwarder (>
  • Get all quarantined email for a catchall if the user's email is setup as recipient for catchall, but exclude mail to other existing mailboxes.

This added some extra queries (to get aliases, other mailboxes (to exclude), etc), and some more complexity to the query getting the relevant quarantined emails.

The resulting hacked version of Amacube can be downloaded from We would be happy for anyone else to use this version of Amacube and to improve on it.


Mailzu was initially the preferred option for managing the Mail Quarantine, but in the end we opted against it, and to just keep it for "us" to be able to check the Mail Quarantine of the entire server. Mailzu does not seem to be under active development for many years already, and it seems unlikely to get picked up again. The nice thing is that it allows "us" (superusers) to see all quarantined email, but that is not a functionality we would want to provide to anyone else. Mailzu does not have domain admins, which would limit access to all emails from one domain - which would be a nice functionality to have.

More importantly, once we came across Amacube it was clear that from a user perspective the best option was to provide all functionality in one place - Roundcube. This means the quarantined email can easily be accessed from this one place. Although Amacube does not provide the option to preview a quarantined email, we did not consider this so crucial as to justify asking a user to log into a separate interface to manage their quarantined email. So a hacked Amacube it is in the end.

Netuxo is retiring Squirrelmail

How to transfer your addressbook to Roundcube

At the end of November 2015 Netuxo will retire Squirrelmail, which we know some of our clients still use to access their email. We started to provide the much more user friendly and modern Roundcube client a long time ago, and now it is time to retire Squirrelmail as it is no longer being developed very actively.

As some of our clients also use the Squirrelmail addressbook, we'll explain in this blog post how you can export your addressbook from Squirrelmail, and import it into Roundcube. This is a straightforward process and you should not encounter any problems.

So, if you are a Squirrelmail user, please follow the steps explained below BEFORE the end of November 2015. If you do not do so, you will no longer be able to access your Squirrelmail addressbook after that date.

1. Export your addressbook from Squirrelmail

Login to your email using the Squirrelmail client (, and click on the "Addresses" link at the top. This will get you to your addressbook. You will now find some import and export options underneath. You can ignore the import options – if you want to import addresses, please do so into Roundcube only. What you need are the export options - at the very bottom.

You do not need to change any of the defaults here. Just click the button which says “Export to CSV File”. Depending on your browser, you might get different options.

In Firefox, you will be given the options “Open with” or “Save File”. You should save the file on your computer. Again, depending on your browser settings, you may be asked where to save the file (remember where you save it to!), or automatically save it in the default download location on your computer.

You have now successfully exported your Squirrelmail addressbook. The next step is to import the addressbook into Roundcube.

2. Import your addressbook into Roundcube

Now login into Roundcube at and click on the "Address Book" link at the top. You will probably see in your left sidebar two address books: Personal addresses and Global addresses. The global addresses is relevant for all email accounts in your domain (ie all mailboxes @yourdomain), so if you want the addressbook to be available to all email accounts, you can import it there.

First, select the addressbook you want to import into. This is likely to be empty if you did not use Roundcube before.

You will see the icon for import at the top – it is marked in the screenshot.

When you click on this icon, you will be shown the import screen:

Under “Browse...”, select the file you exported from Squirrelmail. You can again select which addressbook the contacts will be imported into.

Click on “Import”, and the import will start. If you have a large addressbook, this might take a while.

You should finally get a message displaying the number of successfully imported addresses.

And that is it. Simple.

Of course, in the unlikely event you experience any problems, please do not hesitate to contact us.

Additional address book functions in Roundcube

We already mentioned above that Roundcube provides a global addressbook for each domain. This enables you to share your addresses among several email accounts within the same domain.

Roundcube also enables you to add external addressbooks from other servers in carddav format. Netuxo provides a carddav server, which enables you to synchronize your addressbook between Roundcube and other email clients – on your desktop or mobile phone. Please get in touch if you would like to find out more and/or read additional information here: