{"id":4985,"date":"2018-06-28T00:21:17","date_gmt":"2018-06-28T00:21:17","guid":{"rendered":"http:\/\/www.hexacorn.com\/blog\/?p=4985"},"modified":"2018-06-28T00:21:17","modified_gmt":"2018-06-28T00:21:17","slug":"its-understood-that-eu-dudes-sell-gdprization","status":"publish","type":"post","link":"https:\/\/www.hexacorn.com\/blog\/2018\/06\/28\/its-understood-that-eu-dudes-sell-gdprization\/","title":{"rendered":"It’s understood… that EU dudes… sell GDPRization…"},"content":{"rendered":"

I’ve been recently thinking of GDPR, and its influence on the non-EU websites… in particular, I was curious how the legislation affects the user experience for non-EU sites for visitors from EU. We hear about many websites in US simply denying the access e.g. LA Times<\/a>:<\/p>\n

\"\"<\/a><\/p>\n

but I was curious how many other web sites really do so…<\/p>\n

I came up with a quick & dirty (and pretty simple) idea of checking how the popular web sites respond to the regulation… by visiting them and taking a screenshot.<\/p>\n

Of course, manual check would be too labor-intensive, so I automated it.<\/p>\n

First, I needed a list of top world web sites so I downloaded the Cisco Umbrella<\/a> list. I know it’s biased, but don’t know any better source (since the\u00a0 free Alexa top 1M is long gone, and others – I really don’t know how accurate they are).<\/p>\n

I then created a simple script in perl to extract the first 10000 top unique domains from the list (and exclude all subdomains on the way):<\/p>\n

use strict;\r\nuse warnings;\r\nmy %h;\r\nmy $cnt=0;\r\nwhile (<>)\r\n{\r\n  if (\/,([^\\.]+\\.[^\\.]+$)\/)\r\n  {\r\n    if (!defined($h{$1}))\r\n    {\r\n      print \"$1\";\r\n      $h{$1}++;\r\n      $cnt++;\r\n      exit if $cnt >= 10000;\r\n    }\r\n  }\r\n}\r\n<\/pre>\n
Next, I wrote a simple phantomjs script to grab a screenshot of these domains (all accessed via http and then rerunning for https for these that didn’t work):<\/p>\n
system = require('system')\r\nvar page = require('webpage').create();\r\n     page.viewportSize = { width: 1024, height: 768 };\r\n     page.clipRect = { top: 0, left: 0, width: 1024, height: 768 };\r\naddress = system.args[1];\r\noutput  = system.args[2];\r\npage.open(address, function() {\r\n  page.render(output);\r\n  phantom.exit();\r\n});\r\n<\/pre>\n
And then I ran the phantomjs on domains from this data set… each page visited is saved as a png.<\/p>\n
To my surprise, the experiment didn’t work as I anticipated.<\/p>\n
Most of web sites visited didn’t really make any comment on GDPR and it was business as usual. Some offer an option to accept new privacy policies. In the end I only came up with a bunch of examples.<\/p>\n
Still, it was worth trying…<\/p>\n
Lessons learned…<\/p>\n