Cookie Collector Web Service Update
The Cookie Collector has been busier than ever in the last few weeks, and we now have over 78,000 websites we have collected cookies from since we started gathering data back in May.
It has been a huge success for us, bigger than we had expected, but it has not been without its problems.
The sheer volume of data involved has taken us by surprise, and the cookie database has been re-engineered more than once to handle the amount of data and the rate at which we can process it.
For a while we have been aware that a lot of data being submitted by the Cookie Collector was not getting through because of a processing bottleneck. As a result of this we developed a two stage system which involved first queuing the raw data and then processing it.
This solved the problem of lost data, but it then created a situation where the queue was building up faster than we could process it. In just two days it built up to 7.5 milllion cookies, despite the fact that we were deleting them from the queue as fast as we could process them.
We therefore decided we had to take the service down for a few days, which would give us time to clear the back log. That was last Friday, and it means we have not been collecting any new data since then.
We are working hard on a long term solution but in the meantime the service remains down, which means we are not collecting new cookie data at the moment. We will of course notify everybody when the situation changes.
January 13, 2017
Future of EU Cookie Compliance Webinar: ...
GDPR and now the proposed E-Privacy Regulation mean a stricter regime for cookie compliance, web governance and use of online tracking technologies. Join p...View Article →
December 14, 2016
Draft EU ePrivacy Regulation Leaked...
A draft of the proposed legislation to replace the outdated EU ePrivacy Directive was leaked on the Politico.eu (PDF) website this week. The proposal is fo...View Article →
November 3, 2016
GDPR Compliance Means Cookie Notices Mus...
Are you one of those people that ticked the cookie law box ages ago and not thought about it since? Well the game has changed and now is the time to re-vis...View Article →