About two weeks after I first posted some thoughts about the idea that "robots.txt" might be an example of working DRM, an American lawsuit pops up that addresses exactly that question.
There are some posts on this lawsuit at the Patry Copyright blog, Lessig blog (where I saw it first) , and Infothought.
While most first reactions have been dismissing the idea of "robots.txt as DRM", Seth Finkelstein in the Infothought post quoted above says it might be a "much tougher case than it first appeared".
Since I have thought of robots.txt as DRM before this case even came up, I agree completely.
Let's take another look at the definition (under European law, that is Article 6 Paragraph 3 of the 2001 Copyright Directive:
For the purposes of this Directive, the expression "technological measures" means any technology, device or component that, in the normal course of its operation, is designed to prevent or restrict acts, in respect of works or other subject-matter, which are not authorised by the rightholder of any copyright or any right related to copyright as provided for by law or the sui generis right provided for in Chapter III of Directive 96/9/EC. Technological measures shall be deemed "effective" where the use of a protected work or other subject-matter is controlled by the rightholders through application of an access control or protection process, such as encryption, scrambling or other transformation of the work or other subject-matter or a copy control mechanism, which achieves the protection objective.
First, robots.txt needs to be a "technology, device or component".
That seems to cover a wide area. I see "robots.txt" as "component" of a communications protocol.
Second, this component needs to be designed to restrict or prevent acts concerning works.
That is exactly what "robots.txt" is supposed to do. The owner of the copyright tells other machines on the Internet not to copy or (in the new case) display his content.
Third, the "technological measure" needs to be "effective".
The definition of "effective" is that the measure needs to "achieve the protection objective".
Since this whole definition is in the context of outlawing circumvention (which means that the protection objective has not been achieved), "effective" can't be reasonably understood to mean "always achieve the objective". If some DRM scheme (maybe using "trusted computing") achieves its objective in each case, there is no need to worry about circumvention.
But with robots.txt? All it takes is to ignore the directive in robots.txt, so one might assume that it can never be "effective".
However, as I have noted before, even a low fence easily hopped over is a protective measure that is effective if most people don't start jumping. There is no need to build Berlin walls to protect restricted areas in the city park.
And this case shows that the person wanting access and the person deciding about ignoring or respecting "robots.txt" can be quite different.
The whole problem came from the fact that the Internet Archive seems to keep old content even if the site owner instructs them to stop crawling the site for the future.
I don't see any legitimate reason to do this. If the copyright owner tells the Internet Archive to stop distributing the content, any implied license has clearly been revoked.
Therefore, at the very least, I think that the Internet Archive needs to just delete all old website content if the copyright owner objects to the archiving.
They are treading on thin ice anyway. While the Google cache is clearly illegal under any conceivable reading of European copyright law, the Internet Archive might just be safe by an extremely slight margin.
But they better pay some attention to the potential problems here.
I have one more comment on the comments to this case. Patry and Lessig give the impression that the lawsuit accuses the Internet Archive of violating the DMCA. That is not the true. The DMCA violation count is only against the other defendant.
So I think that Patry's theory that "plaintiff's theory would encourage good government archivists like the Internet Archive not to use voluntary measures on pain of a DMCA violation" doesn't make any sense.
His comparison with "open doors" in a street doesn't make much sense either. The defendants against the DMCA violation claims did not just try a lot of doors once each. They tried the one door they were interested in so long until they got access. If one wanted a real-world analogy, that would be trying the combinations in a three-number bicycle lock until you come up with the correct one.
So yes, this case might be a bit tougher than people thought at first glance.