Real Distributed Gatherering on the Internet?

• Coordinating gatherering effort is tough right now:
  • Coverage is a competitive aspect
  • No agreements on indexing data interchanging
  • Few publish results or indexing data after gatherering
• Few adopted mechanisms for targeting gatherering (e.g., /robots.txt)

• More and more autonomous Robots are being deployed...
  And the Web isn't getting any smaller!