Fifth International World Wide Web Conference
May 6-10, 1996, Paris, France
Panel10: Efficiency of Internet Indexing
Thursday 9 May, 1996 - 11:30-13:00
Panel Chair
Nick Arnett
Internet Evangelist
Verity Inc.
narnett@Verity.COM
Panel Members
- Nick Arnett, Internet Evangelist, Verity Inc., is an author and speaker who
previously managed Verity's Internet products -- servers, spiders and
CD-ROM publishing tools -- from their inception. Arnett also is co-owner
of Multimedia Computing Corp., which has published market analysis and
performed strategic consulting since 1988. His previous professional
experience was in journalism and publishing.
- Darren Hardy, Netscape. is the lead developer of the Netscape Catalog
Server. Previously was a professional research assistant at the University
of Colorado, Boulder, where he was one of the developers of the Harvest
information discovery and access system. Hardy has a B.S. and an M.S. from
the University of Colorado. See http://home.netscape.com/people/dhardy.
- Louis Monier, Digital Equipment Corp., is the Technical Leader of the Alta
Vista project. Monier was born and raised in France. He has a PhD from
the University of Paris, 1980. He spent three years at CMU in Pittsburgh
as a post graduate. He spent 6 years at Xerox PARC developing
computer-aided design tools for semiconductors. He has been with Digital
Equipment Corp. Western Research Lab since 1989.
- Tim Bray, Open Text Corp., is Senior Vice-President, Technology and a
co-founder. He graduated from the University of Guelph in 1981, and after
on-the-job training from Digital and GTE, became Manager of the New Oxford
English Dictionary Project at the University of Waterloo in 1987. That
project produced technology which served as the basis for the foundation of
Open Text. His current areas of interest focus
on the Web; specifically, in bringing to it a sense of place.
- David Eagles, Director - IT&T Division, PlaNET
Consulting Pty. Ltd., Brisbane, Australia. Designer of FunnelWeb.
- Ron Murray is a Technical Evangelist in Microsoft's
Developer Relations
Group. Over the past ten years he has contributed to and lead several
development projects within Microsoft. Within the Windows 95 and Windows
NT products he developed a full text engine which automatically adapts
to multiple linguistic environments and which provides an incremental
query interface. He is currently working on methods for efficient and
disciplined data gathering across the Internet.
Brief Outline
The proliferation of Web robots has led to increasingly redundant
indexing, unnecessarily tying up CPU cycles and bandwidth. The Harvest
project at the University of Colorado developed a more efficient system
(presented at WWW2), but Harvest thus far has failed to achieve widespread
adoption; it is not used by any of the major Web search services. "Robot
wars" loom in the future, as webmasters and indexers fight over
increasingly saturated resources. This is an example of the need for
"cooper-tition" -- erstwhile competing developers must cooperate to allow
the technology to grow to the next step.
The future direction of this technology is only clear in a negative sense
-- the inefficiency of today's non-cooperative robots is the wrong
direction. This panel will discuss Harvest and more recent projects that
would address this issue. Lively discussion should result as the
panelists attempt to reconcile the need for efficiency with the demands of
competition.
Related topics will also be covered: standards to allow robots to identify
and skip multiple copies of documents, aliases and symbolic links;
requirements for robots on heavily trafficked enterprise networks.
Mail to the Organizers
Created: 9 April 1996
Last updated: 26 April 1996