User-agent: * Disallow: /Res/ Disallow: /Prod/Book/Netlabv03a/Excerpts/FrontCover Disallow: /Prod/Book/Netlabv03a/Excerpts/Chap01 Disallow: /Prod/Book/Netlabv03a/Excerpts/Other Disallow: /Site/la/ # . . . . . # Google (Googlebot) # Google ignores the ALL group (above) # if there is a specific group for it # (here). All directives you want # google to disallow must be listed # here, even if they are in the ALL # group above. # User-agent: Googlebot Disallow: /Res/ Disallow: /Prod/Book/Netlabv03a/Excerpts/FrontCover Disallow: /Prod/Book/Netlabv03a/Excerpts/Chap01 Disallow: /Prod/Book/Netlabv03a/Excerpts/Other Disallow: /Site/la/ # . . . . . User-agent: ia_archiver Disallow: /cgi-bin/ Disallow: /Blog/ Disallow: /Prod/ Disallow: /Res/ Disallow: /Site/ # . . . . . User-agent: twiceler Disallow: /