Welcome to AikiWeb Aikido Information
AikiWeb: The Source for Aikido Information
AikiWeb's principal purpose is to serve the Internet community as a repository and dissemination point for aikido information.

Sections
home
aikido articles
columns

Discussions
forums
aikiblogs

Databases
dojo search
seminars
image gallery
supplies
links directory

Reviews
book reviews
video reviews
dvd reviews
equip. reviews

News
submit
archive

Miscellaneous
newsletter
rss feeds
polls
about

Follow us on



Home > AikiWeb Aikido Forums
Go Back   AikiWeb Aikido Forums > Announcements & Feedback

Hello and thank you for visiting AikiWeb, the world's most active online Aikido community! This site is home to over 22,000 aikido practitioners from around the world and covers a wide range of aikido topics including techniques, philosophy, history, humor, beginner issues, the marketplace, and more.

If you wish to join in the discussions or use the other advanced features available, you will need to register first. Registration is absolutely free and takes only a few minutes to complete so sign up today!

Reply
 
Thread Tools
Old 12-19-2009, 07:06 PM   #1
Marc Kupper
Dojo: Aikido of Diablo Valley / ASU
Location: Walnut Creek, California
Join Date: May 2002
Posts: 88
United_States
Offline
Does Aikido work with robots and spiders?

I did a Google search for something and was surprised to get a Page Changes page from the aikiweb wiki. Here are some suggestions for robots.txt under User-agent: *

# The wiki content is under the /wiki/ prefix and much of the
# editing, history, etc. is prefixed with /mediawiki/
Disallow: /mediawiki/

# A bit global but there's no reason to index any of the special
# pages. Plus many of them are computationally intensive.
Disallow: /wiki/Special:
Disallow: /wiki/Special%3A

# However, if consensus is that some/many of the special pages
# should be in the index then below are some specific special
# pages that should get blocked.

# A search engine will index some random page and when someone
# clicks to view that page in the search results they will
# likely get some other random page.
Disallow: /wiki/Special:Random
Disallow: /wiki/Special:RandomPage
Disallow: /wiki/Special:RandomRedirect

Disallow: /wiki/Special%3ARandom
Disallow: /wiki/Special%3ARandomPage
Disallow: /wiki/Special%3ARandomRedirect

# Search is computationaly intensive and does not return
# results useful to a search engine index.
Disallow: /wiki/Special:Search

Disallow: /wiki/Special%3ASearch

Marc
  Reply With Quote
Old 12-21-2009, 08:15 PM   #2
RED
Join Date: Apr 2009
Posts: 903
United_States
Offline
Re: Does Aikido work with robots and spiders?

sweet

MM
  Reply With Quote
Old 12-22-2009, 10:06 AM   #3
akiy
 
akiy's Avatar
Join Date: Jun 2000
Posts: 5,843
Offline
Re: Does Aikido work with robots and spiders?

Hi Marc,

Thanks for that. I've implemented some of your suggestions.

-- Jun

Please help support AikiWeb -- become an AikiWeb Contributing Member!
  Reply With Quote

Please visit our sponsor:

Aikido DVDs and Video Downloads - by George Ledyard Sensei & other great teachers from AikidoDVDS.Com



Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -6. The time now is 06:13 AM.



vBulletin Copyright © 2000-2014 Jelsoft Enterprises Limited
----------
Copyright 1997-2014 AikiWeb and its Authors, All Rights Reserved.
----------
For questions and comments about this website:
Send E-mail
plainlaid-picaresque outchasing-protistan explicantia-altarage seaford-stellionate