Are You Wasting Time On Pointless Website Updates? Here's An Easy Way To Discover Exactly When Your Website Gets Spidered By The Major Search Engines, So You Can Target Your Site Updates For Maximum Benefit. Do you regularly update your websites to make them more attractive to the search engines? And did you realise that your probably wasting a lot of time and effort in doing this? Search engines send out "spiders" that crawl through websites in order to find all the web pages so they can be listed on the search engine.