The goal of this project is to find ways to optimise the crawler frequency for individual web sites. The idea is to avoid crawling a site in case the accessibility to the site has not been changed. This is a challenge for all search engines to focus the resources on actual changes. Another relevant aspect of sampling is to select a significant and representative set of sites.