.Google.com's Gary Illyes reveals Googlebot picks up hyperlinks as opposed to "adhering to" all of them, opposing formal paperwork.Googlebot collects web links as opposed to observing all of them in real-time.Gary Illyes discussed this information in Google's 'Look Off The Document' podcast.Google.com's records opposes the real spider behavior.