

Side-note: did you know the collective noun for a group of frogs is an army of frogs? I didn’t… This makes possible crawling an entire industry faster than you could typically crawl one website using just the computer that you’re reading this article on.
#Screaming frog license key how to#
In the next section, I’m going to explain how to set up a system in which you feed in a series of websites, run some scripts and, in the background, it initiates multiple virtual machines that are each running Screaming Frog Crawler and are each allocated a website from the list you gave it, to start crawling. If you use Google Compute Cloud, though, it suddenly becomes possible. What if, instead, you could run multiple machines at the same time, each running its own copy of Screaming Frog? That way, you could simultaneously crawl every site in an industry. Typically, this isn’t an option because they’d be fighting with each other for network bandwidth and putting in an order to your boss for 10 laptops is unlikely to get the green light. Crawling an entire industry would either take a prohibitive amount of time, if using Screaming Frog, or be expensive if using a SaaS crawler like Deepcrawl. If we’re honest, the reason we don’t create graphs like these is not that they’re ineffective, but because you’d have to crawl an entire industry. We ditch the competitor comparisons and don’t show graphs like these: We start showing relatively complex formulas to explain how fixing a canonical issue, for example, might influence total revenue generated. Something strange happens when we start to discuss technical SEO, though.

Here, instead of saying that 15% of your anchor text uses commercial terms, we might comment that the analysed site has 4x the commercial anchor text than the industry average – that they’d need to increase the size of their link portfolio by 11% with purely branded anchor text just to get back to the industry average.Īs almost every company has that one competitor that it really really hates, when presenting to the C-suite, we find that comparing directly with that one competitor can often yield the fastest results:

We provide industry comparisons to show the depth of the problem or opportunity: Telling someone that their website has a DTOXRisk of 156 is likely meaningless, telling them that their link portfolio has ‘less than half the industry-average risk of a penalty’ is immediately accessible. As a result, it’s very common, when discussing links, to compare companies to their competition. What almost all businesses care about is their competition. Perhaps more importantly, though, these need to be communicated to people who are often not SEO-savvy in a way that’s compelling that’ll inspire action. As a result, we rely on our technical SEOs to have an increasingly deep understanding of the subject to be able to find insights. Technical SEO has become increasingly complex.
#Screaming frog license key free#

Now, I want to share how we took this to the next level – how you can use this to automatically crawl as many websites as you want simultaneously. We’ve previously talked about running Screaming Frog’s Crawler on Google’s Compute Cloud.
