How to use data science to calculate Share of Search

The ability to quantify and explain the value of SEO to stakeholders remains one of the most crucial parts of any agency’s day-to-day. Better communication of how the work being done impacts KPIs and the wider business will lead to stronger client relationships and increased faith in the partnership between agency and client.

Many methods of reporting and recording progress exist in SEO. Google Analytics offers the ability to quantify and track any number of user metrics such as traffic and impressions as well as metrics such as conversions and revenue. Added to this third-party tools such as Ahrefs or SEMrush can expand the analysis into keyword research and scoping new opportunities to explore. SEO has many different metrics to quantify but how can they be easily digested and communicated to clients? To do this we developed a new analysis, Share Of Voice.

What is a Share of Voice?

To add a new dimension to our reporting process we at Distinctly looked at reporting on the ‘Share of Voice’ (SOV) a client possesses. By combining a number of factors such as keyword search volume, click-through-rate and SERP positioning we are able to produce a figure that represents the share of search a particular client possesses. This can then be benchmarked against competitors to produce a graph such as that found below.

The real benefit of using a share-of-voice process is that it enables you to easily analyse keywords by category. Building up a view month on month of how you rank in different categories can provide oversight into where you are strong and where a change of tactic may be needed to increase page rankings. This process allows you to analyse as many keywords as you need, into the hundreds of thousands if need be, without the need for any change of process. 

Analysing such large volumes of data is no easy task, it requires a rigorous process and mitigating the risk of errors is crucial. This is where a platform like Google Sheets falls short. The speed of calculation, sheet size/memory limits and general usability make it easy to make a mistake when editing figures. Moreover it is a very rigid structure, each new iteration of the process would mean repeating the formulas all over again. A better solution is needed. Python, a programming language that is used by the data science community worldwide, is that solution. The ability to handle large volumes of information makes Python ideal for calculating SOV data.

How using Python creates the perfect platform for analysing data

As a general rule anything that is under the umbrella of data science or data analysis works great for some individuals but causes real issues with others. Fluency with Google Sheets or Excel can cause massive discrepancies in abilities to complete analysis and when reporting on data that is being used to guide activity it must be grounded in fact. To combat this I created three different Google Colab Python scripts, each designed to deal with a specific part of the process.  

Each of these three scripts required the absolute minimum input, simple additions of; URLs to the spreadsheets, keyword categories and importing a CSV from a SERP scraper API are all that are required to run the scripts. 

The first part of the process is running a single Share of Voice. Taking just a single spreadsheet URL as well as some information downloaded from an API the script will run a full analysis. Calculating the SOV overall as well as by predefined categories, all the SEO needs to do is hit play in Google Colab. 

However, one SOV on its own isn’t worth much. It is just a snapshot of your visibility at the time the SERPs were scraped. To get a true representation of how your work is of benefit to your client you need to compare month-on-month.

This required a new script. Again building on the principles of simplicity the script needs minimum input, just three spreadsheet URLs are needed. The current month’s data, last month’s data and a blank sheet into which the results can be sent. Using this over several months gives you data with which to analyse the trends.

It is by analysing the trend over time that a SOV process can come to dictate where you spend your time. Targeting and improving categories which perform weakly whilst maintaining those which dominate their SERP. 

Overall the value of SEO can be reported on by many metrics. When reporting to stakeholders some are easy to understand, like traffic and some, like domain rating or authority, require more explanation. Adding Share of Voice data into the reporting narrative makes for an easily digestible metric for the client to appreciate the improvements made by the agency.