Close

Browse by...

Dates click to expand

2008
3
1

Juiced Google Analytics Python API

It is not official. It is not from Google. It is, however, very functional and very here. I present to you pyGAPI, the Juiced Google Analytics Python API. This module allows you to pull information from your incarnation of Google Analytics and employ it programatically into your reporting code.

Let us use iPython to peek through some code using pyGAPI.

In [3]: from datetime import date
In [4]: import pyGAPI
In [5]: connector = pyGAPI.pyGAPI(username, password, website_id="1234567")

Here we create a pyGAPI object. Behind the scenes, pyGAPI logs into Google Analytics, and downloads an identifier cookie. website_id is optional. If omitted, pyGAPI accesses the first website on the account's list. To get a list of all the site IDs to which your site has access, run the function connector.list_sites().

In [6]: connector.download_report('KeywordsReport', (date(2008,3,10), date(2008,3,31)), limit=5)

Download a report into your pyGAPI object. KeywordsReport is the name of the report. It is followed by a tuple containing the start and end dates in python date format. limit is an optional parameter that specifies the number of entries that pyGAPI should pull down. By default, it will pull in all the entries up to a maximum of 10000. Lowering this number will certainly improve performance. The entries returned are ranked by Visits, so you should get the most significant values of the bunch.

In [7]: print connector.csv()
Keyword,Visits,Pages/Visit,Avg. Time on Site,% New Visits,Bounce Rate,Visits,Subscribe,Solutions,Goal Conversion Rate,Per Visit Goal Value
juice analytics,356,5.935393258426966,314.061797752809,0.38764044642448425,0.29494380950927734,356,1.0,0.16292135417461395,1.1629213094711304,0.0
excel training,142,1.971830985915493,98.0774647887324,0.908450722694397,0.6901408433914185,142,1.0,0.0211267601698637,1.0211267471313477,0.0
excel charts,77,1.7922077922077921,95.0,0.9090909361839294,0.7792207598686218,77,1.0,0.03896103799343109,1.0389610528945923,0.0
excel skills,72,1.6527777777777777,75.29166666666667,0.9444444179534912,0.7083333134651184,72,1.0,0.0,1.0,0.0
colbert bump,70,1.3142857142857143,113.77142857142857,0.6428571343421936,0.8428571224212646,70,1.0,0.0,1.0,0.0

This function displays your report in a nice excel-ready CSV format.

In [8]: print connector.parse_csv_as_dicts(convert_numbers=True)
[{'Avg. Time on Site': 314.06179775280901, 'Per Visit Goal Value': 0.0, 'Bounce Rate': 0.29494380950927734, 'Keyword': 'juice analytics', 'Visits': 356.0, 'Pages/Visit': 5.9353932584269664, 'Subscribe': 1.0, 'Solutions': 0.16292135417461395, '% New Visits': 0.38764044642448425, 'Goal Conversion Rate': 1.1629213094711304}, {'Avg. Time on Site': 98.077464788732399, 'Per Visit Goal Value': 0.0, 'Bounce Rate': 0.69014084339141846, 'Keyword': 'excel training', 'Visits': 142.0, 'Pages/Visit': 1.971830985915493, 'Subscribe': 1.0, 'Solutions': 0.021126760169863701, '% New Visits': 0.90845072269439697, 'Goal Conversion Rate': 1.0211267471313477}, {'Avg. Time on Site': 95.0, 'Per Visit Goal Value': 0.0, 'Bounce Rate': 0.77922075986862183, 'Keyword': 'excel charts', 'Visits': 77.0, 'Pages/Visit': 1.7922077922077921, 'Subscribe': 1.0, 'Solutions': 0.038961037993431091, '% New Visits': 0.90909093618392944, 'Goal Conversion Rate': 1.0389610528945923}, {'Avg. Time on Site': 75.291666666666671, 'Per Visit Goal Value': 0.0, 'Bounce Rate': 0.70833331346511841, 'Keyword': 'excel skills', 'Visits': 72.0, 'Pages/Visit': 1.6527777777777777, 'Subscribe': 1.0, 'Solutions': 0.0, '% New Visits': 0.94444441795349121, 'Goal Conversion Rate': 1.0}, {'Avg. Time on Site': 113.77142857142857, 'Per Visit Goal Value': 0.0, 'Bounce Rate': 0.84285712242126465, 'Keyword': 'colbert bump', 'Visits': 70.0, 'Pages/Visit': 1.3142857142857143, 'Subscribe': 1.0, 'Solutions': 0.0, '% New Visits': 0.6428571343421936, 'Goal Conversion Rate': 1.0}]

This function goes the extra step and converts the CSV into a dictionary for easier programmatic use. By default, all entries will be returned as python strings. Setting convert_numbers to True, as we did here, will additionally parse the dictionary to turn all numbers into float values.

In [9]: print connector.list_reports()
('ReferringSourcesReport', 'SearchEnginesReport', 'AllSourcesReport', 'KeywordsReport', 'CampaignsReport', 'AdVersionsReport', 'TopContentReport', 'ContentByTitleReport', 'ContentDrilldownReport', 'EntrancesReport', 'ExitsReport', 'GeoMapReport', 'LanguagesReport', 'HostnamesReport', 'SpeedsReport')

This gets a list of all the reports that I have successfully tested thus far. All site-specific reports should work. A couple site-section specific reports should be included in the next update of pyGAPI.

Google is great and will release a real API soon, but until then you can download pyGAPI.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. All source code is released under a BSD License unless otherwise specified.

11 comments | Show all comments only the last 5 are shown


June 21, 2008
Matt Webb said:

This is awesome work. Do you think this python script could work in conjunction with superkaramba on Linux?


June 27, 2008
Rodrigo said:

This is great. I put this together with a Samurize desktop to display Analytics data on my desktop.
Thanks!


August 7, 2008
Ludovic said:

Very nice work. Very useful to, let's say get your most visited pages without having to maintain parallel accounting. May I ask you to licence it to an OSS licence and put it on Google Code ? Would be great.


August 20, 2008
Sebastian said:

Hello,

it work well! Great.
How can i pull the "keyword" or "country" report for a specific URL?
(use segmention)

Thanks


September 5, 2008
Thierry said:

Awesome work !

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment


Add a comment





Keyword Trends in Google Analytics With Greasemonkey

Note: We've updated the script to work on Firefox 3 as well as Firefox 2.

After the warm reception for the first version of our Enhanced Google Analytics, we decided to add some new functionality. (Nothing like a few kinds words to keep us in the giving mood.) The first script created a couple new tables in the Google Analytics interface that highlight recent changes in referral visits. It uses Greasemonkey, an add-on for Firefox that allows a user to insert javascript directly into a webpage.

Our update gives you even more ability to understand the data in Google Analytics:

  • At the suggestion of Avinash Kaushik, the new script works for keyword data, helping you see how organic search traffic is changing. An increase in a keyword may indicate a general change in user interests and/or improved performance on search results.
  • My coworker Pete Skomoroch also suggested that I add the ability to see declines in referrals and new keyword searches.
  • With the help of Paul Irish, the script is now better able to interface with the date widget on the Google Analytics site.

(Click the above button for a simulation.)

Keyword Growth Keyword Decline

When you click the button, your browser will download some historical data behind the scenes, and display a nice summary of the best and worst performing keywords/referring domains.

Installation Instructions:
Firefox 2.0+
Greasemonkey
googleanalyticsdownloade.user.js

If you don't already have Firefox, install it. Install Greasemonkey, and do the required Firefox restart. You should see a handsome monkey peeking at you from the bottom right hand corner of your browser. Open the script file in your firefox browser, and Greasemonkey should give you an option to install the script.

Afterwards, log into Google Analytics, and navigate to your Referring Sources or Keywords Tab. Click the button.

Configuring the script:

We spent some time trying to find convenient default settings here at Juice Analytics, so the script should work straight out of the box. Some users, however, may find it convenient to alter some of these configurations. To do so, in Firefox, go to Tools=>Greasemonkey=>Manage User Scripts..., select Google Analytics Downloader, and then click Edit in the lower left corner of the window. This should open up the script file in a text editor. If your computer does not have a default text editor configured, you may have to choose one. 'c:\windows\notepad' is a good bet for Windows machines.

This is what you should see:

Code Blurb

The bracket labeled 'keywords?' controls defaults for the Keywords page, and correspondingly, 'referring_sources?' controls the Referring Sources page.

To change the settings, simply change the corresponding variable to your preferred default. Make sure to refresh your Google Analytics webpage, if you have it open, so the new settings are loaded.

Now for the nitty gritty configuration details:

  • display_limit: This controls the maximum entries that each table will contain. This may be useful for large, sprawling sites.
  • growth_tolerance: This is the percentage growth parameter. Changing it to .10, for example, will catch everything that has grown by 10%, as opposed to the default 50% and 20%, respectively.
  • minimum_number_elements: This is a significance benchmark that can be used to limit what is displayed upon the screen. By default, only keywords with at least 10 elements are displayed upon the screen. Referring Sites does not have a minimum by default, but one can be set if desired.
  • limit: Limit is more of an internal parameter that determines how many entries should be downloaded from Google in order to get the results that are visible here on the page. Lower the limit to increase speed. If the limit is set to a very high number, you will get the largest result set, but you will have to sit around for a while for the results to load. Since the results are downloaded ordered by volume, raising the limit from the default numbers will not actually give more significant results. You will simply get more of the smaller results, such as keywords with only 1 hit.
  • look_back: This is a very important parameter. The script uses the date displayed upon your Google Analytics page to determine the full range that you want to consider in your results, but 'look_back' determines how many of those days are used for the significance test. So, say the range you have displayed in Google is March 23 - April 22 and your look_back is 7 days. The script will compare the average referrals for a given keyword from April 16-22 to the average from March 23-April 15, and will return the keyword only if the recent average is 20% higher than the rest of the time period. Thus, if you want to increase the total range of the data, change the dates on the actual webpage. Change 'look_back' only if you want to change the period of significance.

Happy analyzing!

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. All source code is released under a BSD License unless otherwise specified.

23 comments | Show all comments only the last 5 are shown


September 16, 2008
Chris said:

Hmm, I think this is a new problem, I am looking for a solution, if I fine one I will post it here.

Chris Avery


September 16, 2008
Sal Uryasev said:

I updated the script. Google Analytics changed their URL around slightly, adding "#lts=1221579205724", which messed with the script.

Uninstalling and reinstalling the script would probably be the easiest way to install the update.


September 16, 2008
Chris said:

Hey,

Thanks for taking the time to do that , we all appreciate it

Cheers


September 16, 2008
Bjoern said:

Hi Sal,

thanks for updating this so quickly. I'll check it out. It really is a great script.

@ All: Sal and the Juice Analytics Team might appreciate it, if users became "fans" of the script at the download location at userscripts.org - so that there is some recognition.


September 25, 2008
Katie said:

Thank you so much for your clear explanation of how to intgrate this awesome script into GA. Anytime you can dumb it down a little for us non-web developers/coders it is much appreciated. Marketing Mavens everywhere thank you!

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment


Add a comment





Enhancing Google Analytics Using Greasemonkey

There is a new post with updates to the script. Find it here.

My boss Zach has a problem. Every four hours the craving strikes him. No matter where he is, he pulls out his shiny Macbook Pro and navigates to Juice's Google Analytics site. He pulls up the list of referrers to our site and meticulously searches for new domains. He has an freakish ability to pick out IP addresses that have never linked to us before. Even so, there had to be a better way.

I wondered whether Greasemonkey might be able to help. Greasemonkey is an extension for Firefox that allows users to install custom javascript when you visit a specific website. These scripts can add a delete button for Gmail, automatically display lyrics to your YouTube music video, or do pretty much anything else you would want to enhance the functionality of a website.

After poking around the subtleties of the Google Analytics interface, I came up with a little script that can identify the new referrals that Zach so desperately craves. When navigating to the "Referring Sites" section of Google Analytics, the script add the following button to the interface.

Google Analytics Button

Pushing the button downloads all the referrer data for the date displayed in the Google Analytics range, as well as a similar set of data for the range up to, but not including, the last three days. The difference between the two data sources is used to calculate all of the results. The specific number of days can be changed by editing the first line of the script. Greasemonkey then displays the results in two tables above the original Referrer table. (Greasemonkey works entirely within your browser shell, so your data should be quite secure.)

Google Analytics Data

The first table shows any sites that have displayed more than a 50% increase in visits over the last 3 days as compared to the rest of the time range. The second shows all new recent sites that do not appear at all more than 3 days ago. This can be quite useful to anyone, who, like Zach, absolutely needs to know about any new and exciting inbound links.

Installation Instructions:
Firefox 2.0+
Greasemonkey
googleanalyticsdownloade.user.js

If you don't already have Firefox, install it. Install Greasemonkey, and do the required Firefox restart. You should see a handsome monkey peeking at you from the bottom right hand corner of your browser. Open the script file in your firefox browser, and Greasemonkey should give you an option to install the script.

Afterwards, log into Google Analytics, and navigate to your Referring Sources Tab. Click the button.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. All source code is released under a BSD License unless otherwise specified.

19 comments | Show all comments only the last 5 are shown


April 23, 2008
Sal Uryasev said:

Matthew/Dave: The new post (linked at the top of the page) should have the details you need.
Tim: It sounds as if you did everything correctly. Is the script visible if you go to Tools=>Greasemonkey=>Manage User Scripts?
Neerav: There really is no great way to subscribe to just this script. I will link in the relevant places if anything changes though.


April 23, 2008
Neerav said:

Hi Sal

Enabling the "subscribe to comments" plugin could help. That way you can post a comment saying there's a new version and everyone who commented gets that comment as an email


May 1, 2008
Tim said:

Hi Sal,
thank you so much - I just reinstalled it and now it works! This is really fantastic!
Thanks,
Tim


May 7, 2008
Nate Sidmore said:

Awesome tool Sal, (thanks to Avinash for the tip).

I did run into a problem with the Firefox pop-up message "Warning: Unresponsive script". However that problem can be solved by lengthening the time allowed for scripts to run. For more details go to http://lifehacker.com/software/firefox/put-off-firefox-15s-unresponsive-script-dialogue-162574.php

However I was bummed when after setting the time allowance to 10 minutes, and clicking the "Who Sent Me Unusual Traffic" button in GA, the script ran for 9 min 38 sec before returning results. Any tips on getting quicker returns?


May 7, 2008
Chris Gemignani said:

Nate:

Thanks for the encouragement. If you check our "Keyword Trends" Greasemonkey script (linked at the start of this post), we write about how to change the parameters in the script to make things run faster.

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment


Add a comment