Web Analytics TV #17

Welcome to yet another exciting magical episode of Web Analytics TV!

Web Analytics TV is powered by your questions, in this episode we had questions from Australia, India, Russia, Germany, Israel, Poland, Argentina, Dubai, Cyprus and so many other places. It is amazing to see such an engaged audience from around the world. You all rock!

Our process is simple.

Step 1: You ask, or vote on, your favorite web analytics questions via the Web Analytics TV Google Moderator site.

Step 2: From a secret undisclosed location at the Googleplex Avinash Kaushik & Nick Mihailovski answer them. : )

We are also thrilled to present Lloyd from Cape Town with the “Analysis Ninja of the Episode” award for his excellent question about the differences about custom variables and advanced segments. Lloyd, just send us an email and we’ll send you an autographed, personalized, copy of Web Analytics 2.0.

Here is the list of last weeks questions.

In this action-packed episode we discuss:

• (0:38) Why you see differences between clicks and visits from Facebook metrics
• (2:35) Setting up goals to match multiple conditions
• (3:55) The best way to create inverse advanced segments
• (4:36) Is there a way to get the path of sources for a user?
• (5:44) Comparing Website Optimizer experiments against different success criteria
• (7:30) How you should track banners for internal campaigns
• (9:01) Targeting visitors using Google Analytics data
• (10:25) What Google Analytics reports for URLs that get rewritten by the server
• (11:41) Is it “cheating” to track virtual pageviews on tabs?
• (13:12) How to group and compare keywords in the longtail
• (15:18) Differences between custom variables and advanced segments
• (18:32) Best ways to test regular expressions on your own data
• (20:30) Testing if multiple tracking codes on the same page are sending data
• (21:45) Generating dashboards for data across profiles
• (23:08) Why conversion goals can be over 100%
• (25:14) Whats up with providing a podcast of web analytics TV?
• (25:53) An easy / automated way to track the download of PDF files
• (27:22) Using events to track more realistic time on site
• (29:11) Removing extra referrals from Google Analytics

Here are the links to the topics we discuss:

• Using regex to configure goals and filters in Google Analytics
• Using the BTBuckets / Google Analytics integration to target segments of users
• How to diagnose and troubleshoot data sent to Google Analytics
• 3rd party tools to test regular expressions
• Compare data across accounts with dashboards from Trakkboard
• Automate tracking download links using the Analytics Engine
• Removing extra referrals from Google Analytics

As always, if you need help setting up Google Analytics or leveraging the advanced configuration options, we recommend hiring a Google Analytics Certified Partner.

If you found this post or video helpful, we'd love to hear your comments. Please share them via the comment form below.

This series would not be possible without your awesome questions. Please submit them on our public Google Moderator site, and while you are there don’t forget to vote for your favorite questions. Avinash and Nick will answer them in a couple of weeks with yet another entertaining video.

Thanks!

Posted by Nick Mihailovski, Google Analytics Team

The New Google Analytics: Quick insights with Plot Rows

This is part of our series of posts highlighting the new Google Analytics. The new version of Google Analytics is currently available in beta to a small number of Analytics users. We’ll be giving access to more users soon. Sign up for early access.

The graph on top of most Google Analytics reports is designed to give you a quick overview of your site’s performance over time. From the graph it’s easy to spot trends and understand how your traffic has changed over time. One request we heard was the ability to quickly focus the graph on a particular row of data. While you could do this with a drill-down report or using an advanced segment, we saw this as an opportunity to provide an easy way to do quick comparisons in the new Google Analytics.

Say for example you’re examining your site’s traffic by traffic source. You can see there are peaks and valleys in the traffic, but if you want a sense of the major contributors, you need to dig into the table.

With Plot Rows, you can graph any two rows alongside the overview. You can then easily determine how much a row contributes to the whole. Or you can compare two lines against each other to look for comparison trends.

To use Plot Rows, just tick any one or two checkboxes next to the rows you want to plot, then at the bottom of the table, hits the Plot Rows button.

Remember, that some reports like New vs. Returning default to a Pie Chart view. This doesn’t mean you can’t use Plot Rows, just switch the view to Data, and you’re good to go.

Here’s a quick video showing this in action:

Usage Tips

When looking at continuous metrics, like Visits, Plot Rows is most revealing when exploring the rows of similar scale, for example to see how they contribute to the whole and change over time. When looking at rows at different scales the graph will be more informative when using percentage metrics like Bounce Rate.

In this example, we’re looking at organic search traffic driven to the Google Store from Google and Bing. One would not expect that Bing users are actively looking to buy Google merchandise (like this awesome t-shirt), so the number of visits is understandably low. Since the traffic from Bing is relatively low, the graph doesn’t share much we didn’t already know from the table.

In new version of Analytics you can quickly graph any of the metrics in the scorecard (the bar on top of the graph) by clicking on the metric in the scorecard. Looking at Bounce Rate, we can see that over time the Bounce Rate from Google search (orange) has dropped, which has reduced the overall Bounce Rate of the site (blue), while the Bounce Rate from Bing (green) has more or less stayed constant.

You can use Plot Rows in just about any report that has a data table. Let us know if you find a place you want this functionality that doesn’t already have it. Also, we’re planning to give a bunch more of you access to the new version this week. Be on the look out!

Posted by Trevor Claiborne, Google Analytics Team

Accessing More Than 10,000 Rows of Data Using The API

Did you know that you can easily get more than 10,000 rows of data from Analytics using the API?

Auto-Pagination through the API

Here is a quick overview on how to do auto-pagination with our API. You might also follow along by checking the fully working sample code in Python which you can use in your own applications.

The following sample query fetches the first 10,000 keywords by conversion rate for the month of February:

https://www.google.com/analytics/feeds/data
?ids=ga:12345
&dimensions=ga:keywords
&metrics=ga:conversionRateAll
&sort=-ga:conversionRateAll
&start-date=2011-02-01
&end-date=2011-02-28
&start-index=1
&max-results=10000

Notice how start-index is set to 1 and max-results is set to 10,000. When this query is issued to the API, the API will return up to 10,000 results. The API also returns the number rows found in Google Analytics in the openSearch:totalResult XML element

14,654

To get the total number of pages in this request, we can use the following python code:

num_pages = math.ceil(total_results / 10000)

Then getting the start-index for each additional page is trivial:

for page_number in range(num_pages)[1:]: # skips the first page.
start_index_for_page = page_number * 10000 + 1

Thats it! If you want to start doing this today, or just see how it should work, we’ve included a fully working example. If you liked this, and want to see more example, let us know what we should do next in our comments.

Posted by Nick Mihailovski, Google Analytics Team

The New Google Analytics: Dashboards

This is the first in a series of posts highlighting the new Google Analytics. The new version of Google Analytics is currently available in beta to a small number of Analytics users. We’ll be giving access to more users soon. Sign up for early access.

Today, we would like to introduce you to the new dashboards that is available in the new version of Google Analytics announced last week.

View the full-size

Custom Dashboards

For most Google Analytics users, the dashboard is the first thing you see when viewing your reports. Dashboards in the new version of Google Analytics have been redesigned to be completely widget-based and highly customizable. There are four types of widgets: Metric, Pie Chart, Timeline, and Table. This gives you the ability to choose the visualization that best suits the data you want in your dashboard. The Dashboard uses a three-column layout, and you can customize the layout by dragging and dropping the widgets as you’d like.

• Metric: Shows the value of a metric and a sparkline of that metric over the selected time period
• Pie Chart: Best suited for displaying breakdowns of a metric by a certain dimension. E.g., Visits by Browser Type.
• Timeline: A graph of any metric over time. You can also compare two metrics in the same graph.
• Table: Think of this as a mini-custom report. You can show one dimension with two metrics and up to 10 rows of data in a table.

Multiple Dashboards

A common request we heard from you all is that one dashboard isn’t enough. You told us that wanted the ability to customize multiple dashboards for different analyses. Thanks to the new Google Analytics platform, we’re happy to give you the power to create multiple custom dashboards, up to 20 per profile. You might start with an overall “Company KPIs” Dashboard that includes the most important performance indicators for your company, then create an “SEO” Dashboard for your search engine optimization efforts, and a “Content” Dashboard that centers around the content of your website. We’d love to hear what you do with the other 17 slots.

Going Forward

The core functionality for the new dashboards is ready for you to use, but this isn’t everything we’ve planned for dashboards in the new version. Because we are so excited about the new Dashboard, we decided to make its core features available to you first. For starters, we still need to bring some of the features from the current dashboard over, such as the ability to share, email, and export a Dashboard. We are actively working on them (along with many other improvements), so please stay tuned.

We hope you will find the Dashboards in the new Analytics easy and fun to use. Happy Dashboarding!

Posted by Yi Wang, Product Manager, Google Analytics Team

A fond farewell to Connection Speed

Google Analytics helps you measure how visitors arrive to your site. For example, by traffic source, geography, landing pages, and other dimensions. Unfortunately, we’re no longer able to offer data on one of these dimensions: Connection Speed, and we are now in the process of deprecating the Connection Speed dimension.

You will still be able to access historical connection speed data prior to February 25th, 2011 through the current version of Google Analytics and the Google Analytics API. The Connection Speed report in the current version of Analytics will be removed next week. However, you can continue to use Connection Speed as a dimension in custom reports. For the Google Analytics API, we plan to remove this dimension in three months. Again, this is only for historical data before February 25th, 2011.

If you rely on Connection Speed to understand the effect of page load time and latency, we’re working on a solution for you that we expect to have ready in the coming weeks.

Posted by Trevor Claiborne, Google Analytics Team

Issue with AdSense reports

Users of the AdSense reports inside of Google Analytics may have noticed that there have been discrepancies between the AdSense reports in Google Analytics and AdSense since March 16th. We’re aware of the issue and a fix is being rolled out at this time. The cause of the issue is not related to the new version of Google Analytics.

We’ll continue to update this post when the issue is resolved, and you can also follow along on the Google Analytics Status Dashboard.

Update: 3/22/2011 5:15pm PST - A fix for this issue was released early this morning. It will take a few more hours for the fix reach all users. AdSense reports should be working normally after that. Unfortunately, we are not able to reprocess the data that was affected by this issue.

Posted by Trevor Claiborne, Google Analytics Team

Looking towards the future of Google Analytics

Today at the Google Analytics User Conference in San Francisco, we shared a look at a new version of Google Analytics. We’ve also reached out to a small group of Analytics users to participate in the testing. If you’re part of this group you’ll see a link to the new version in your Analytics account. We’re starting small, and we’ll gradually roll the new version out to everyone.

Our goals for the new version are to make it easier and faster to get to the data you want and to enhance the Google Analytics platform to bring you major new functionality. Many of the changes in the new version are the result of your feedback. For example, you can now view multiple advanced segments without needing to also use All Visits. You’ll find some of the other most requested features like multiple dashboards in the new version as well.

We’ll be sharing many more details about what’s new along the way with a new series on the blog focused exclusively on the new Analytics. In the meantime, if you want to be considered for the test, visit the beta sign-up page. Using the new version won’t change how Google Analytics reports your website traffic, and you can continue to use the current version while testing the new Analytics.

As you start using the new Google Analytics, you should also check out the updated Google Analytics Help Center and the Report Finder, which will show you where to find your favorite reports in the new version. We’ve also set up a new category in the Help Forum where you can ask questions and discuss the new version.

This release wouldn’t have been possible without your feedback. Please continue to send us your feedback as you start using the new interface.

Happy analyzing!

UPDATE: 3/17/11 1:40pm PST, we fixed an issue that prevented us from collecting sign-ups. If you signed-up before 1:40pm PST, please sign-up again: beta sign-up page.

Posted by Trevor Claiborne, Google Analytics Team

Google's Data Viz Challenge

If you're reading this blog, chances are very high that you value data, and chances are also high that you appreciate the presentation of data. Going beyond a spreadsheet of rows and columns and creating a visualization is not only pleasing to the eye, but it can also help put meaning and context to the data and help surface important meanings. As practical as rows and columns of numbers are, sometimes a lot more can be divined a lot faster from a simple pie chart. Or a bar graph. Or an animated motion chart :-)

Data visualization is becoming an art and a discipline. There are blogs and books devoted to it. And we're obviously believers in visualizing data, and so is Google. That's why we're giving a shout out to another Google initiative, the Data Viz Challenge.

Google worked with a website called WhatWePayFor.com that shows the entire federal budget for the past several years, and breaks down spend by program, such as Social Security, national defense, transportation, medicare, etc., and how much is spent on each. It also allows you to enter an income amount and shows you how much of your own taxes goes to each program. Great data to make into a visualization.

WhatWePayFor has made their data available via an API to contestants who want to take a crack at visualizing this data, and Google is offering a $5,000 prize to the best visualization, as decided on by a panel of judges, as well as other prizes for cool entries. Deadline for entry is March 27, so get cracking - we want a Google Analytics data head to win this thing. :-)

Posted by Jeff Gillis, Google Analytics Team

Evolution of Analytics Benchmarking Report

This week, we are beginning a new way of providing benchmarking data to Google Analytics users. For almost three years, Google Analytics has provided a Benchmarking report for users who opt in for anonymous data sharing. In a few days, however, we will be removing the Benchmarking report from the Google Analytics interface, and replacing it with an expanded report that will be emailed directly to you.

In the coming few weeks, we’ll be delivering the new benchmarking report to all account administrators for accounts that are opted-in to anonymous data sharing. The report will expand on the type of data delivered to include broader trends, such as geographic or traffic source differences in visitor engagement. If you are interested in receiving the new benchmarking report, you can opt-in to anonymous data sharing.

1. Sign into your Google Analytics account
2. Select the account you want to opt-in from the account list
3. Select Edit account settings
4. Check the box next to Anonymously with Google and others

If you want one last look at the current reports make sure you visit them in the next few days. We’re looking forward to your feedback on the new format of the benchmarking report. Look out for it in the coming weeks.

Posted by Phil Mui, Google Analytics Team

Update: June, 2012: We will no longer be emailing this benchmarking report on a regular basis. We heard great feedback after our initial mailing and it turns out that a simple email is not able to meet the segmentation and analysis needs of our wide user base. We are exploring new ways to provide rich benchmarking data to our users in the future.

Client Side Authorization in the API

This post is inspired by Henrik who posted a comment on last week’s Mastering Unique Visitor blog post. His question was, “Most API users would not create web apps but rather easily access their data without account authentication. Is this possible? Could you describe it?”

Great question Henrik! While you do have to authenticate to the Google Analytics Data Export API in order to access data, your application doesn’t have to re-authenticate every time. In this post we show you an easy way to re-use tokens and give you a Python class to simplify the process.

(Note: we recommend you follow the actual sample code to see how all the pieces described below work together.)


Background
Just like you have to log in to view Google Analytics data, you also have to grant an application access to your Analytics data when you use the the API. This is called authorization and is done through the Google Accounts API.

Different Authorization Protocols
When you log into the Analytics interface, your login credentials are protected from being exposed. In the same way, you use the Google Accounts API to protect credentials by requesting an authorization token in place of actual user names and passwords. That authorization token is used in the header of each Google Analytics API request.

There are a two authorization routines that applications can choose in order to obtain this token:

Client Login - a legacy, Google proprietary, authorization routine. The user of your application must supply their user name and password in your application. Your application communicates to Analytics on the user’s behalf and retrieves an authentication token. This token only lasts for 14 days. The issue with this method is that if the token is stolen, others can use it and the user has no way to revoke it.

OAuth For Installed Apps - is Google-supported open standard. Client apps must send users to a web page where the user logs in and grants account access to the application. The web page presents the user with a verification code which they must input back into the application. The application can exchange this verification code for a long-lived token. The long-lived token lasts forever and can be revoked by the user through the Google Accounts interface. This routine is more secure because a user can revoke the token and prevent an application from accessing their data at any time.

Obviously OAuth For Installed Apps is a better way to generate authorization tokens.

Token Management
Because authorization tokens are valid for multiple requests, applications should reuse them until they become invalid. For a client-side app, this usually means saving the token to disk. Here’s the logic to handle this in python:

def GetAuthToken(self):
  if self.auth_token:
    return self.auth_token


  self.auth_token =
    self.auth_routine_util.LoadAuthToken(self.token_obj_name)


  if not self.auth_token:
    self.auth_token = self.RequestAuthToken()


  self.auth_routine_util.SaveAuthToken(self.auth_token)
  return self.auth_token

Here the code tries to load a token from disk. If it exists, the token is set in the client object that makes requests to Google Analytics. If the token does not exist, the code tries to obtain it from the Google Accounts API, then save it to disk...all pretty easy.

Retrieving a New OAuth Token

To get a new OAuth token, the code uses helper methods in the Python Client Library. The my_client parameter is a reference to the gdata.analytics.client.AnalyticsClient object provided by the Google Data Python Client Library. It handles all the logic to make requests to the Google Analytics API.

First, the my_client object is used retrieve the URL to the OAuth authorization end point. Next it retrieves a request_token from the Google Accounts API and use that token to generate a URL for the user to go to.

The URL is printed to the screen and the code tries to pop open a browser and take the user directly there. The user will be prompted to login and grant access to the application. Once complete, the web page gives the user a verification code.

The code prompts the user for the verification code, and once inputted, the application uses it to get a long-lived access token. This access token will be valid until the user revokes the token.

Here’s the sample code:

def GetAuthToken(self, my_client):
  url = ('%s?xoauth_displayname=%s' %
      (gdata.gauth.REQUEST_TOKEN_URL, self.my_client.source))


  request_token = self.my_client.GetOAuthToken(
      OAuthRoutine.OAUTH_SCOPES,
      next='oob',
      consumer_key='anonymous',
      consumer_secret='anonymous',
      url=url)


  verify_url = request_token.generate_authorization_url(
      google_apps_domain='default')


  print 'Please log in and/or grant access at: %s\n' % verify_url
  webbrowser.open(str(verify_url))


  request_token.verifier =
      raw_input('Please enter the verification code '
                'on the success page: ')


  try:
    return self.my_client.GetAccessToken(request_token)


  except gdata.client.RequestError, err:
    raise AuthError(msg='Error upgrading token: %s' % err)

Once we have the long-lived token, it’s returned and the GetAuthToken method above saves it to disk.

Making Google Analytics API Requests

To use the token retrieved above, we simply set it in the gdata.analytics.client.AnalyticsClient object:

try:
  my_client.auth_token = my_auth.GetAuthToken()


except auth.AuthError, error:
  print error.msg
  sys.exit(1)

At this point an application can use the client object to retrieve data from the Google Analytics API. Because a user can revoke a token from the Google Accounts API and the code loads the token from disk, there can be a case where the token on the disk is invalid. In this case, the Google Analytics API returns a 401 status code and the client library raises a gdata.client.Unauthorized exception. If this happens, we need to catch the exception, delete the saved token and prompt the user to get a new token.

try:
  feed = my_client.GetDataFeed(data_query)


except gdata.client.Unauthorized, error:
  print '%s\nDeleting token file.' % error
  my_auth.DeleteTokenFile()
  sys.exit(1)

So there you have it, a quick overview of client side authorization with the Google Analytics API. As we mentioned above, you can view and use the full sample code.

Submit your own questions
We’re thrilled to get feedback from our developers. You guys rock! If you have a question about our API, like Henrik, please leave it in our comments. We’ll go through them and try to highlight the code solutions.


Posted by Nick Mihailovski, Google Analytics API Team

Don’t miss the next Google Analytics User Conference in San Francisco

The next Google Analytics User Conference is coming up on Thursday, March 17th in San Francisco. Attend and you’ll connect with other Google Analytics users, industry experts, authors and even folks here from the Google Analytics team. GAUGE (as in Google Analytics Users’ Great Event) is being produced by several of our Google Analytics Certified Professionals, and will be collocated with eMetrics and Conversion Conference, which are being held the week of March 14. If you’re already planning to attend those shows (or need another reason to go), now you have an event focused solely on making the most of Google Analytics.

The event provides valuable insights through expert-led, user-to-user collaborative sessions that are insanely practical. This is not a “come and be spoken to” conference but rather an opportunity to learn from Google Analytics pros and peers and share your experience and knowledge.

Here are some highlights from the agenda:

• Hear from industry experts and Google Analytics team members, including:
• Keynote address by Justin Cutroni (author of 3 books on Google Analytics), blogger, and veteran GA consultant and user
• Presentation on What’s New with Google Analytics by Phil Mui, Group Product Manager for Google Analytics
• Sessions led by Nick Mihailovski of Google, Caleb Whitmore of Analytics Pros, John Hossack and Dave Eckman of VKI Studios, David Booth and Corey Kobergh of WebShare, Feras Ahlou of E-Nor, and Alex Langshur of PublicInsite
• Q&A Session with members of the Google Analytics engineering team
• An Analytics Helpdesk will be onsite where attendees can receive expert one-on-one help with GA during conference breaks
• Evening reception following close of the User Conference day
• Day 2 features three Google Analytics training seminars covering Google Analytics fundamentals, marketing analysis and analyst training, and technical/implementation training.

The list price is $495 per day or $895 for both the conference and training days. We’re giving readers of the Google Analytics blog a 10% discount on all registrations using the code: GBLOG10. Register at the event’s site: GAUGE: Google Analytics User Conference + Training Day. There’s also discounts available for non-profit and government workers. Contact gauge@analyticspros.com for details.

We’re looking forward to meeting many of you at GAUGE. See you soon!

Posted by Trevor Claiborne, Google Analytics team