Skip to main content

GCP BigQuery Export

Important: As of August 13, 2024, this page will no longer be actively maintained. Please refer to the current version of this content here.

 
✓ Enterprise
 

BigQuery as a Destination

Overview

Analytics for BigQuery is a fully managed data warehousing service that encapsulates the effort required to fully support, maintain and load your Analytics data into BigQuery. Because there is no infrastructure to manage, you can focus on analyzing data to find meaningful insights. The modern and infinitely scalable data warehouse provided by BigQuery combined with the enriched data from Analytics allows you to leverage additional value from your investment through a built-in SQL interface.

The BigQuery Export Integration provided by Analytics allows customers to easily export their raw Analytics data to BigQuery for further analysis via a SQL interface. This data do not undergo any additional enrichment, including identity resolution. The BigQuery Export Integration does not include derived properties, such as IP address or IP address based location information. The purpose of this section is to provide an overview of how Analytics loads raw data into BigQuery and what customers need to provide to configure and maintain the integration.

This document is intended for customers who are familiar with both Google BigQuery and the Analytics data model.  

Getting Started

In order to use the BigQuery Export Integration for Analytics, the customer must provide programmatic BigQuery access to the Analytics Platform. The customer is required to grant dataViewer BigQuery access to Analytics.

In order to perform the following steps you must have administrative access to the BigQuery Console as well as your BigQuery database.

Create a BigQuery Dataset (Optional)

  1. Open the BigQuery web UI in the GCP Console.

  2. In the navigation panel, in the Resources section, select your project.

  3. On the right side of the window, in the details panel, click Create dataset.

  4. On the Create dataset page:

    • For Dataset ID, enter a unique dataset name.

    • (Optional) For Data location, choose a geographic location for the dataset. If you leave the value set to Default, the location is set to US. After a dataset is created, the location can't be changed.

    • For Default table expiration, choose one of the following options:

      • Never: (Default) Tables created in the dataset are never automatically deleted. You must delete them manually.
      • Number of days after table creation: This value determines when a newly created table in the dataset is deleted. This value is applied if you do not set a table expiration when the table is created.

    • Click Create dataset.

Share Your Dataset with Analytics

1. In the menu panel along the left side of your BigQuery instance, under Resources, click on the triangle to the left of your project name to expand the view. This will display the available datasets.
 
2. Select a dataset from Resources, then click Share Dataset near the right side of the window.
 
share_dataset.png
 
3. In the Share dataset panel, in the Dataset permissions tab, click Add members.
 
4. In the Add members panel, enter " event-stream@indicative- production.iam. gserviceaccount.com " into the New members text box.
 
5. For Select a role, select BigQuery and choose bigquery.dataEditor.
 
6. Click Done.
 
7. Contact your account manager with the following information once you have granted Analytics 'bigquery.dataViewer' access:
  • ProjectId: the Analytics project ID to export
  • BQProject: the GCP project name to write data into
  • BQDataset: the BQ dataset name to write data into

Related articles