Skip to main content

AWS S3 Integration: Define Your Own Schema

✗ Growth ✓ Enterprise

The S3 integration with Indicative is available for Enterprise customers only. If interested, please contact us. You are required to grant S3 access to Indicative by editing the IAM policy of an existing S3 bucket.

In order to perform the following steps you must have administrative access to the AWS Console as well as your S3 database.

If you are integrating Snowplow data via AWS S3, please use our Snowplow Integration documentation instead. 

Instructions

You are required to configure an IAM policy to grant Indicative programmatic access to the respective S3 bucket.

In Indicative

  1. Click on Settings and select Data Sources.
    projectsettings.png
  2. Click on New Data Source.
    datasources.png
  3. Select Amazon S3 and Define your own schema. Click Connectmceclip0.png
  4. You should see this screen.
    mceclip1.png
  5. Click Next. 

Connection Information

mceclip2.png

 

  1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
  2. For Source Format, select the file format of the data in your S3 bucket.
  3. For Bucket Name, enter the name of the S3 bucket that we should connect to. Click on the Show Me on the right to see how to get this information.
  4. For File Path, enter the file path corresponding to the data you want to use in Indicative. Click on the Show Me on the right to see how to get this information.
  5. Click Next

Grant Permissions

  1. In this section click on the box that contains the policy to copy to your clipboard. You will need to use this in step 4 of this section.
    mceclip1.png

  2. Go back to the AWS Console. Select the bucket and click on the Permissions tab.
    s2.png
  3. Click on Bucket Policy
    s3.png
  4. Enter the copied policy from step 1 into the editor and click Save
  5. Click Next in Indicative.

Event Modeling

mceclip5.png

  1. Events Field - enter the name of the field that should be used to derive your Indicative event names
  2. Timestamp - enter the name of the field that should be used for querying in Indicative.
  3. Click Next

User Modeling

mceclip6.png

For more information on User Identification (Aliasing), please refer to this article.

  1. If you choose to enable Aliasing:
    1. Unauthenticated ID - Input the field used to identify anonymous users.
    2. Authenticated ID - Input the field used to identify known users.
  2. I you choose to disable Aliasing, press Disabled:
    1. Unauthenticated ID - Enter the field used to identify your users. All users must have a value for this field.
  3.  Press Next

Scheduling

mceclip7.png

  1. Select the Schedule Interval to adjust the frequency at which new data is available in Indicative.
  2. Set the Schedule Time for when the data should be extracted from your BigQuery environment. It is critical that 100% of the data is available by this time to avoid loading partial data.
  3. Select Save

Waiting for Datamceclip3.png

Advanced Settings

For additional advanced settings such as excluding certain events and properties, please refer to this page

If you have any questions or concerns about the above Integration, please contact your Customer Support Manager, or email support@indicative.com.

Related articles