Skip to main content

Introduction

With MoEngage Event Exports, you can forward events to various third-party tools. We offer the following capabilities:
  1. MoEngage Streams - Streams allow you to forward events like user actions, campaign interaction events, and so on available in MoEngage to a pre-defined API endpoint in near real-time.
  2. Data Warehouses - MoEngage supports multiple data warehouses as destinations to forward events directly to your tables. We support:
    1. Amazon Redshift
    2. Google BigQuery (via GCS)
    3. Snowflake
  3. Cloud Storages - MoEngage also supports multiple cloud storages as destinations to forward events as JSON files in your buckets.
    1. Amazon S3
    2. Google Cloud Storages
    3. SFTP

Export Schema

The export schema depends on the type of destination. Each category (Data Warehouse, Cloud Storages, APIs) has a consistent schema to account for various MoEngage Events and User-Defined Events.

Streams (API)

MoEngage will make a Bulk API call to your provided API Endpoint. Here is the generic structure: JSON

Schema

Event Object Schema

Event Attributes Object

The event attributes might vary according to the event being exported. Refer to the event definitions below to see which attributes are exported.

User Attributes Object

The user attributes might vary according to your configuration. Refer to the user properties definitions below to see which attributes are exported.
  • moengage_user_id - This attribute is always exported. This is the internal MoEngage ID of your user.

Device Attributes Object

The device attributes might vary according to your configuration. Refer to the device properties definitions below to see which attributes are exported.
  • moengage_device_id - This attribute is always exported. This is the internal Device ID of your user.

Data Warehouses Destinations

MoEngage will create a single table in your Data warehouse and export all your selected events to that table.

Amazon Redshift

We will create a table called moengage_events.

Snowflake

We will create a table called moe_events_<table_creation_timestamp>.

Table Structure

The overall structure of the table will remain fixed for all types of events. Any new user property or event attribute that is exported will be part of the user_attributes or event_attributes column respectively. All the events will be dumped in a single table containing all necessary information such as event names, event times, etc for easy analysis. We follow the below generic schema:
  • Any column beginning with event_attributes_ contains an event attribute. This will be followed by the event attribute name like event_attributes_campaign_name.
  • Any column beginning with user_attributes_ contains a user attribute. This will be followed by the user attribute name like user_attributes_moengage_user_id.
  • Any column beginning with device_attributes contains a device attribute. This will be followed by the device attribute name like device_attributes_moengage_device_id.
  • We also have three JSON (or equivalent) fields called event_attributes, user_attributes, and device_attributes. Each of these fields contains raw JSON fields in the form of a dictionary.

Google BigQuery (Generic)

MoEngage exports events to your BigQuery tables via GCS Buckets using the BigQuery Transfer File service. There are two ways you can create a table for your events in BigQuery:
  1. Create a table specific to your exported events by using the schema of your exported files.
  2. Create a table that has a generic schema.
If you want to create a generic schema and are unsure of your events, you can use this schema: JSON

Cloud Storages Destinations

MoEngage will export compressed files to your cloud storage.

Generic Export Schema

  • File format: .json.gz
  • Each uncompressed file will have newline-delimited events in JSON format (each line is a single event).
  • Each file can have multiple types of events in them. We don’t create a single file for each type of event.
  • The event JSON format is described below:
JSON

Schema

Event Object Schema

Event Attributes Object

The event attributes might vary according to the event being exported. Refer to the event definitions below to see which attributes are exported.

User Attributes Object

The user attributes might vary according to your configuration. Refer to the user properties definitions below to see which attributes are exported.
  • moengage_user_id - This attribute is always exported. This is the internal MoEngage ID of your user.

Device Attributes Object

The device attributes might vary according to your configuration. Refer to the device properties definitions below to see which attributes are exported.
  • moengage_device_id - This attribute is always exported. This is the internal Device ID of your user.

Amazon S3

MoEngage will create a pre-defined file path inside the folder you specify while setting up the S3 Exports. We will dump files in the following locations: s3://<your_bucket_name>/[<folder_path>_]<connection_name>/export_day=<DD-MM-YYYY>/event_exports_<export_timestamp>.json.gz

Google Cloud Storage

MoEngage will create a pre-defined file path in your GCS Bucket. We will dump files in the following locations: gs://<your_bucket_name>/event-exports/<your_app_name>/<connection_id>/export_day=YYYY-MM-DD/export_hour=HH/<file-name>.json.gz

SFTP

MoEngage will create a pre-defined file path on your SFTP Server. We will dump files in the following locations: sftp://<your_host_name>/[<folder_path>/]<connection_name>/export_day=<DD-MM-YYYY>/event_exports_<export_timestamp>.json.gz

Events Exported

All MoEngage-tracked attributes start with moe_.

User Attributes

Device Attributes

These are properties derived from your users’ device data. Note that these attributes will still be shown under user attributes in your exports.

MoEngage (Standard) Events

MoEngage, by default, tracks a few standard events.

Common Event Attributes

These attributes are automatically tracked when MoEngage captures a standard event.

Standard Events

These events are automatically tracked with MoEngage.

User-Defined (Custom) Events

Common Event Attributes

These attributes are automatically tracked when you track a custom/user-defined event.