Databricks

OneSignal Databricks integration overview

​

Overview

• Export OneSignal message events to Databricks for analytics, dashboards, and reporting.
• Import custom events from Databricks to OneSignal to trigger Journeys and personalized campaigns.

​

Export OneSignal events to Databricks

• Professional Plan
• Custom Events enabled (for event imports)
• Databricks Platform: AWS, Azure, or GCP
• Databricks Plan: Premium or higher
• Databricks Unity Catalog (recommended for governance)
• Databricks SQL Warehouse for querying
• Delta Lake event tables (for custom event import)

​

1. Collect SQL warehouse details

1

Log in to your Databricks workspace

Go to SQL Warehouses in your Databricks workspace.

2

Select your warehouse

Select your warehouse and open the Connection details tab.

3

Save the following details

• Server Hostname
• Port
• HTTP Path

Databricks SQL connection details for Fivetran setup

​

2. Create a service principal

1

Go to the Service Principals page

Go to Workspace Settings > Identity and Access > Service Principals.

2

Add a new service principal

Click Add Service Principal, then Add New.

3

Name the service principal

Name it (e.g., onesignal-sync).

Modal for adding a service principal, with the 'Add new' option highlighted

​

3. Generate a secret

1

Click into the created principal

2

Go to the Secrets tab

3

Generate a secret

Click Generate Secret and save it securely.

The secret is only visible once—store it safely.

Databricks 'Generate secret' modal showing OAuth secret and client ID for API authentication

​

4. Assign permissions

1

Navigate to your Catalog and open the Permissions tab

2

Click Grant

3

Assign the following permissions to the service principal

• USE CATALOG
• USE SCHEMA
• SELECT
• MODIFY
• CREATE SCHEMA
• CREATE TABLE

Privilege assignment screen for a Databricks principal with custom catalog permissions selected.

​

5. Connect OneSignal

1

Activate the integration

In OneSignal, navigate to Data > Integrations > Databricks.

2

Enter the details

• Server Hostname
• Port
• HTTP Path
• Catalog Name
• Schema Name
• Service Principal credentials (ID + Secret)

OneSignal Databricks Configuration form with fields for catalog, hostname, HTTP path, and OAuth credentials.

3

Configure the integration

• Sync Frequency: as often as every 15 minutes
• Dataset/Table Names: pre-set as onesignal_events_<app-id> and message_events (editable)
• Event Types: choose which to sync—select all or just what you need

4

Select events

Select the events you care to receive in your Databricks catalog.

OneSignal event export settings screen showing sync status, dataset configuration, and selected message event types.

5

Complete the setup

Click Save and wait for the success confirmation

​

6. View data in Databricks

1. Open your Catalog in Databricks.
2. Once syncing completes, your configured schema will appear.
3. Access and query the message_events table.
   

   Databricks Catalog view showing OneSignal message events table under a production schema.
4. Click into tables for sample data preview.
   

   Sample data from the message_events_1 table with synced OneSignal event fields.

​

Message events and properties

​

Message event kinds

​

Event data schema

​

Notes

• Syncs after saving/activating may take an additional 15-30 minutes to complete.
• Deactivating may still result in one final sync after deactivation.
• To ensure efficient data synchronization, our system automatically creates and manages staging datasets. These datasets, named with a pattern like fivetran_{two random words}_staging, temporarily store data during processing before it’s integrated into your main schema. These staging datasets are essential for maintaining a streamlined workflow and should not be deleted, as they will be automatically recreated.

​

Import events from Databricks

• Trigger Journeys based on user activity
• Personalize messaging based on behavioral data

• Access to Custom Events (currently in beta)
• Updated Account Plan (not available on free apps).

• Databricks workspace with SQL Warehouse or compute cluster
• Personal Access Token with appropriate permissions
• Event data tables containing behavioral data in Delta Lake format
• Unity Catalog (recommended for data governance)

-- For Unity Catalog enabled workspaces
GRANT SELECT ON TABLE catalog.schema.event_table TO `onesignal@yourdomain.com`;

-- For Hive metastore tables
GRANT SELECT ON TABLE database.event_table TO `onesignal@yourdomain.com`;

​

Event data mapping

​

Advanced configuration

​

Unity Catalog Integration

SELECT
  event_name,
  user_id,
  event_timestamp,
  to_json(
    named_struct(
      'product_id', product_id,
      'purchase_amount', purchase_amount,
      'category', category
    )
  ) as payload
FROM catalog.schema.user_events
WHERE event_timestamp >= current_timestamp() - INTERVAL 7 DAYS

​

Delta Lake Optimization

• Partitioning: Partition by date (event_date) for faster time-based queries
• Z-Ordering: Z-order by user_id and event_name for better filtering
• Delta Lake Features: Use liquid clustering for automatic optimization

​

Streaming Event Processing

• Structured Streaming: Process events as they arrive
• Delta Live Tables: Build robust event processing pipelines
• Auto Loader: Continuously ingest new event files

​

FAQ

​

Why do I see different message IDs with the same content?

​

How often does OneSignal sync events from Databricks?

​

Can I use Databricks notebooks for event processing?

​

What about cost optimization for event queries?