Databricks

​

Overview

• Export: Send OneSignal message events to Databricks for analytics and reporting.
• Import: Send custom events from Databricks to OneSignal to trigger Journeys and personalize campaigns.

​

Export OneSignal message events to Databricks

• Professional Plan or higher
• Custom Events enabled (for event imports)
• Databricks Platform: AWS, Azure, or GCP
• Databricks Plan: Premium or higher
• Databricks Unity Catalog (recommended for governance)
• Databricks SQL Warehouse for querying
• Delta Lake event tables (for custom event import)

​

1. Collect SQL warehouse details

1

Log in to your Databricks workspace

Go to SQL Warehouses in your Databricks workspace.

2

Select your warehouse

Select your warehouse and open the Connection details tab.

3

Save the following details

• Server Hostname
• Port
• HTTP Path

​

2. Choose authentication method

• OAuth M2M
• Personal Access Token

When to use:

• Direct connections (not using PrivateLink)
• Organizations with OAuth infrastructure

Note: Does not support PrivateLink connections.

​

Create a service principal

1

Go to the Service Principals page

Go to Workspace Settings > Identity and Access > Service Principals.

2

Add a new service principal

Click Add Service Principal, then Add New.

3

Name the service principal

Name it (e.g., onesignal-sync).

​

Generate a secret

1

Click into the created principal

2

Go to the Secrets tab

3

Generate a secret

Click Generate Secret and save it securely.

The secret is only visible once—store it safely.

When to use:

• Using PrivateLink (AWS or Azure)
• Simpler setup
• Maximum compatibility

Note: Works with all Databricks deployment types.

​

Create a Personal Access Token

1

Access Settings

In your Databricks workspace, click your username in the top bar and select Settings.

2

Navigate to Developer section

Click Developer.

3

Manage tokens

Next to Access tokens, click Manage.

4

Generate new token

Click Generate new token.

5

Configure token

• Enter a comment like “OneSignal Integration” to help identify this token
• Set the token’s lifetime in days (check your workspace documentation for maximum allowed duration)
• Click Generate

6

Save the token

Copy the displayed token to a secure location, then click Done.

Save the token securely and don’t share it. If you lose it, you must create a new token. Tokens that remain unused for 90 days are automatically revoked by Databricks.

​

3. Assign permissions

1

Navigate to your Catalog and open the Permissions tab

2

Click Grant

3

Assign the following permissions to the service principal or user

• USE CATALOG
• USE SCHEMA
• SELECT
• MODIFY
• CREATE SCHEMA
• CREATE TABLE

​

4. Connect OneSignal

1

Activate the integration

In OneSignal, navigate to Data > Integrations > Databricks.

2

Enter the details

• Server Hostname
• Port
• HTTP Path
• Catalog Name
• Schema Name
• Credentials

3

Configure the integration

• Sync Frequency: as often as every 15 minutes
• Dataset/Table Names: pre-set as onesignal_events_<app-id> and message_events (editable)
• Event Types: choose which to sync—select all or just what you need

4

Select events

Select the events you care to receive in your Databricks catalog.

5

Complete the setup

Click Save and wait for the success confirmation

​

5. View data in Databricks

1. Open your Catalog in Databricks.
2. Once syncing completes, your configured schema will appear.
3. Access and query the message_events table.
   
4. Click into tables for sample data preview.
   

​

Message events and properties

​

Message event kinds

​

Event data schema

​

Notes

• Syncs after saving/activating may take an additional 15-30 minutes to complete.
• Deactivating may still result in one final sync after deactivation.
• To ensure efficient data synchronization, our system automatically creates and manages staging datasets. These datasets, named with a pattern like fivetran_{two random words}_staging, temporarily store data during processing before it’s integrated into your main schema. These staging datasets are essential for maintaining a streamlined workflow and should not be deleted, as they will be automatically recreated.

​

Import events from Databricks

• Trigger Journeys based on user activity
• Personalize messaging based on behavioral data

• Access to Event Streams for outbound message events (Plan limitations and overages apply)
• Access to Custom Events for inbound event syncing (Plan limitations and overages apply)
• Updated Account Plan (not available on free apps)

• Databricks workspace with SQL Warehouse or compute cluster
• Personal Access Token with appropriate permissions
• Event data tables containing behavioral data in Delta Lake format
• Unity Catalog (recommended for data governance)

-- For Unity Catalog enabled workspaces
GRANT SELECT ON TABLE catalog.schema.event_table TO `[email protected]`;

-- For Hive metastore tables
GRANT SELECT ON TABLE database.event_table TO `[email protected]`;

​

Event data mapping

​

Advanced configuration

​

Unity Catalog Integration

SELECT
  event_name,
  user_id,
  event_timestamp,
  to_json(
    named_struct(
      'product_id', product_id,
      'purchase_amount', purchase_amount,
      'category', category
    )
  ) as payload
FROM catalog.schema.user_events
WHERE event_timestamp >= current_timestamp() - INTERVAL 7 DAYS

​

Delta Lake Optimization

• Partitioning: Partition by date (event_date) for faster time-based queries
• Z-Ordering: Z-order by user_id and event_name for better filtering
• Delta Lake Features: Use liquid clustering for automatic optimization

​

Streaming Event Processing

• Structured Streaming: Process events as they arrive
• Delta Live Tables: Build robust event processing pipelines
• Auto Loader: Continuously ingest new event files

​

FAQ

​

Why do I see different message IDs with the same content?

​

How often does OneSignal sync events from Databricks?

​

Can I use Databricks notebooks for event processing?

​

What about cost optimization for event queries?