SAP to Databricks ingestion tools comparison

🏠
Home

Data Ingestion Tools Comparison

icon
Feel free to contact us so we can help you select the best tool for your use-cases.

There are two main ways to get data out of an SAP ERP system reliably:

  • Database-level data replication
  • Application-level data replication.

1. Environments, setup

β€£
Run options
β€£
Installation needed:
Name
Environment
Run options
Installation needed
Independent of SAP SLT
Azure DataFactory

Azure only

SaaS + agent

integration runtime needs to be installed on an on-premises computer or on a virtual machine (VM).

β˜‘οΈ (SLT is recommended and needed for SAP tables extraction)

SNP Glue

Azure, AWS, GCP

on-prem

installed as ABAP add-on on ERP server (no additional hardware required)

βœ… (but can work with SLT if already in place)

Qlik Replicate

Azure, AWS, GCP

on-premSaaS + agent

full app installation (on-prem version)

βœ… (see release log)

SAP Data Intelligence

Azure, AWS, GCP

on-premSaaS

full app installation (on-prem version, Kubernetes)

AWS AppFlow

AWS only

SaaS

❌ (fully SaaS)

β˜‘οΈ

Cloud Data Fusion

GCP only

SaaS

❌ (fully SaaS)

β˜‘οΈ

Asapio

Azure, AWS

on-prem

βœ…

βœ…

SAP Datasphere

Azure, AWS, GCP

SaaS

βœ…

βœ…

2. Costs

β€£
Pricing
β€£
Free of additional costs
Name
Pricing / 100 sources
Free of additional costs
Independent of SAP SLT
Azure DataFactory

# of runs, per hour of run

❌ (integration runtime needs additional HW)

β˜‘οΈ (SLT is recommended and needed for SAP tables extraction)

SNP Glue

# of ingestion pipelines/ingested tables (tiered)

βœ… (installed as ABAP addon onto an existing SAP Netweaver machine)

βœ… (but can work with SLT if already in place)

Qlik Replicate

❌ (installation needs additional HW)

βœ… (see release log)

SAP Data Intelligence

# of capacity units / month

❌ (installation needs additional HW)

AWS AppFlow

# of successful runs + amount of data processed

βœ… (it’s SaaS)

β˜‘οΈ

Cloud Data Fusion

paying for pipeline development and execution

βœ… (it’s SaaS)

β˜‘οΈ

SAP Datasphere

~$60k 1 + $5k/20 GBs outbound data transfer

βœ…

βœ…

Asapio

# of ingestion pipelines/ingested tables (tiered)

βœ…

βœ…

3. Historical SAP data extraction

Name
Historical data load (primary system)
Filtered historical load
Azure DataFactory

via the SAP Application Layer

❌ (source)

SNP Glue

utilize standard SAP select options

Qlik Replicate

βœ…

βœ…

SAP Data Intelligence

AWS AppFlow

ODP via oData only (slower)

Cloud Data Fusion

SAP Datasphere

βœ…

βœ…

Asapio

βœ…

βœ…

4. Continuous SAP data extraction

β€£
Real-time support
β€£
SAP Extractors, CDS Views support
β€£
BW data extraction support
β€£
New records (deltas) processing for CDSViews/HANA Calculation views
Name
Real-time support
Extractors support
BW data extraction support
New records (deltas) processing for CDSViews/HANA Calculation views
Application layer extraction
Database layer extraction
Azure DataFactory

❌ (batches only, every 5 minutes)

βœ…

βœ… (via ODP)

βœ…

βœ…

βœ… (via SLT and ODP)

SNP Glue

βœ… (near-real time)

βœ…

βœ…

βœ… (via ODP)

βœ…

βœ… (via triggers)

Qlik Replicate

βœ… (log-based + trigger based)

βœ…

βœ… (via ODP)

βœ… (for the ODP connector only?)

βœ…

βœ… (via CDC, or via triggers for HANA)

SAP Data Intelligence

βœ…

βœ…

βœ…

βœ…

βœ…

βœ…

AWS AppFlow

❌ (micro-batches only)

βœ…

βœ… (via ODP)

βœ… (ODP via OData)

❌ (ODP only)

Cloud Data Fusion

βœ… (via SLT)

βœ…

βœ… (via ODP)

βœ…

βœ… (via SLT)

SAP Datasphere

βœ…

βœ…

βœ…

βœ…

βœ…

βœ…

Asapio

βœ…

❌

❌

βœ…

βœ…

βœ… (via triggers)

5. Writing data to target

β€£
Schema conversion
β€£
DeltaLake support
Name
Schema conversion
DeltaLake support
Azure DataFactory

βœ…

βœ… (source)

SNP Glue

βœ…

βœ…

Qlik Replicate

βœ… (source)

βœ… (source)

SAP Data Intelligence

❌

❌ (custom connector needed)

AWS AppFlow

❌ (parquet or CSV only)

Cloud Data Fusion

SAP Datasphere

βœ…

βœ…

Asapio

βœ…

How does the incoming data processing work?

The data ingestion tool generates smaller (real-time/micro-batching) or larger (hourly/daily processing) batches of data that are written into CSV, parquet, delta or some other format and then written into the cloud storage (AWS S3, Azure Blob Storage, GCP Cloud Storage).

As soon as the file (P&L planning data of the diagram) lands in the bronze/landing zone of the data lake, Databricks can pick the file and append it to the overall P&L planning table stored in the silver layer.

image
icon
Feel free to contact us so we can help you select the best tool for your use-cases.

← Return to Data Ingestion

Application-level SAP data replicationDatabase-level SAP data replication