SAP to Databricks ingestion tools comparison

Data Ingestion Tools Comparison

icon
Feel free to contact us so we can help you select the best tool for your use-cases.

There are two main ways to get data out of an SAP ERP system reliably:

1. Environments, setup

β€£
Run options
β€£
Installation needed:
NameEnvironmentRun optionsInstallation neededIndependent of SAP SLT

Azure only

SaaS + agent

integration runtime needs to be installed on an on-premises computer or on a virtual machine (VM).

β˜‘οΈ (SLT is recommended and needed for SAP tables extraction)

SNP Glue

Azure, AWS, GCP

on-prem

installed as ABAP add-on on ERP server (no additional hardware required)

βœ… (but can work with SLT if already in place)

Qlik Replicate

Azure, AWS, GCP

on-premSaaS + agent

full app installation (on-prem version)

βœ… (see release log)

Azure, AWS, GCP

on-premSaaS

full app installation (on-prem version, Kubernetes)

AWS only

SaaS

❌ (fully SaaS)

β˜‘οΈ

GCP only

SaaS

❌ (fully SaaS)

β˜‘οΈ

Asapio

Azure, AWS

on-prem

βœ…

βœ…

SAP Datasphere

Azure, AWS, GCP

SaaS

βœ…

βœ…

2. Costs

β€£
Pricing
β€£
Free of additional costs
NamePricing / 100 sourcesFree of additional costsIndependent of SAP SLT

# of runs, per hour of run

❌ (integration runtime needs additional HW)

β˜‘οΈ (SLT is recommended and needed for SAP tables extraction)

SNP Glue

# of ingestion pipelines/ingested tables (tiered)

βœ… (installed as ABAP addon onto an existing SAP Netweaver machine)

βœ… (but can work with SLT if already in place)

Qlik Replicate

❌ (installation needs additional HW)

βœ… (see release log)

# of capacity units / month

❌ (installation needs additional HW)

# of successful runs + amount of data processed

βœ… (it’s SaaS)

β˜‘οΈ

paying for pipeline development and execution

βœ… (it’s SaaS)

β˜‘οΈ

SAP Datasphere

~$60k 1 + $5k/20 GBs outbound data transfer

βœ…

βœ…

Asapio

# of ingestion pipelines/ingested tables (tiered)

βœ…

βœ…

3. Historical SAP data extraction

NameHistorical data load (primary system)Filtered historical load

via the SAP Application Layer

❌ (source)

SNP Glue

utilize standard SAP select options

Qlik Replicate

βœ…

βœ…

ODP via oData only (slower)

SAP Datasphere

βœ…

βœ…

Asapio

βœ…

βœ…

4. Continuous SAP data extraction

β€£
Real-time support
β€£
SAP Extractors, CDS Views support
β€£
BW data extraction support
β€£
New records (deltas) processing for CDSViews/HANA Calculation views
NameReal-time supportExtractors supportBW data extraction supportNew records (deltas) processing for CDSViews/HANA Calculation viewsApplication layer extractionDatabase layer extraction

❌ (batches only, every 5 minutes)

βœ…

βœ… (via ODP)

βœ…

βœ…

βœ… (via SLT and ODP)

SNP Glue

βœ… (near-real time)

βœ…

βœ…

βœ… (via ODP)

βœ…

βœ… (via triggers)

Qlik Replicate

βœ… (log-based + trigger based)

βœ…

βœ… (via ODP)

βœ… (for the ODP connector only?)

βœ…

βœ… (via CDC, or via triggers for HANA)

βœ…

βœ…

βœ…

βœ…

βœ…

βœ…

❌ (micro-batches only)

βœ…

βœ… (via ODP)

βœ… (ODP via OData)

❌ (ODP only)

βœ… (via SLT)

βœ…

βœ… (via ODP)

βœ…

βœ… (via SLT)

SAP Datasphere

βœ…

βœ…

βœ…

βœ…

βœ…

βœ…

Asapio

βœ…

❌

❌

βœ…

βœ…

βœ… (via triggers)

5. Writing data to target

β€£
Schema conversion
β€£
DeltaLake support
NameSchema conversionDeltaLake support

βœ…

βœ… (source)

SNP Glue

βœ…

βœ…

Qlik Replicate

βœ… (source)

βœ… (source)

❌

❌ (custom connector needed)

❌ (parquet or CSV only)

SAP Datasphere

βœ…

βœ…

Asapio

❌ (CSV only)

How does the incoming data processing work?

The data ingestion tool generates smaller (real-time/micro-batching) or larger (hourly/daily processing) batches of data that are written into CSV, parquet, delta or some other format and then written into the cloud storage (AWS S3, Azure Blob Storage, GCP Cloud Storage).

As soon as the file (P&L planning data of the diagram) lands in the bronze/landing zone of the data lake, Databricks can pick the file and append it to the overall P&L planning table stored in the silver layer.

image
icon
Feel free to contact us so we can help you select the best tool for your use-cases.

← Return to Data Ingestion

Application-level SAP data replicationDatabase-level SAP data replication