Eviden’s SAP Data Model for Databricks
We’ve built our SAP Accelerator as a foundational SAP BW-like data model in Databricks for business intelligence and machine learning use-cases. The model can also be easily combined with non-SAP data sources and models.
Packages
Financial Suite
- General Ledger
- Accounts Payable
- Accounts Receivable
- Controlling and Profitability
- Asset management
Operational Suite
- Sales
- Delivery
- Billing
- Shipments
HR & TPM
- Employee Information
- Employee Attendance
- Payroll
- Trade Promotion
- Vehicle Utilization
Supply Chain
- Invoice Verification
- Procurement
- Inventory Management
High-level architecture of the Eviden’s SAP Accelerator
SAP data processing steps:
‣
1. Data Ingestion
- Ingest data from primary SAP systems using any of the popular ingestion tool:
- either directly from the database tables (low-level data, real-time/fastest)
- or using BW extractors (higher-level data model, no real-time, slower for bigger data)
- Persist the ingested data into your Lakehouse
‣
2. SAP BW-like data model initialization
🥉 Bronze
All raw tables are loaded without any transformation.
🥈Silver
Converting field names to readable form
Data types casting
Dimensions & Facts modeling
Aggregate tables creation
Hierarchies flattening
🥇Gold
Views for specific use-cases/reports
Features for ML models & predictions
Pre-built KPI application
‣
3. Data model customization
Convert data from SAP-specific format into relational databases compatible format
- refactor SAP recursions, for loops into a standard relational data model (star schema)
- migrate data types - currency shifting, …
‣
4. Pre-built ML models customization
- Late Payment Prediction
- Stock in Transit material overdue
- Supplier Delivery Delay
- … more to come soon
DEMO of SAP-like data structures in Databricks
Demonstration repository
The demonstration repository of our SAP accelerator’s Order to Cash module can be found at https://github.com/DataSentics/sap-order-to-cash
Technology stack
- business logic & transformations to produce tables in the Silver + Gold layers
- dbt-powered data models
- orchestrated using Databricks Workflows
- full Unity Catalog integration including data lineage
- CI/CD templates (GitHub, dbx for deployment)