Why
Shared applications — data lakes, ML platforms, event buses, shared API gateways, container registries — are actively consumed by multiple teams with varying usage. A data lake that is 80% populated by one team should not be split evenly. Without proportional allocation, consuming teams have no incentive to optimise their usage. Shared applications can represent 15–30% of total cloud spend.
What
Implement proportional cost allocation for shared applications using a measurable usage driver (storage consumed, queries executed, API calls, compute hours). This requires instrumenting usage metrics and building allocation logic in the billing pipeline.
How
Inventory Shared Applications
List all accounts/subscriptions/projects running shared applications consumed by multiple teams. For each, identify the cost and the consuming teams.
Select a Usage Driver per Application
| Shared Application | Recommended Driver | How to Measure |
|---|---|---|
| Data lake | Storage volume per team | S3/GCS/ADLS metrics by prefix/path |
| ML platform | Compute hours per team | Job logs, namespace labels |
| Event bus (Kafka) | Messages produced/consumed | Kafka metrics per topic/consumer |
| Shared API gateway | API calls per consumer | API key or header-based attribution |
| Container registry | Image pulls per team | Registry access logs |
Decision framework:
Do you have per-consumer usage metrics?
YES → Usage-based split (fairest)
NO → Is the cost > $10K/month?
YES → Invest in telemetry, then switch
NO → Spend-based or even split (good enough)Build the Allocation Logic
Formula: Allocated Cost per BU = Total Shared Cost × (BU Usage / Sum of All Usage)
Option A — Cloud-native tools:
| Provider | Tool | Approach |
|---|---|---|
| AWS | Cost Categories | Split Charge Rule with proportional weights |
| Azure | Cost Allocation Rules | Source subscriptions → target subscriptions |
| GCP | BigQuery SQL / dbt | Custom allocation model in billing pipeline |
Option B — Custom pipeline (for usage-based):
Build a scheduled job (Lambda, Azure Function, Cloud Function, or dbt model) that:
- Pulls usage metrics per team from telemetry (CloudWatch, Azure Monitor, BigQuery)
- Calculates proportional weights
- Writes allocation records into the billing data store
- Updates showback dashboards
Validate and Publish
Run the allocation model for one month in parallel with current reporting. Compare outputs. Present to consuming teams for sign-off before switching showback reports.
Deliverable Checklist
- Shared applications inventoried with consuming teams
- Usage driver selected per application
- Telemetry instrumented for usage measurement
- Allocation logic implemented (cloud-native or custom pipeline)
- One month parallel run validated
- Consuming team sign-off obtained