Skip to content

Data APIs and the backend-for-frontend pattern#

A data API serves platform data to applications through a defined HTTP interface instead of direct database access; a backend-for-frontend (BFF) is an API built for one specific application's screens.

Before reading

Semantic layer · Authentication and authorization: OIDC and JWT

Why it matters#

Applications increasingly show data produced by the platform: order history, recommendations, seller dashboards, account statistics. If each application queries the lake directly, it depends on table structures, holds engine credentials, sends unpredictable queries and gets analytical latency where users expect a fast page. An API in between gives applications a stable contract and gives the platform control over load, security and change.

How it works#

Two kinds of interface serve applications. A metrics API returns aggregates: revenue by day, order count by category. It is what a semantic layer offers. A data API returns records: the orders of one customer, the products of one seller, the profile of one account. Most application screens need records, filtered by the user, paged, and fast.

A data API has a few standard parts.

Endpoints are designed around use, not tables. GET /v1/customers/{id}/orders returns what the order history page needs, with stable field names, regardless of how many tables it is built from. The table structure can change behind the API without breaking the app.

Authentication and authorization follow the identity rules: the application sends the user's access token; the API verifies it and decides what the user may see from the token's claims, never from request parameters alone. The API passes the user's identity on, or applies the restrictions itself, so that row-level rules hold.

Pagination keeps responses bounded. Cursor-based pagination, returning a token for the next page, stays correct when data changes between requests, unlike page numbers with offsets.

Caching absorbs repeated requests. Responses that change rarely, such as last month's statistics, can be cached for minutes or hours; responses that must be current are not.

A data API usually reads from a serving store loaded from the platform, not from the lake, because lake engines answer in seconds with limited concurrency. The lake stays the source; the store is a copy shaped for the API. Options are covered in Low-latency serving stores.

The backend-for-frontend pattern goes one step further: one API per client application, owned by the team that builds that application. The mobile app's BFF returns exactly what its screens show, combining data from the platform's data API with other services. It keeps application-specific shaping out of the shared data API.

flowchart LR
  APP[Web app:<br/>order history page] -->|user token| BFF[Web BFF:<br/>shapes the page]
  MOB[Mobile app] -->|user token| BFF2[Mobile BFF]
  BFF --> API["Data API:<br/>GET /v1/customers/{id}/orders"]
  BFF2 --> API
  API --> SS[(Serving store:<br/>orders by customer)]
  SS -.->|loaded from| LAKE[(Lake: clean_orders)]
  BFF --> OTH[Other services:<br/>shipping status]

Key terms#

  • Data API An HTTP interface returning records from platform data.
  • Metrics API An interface returning aggregated metric values.
  • Backend-for-frontend (BFF) An API dedicated to one client application.
  • Serving store A database optimised for fast application queries, fed by the platform.
  • Cursor pagination Paging with a token that marks where the next page starts.
  • API contract The stable definition of endpoints and fields that clients rely on.

Example#

The shop's order history page. The web BFF calls the data API with the customer's token:

GET /v1/customers/c-88/orders?limit=20&cursor=eyJvcmRlcmVkX2F0Ijoi...
Authorization: Bearer eyJhbGciOi...

{
  "orders": [
    { "order_id": "o-1002", "ordered_at": "2026-10-28T09:14:00Z",
      "status": "cancelled", "amount": 129.00, "currency": "EUR" }
  ],
  "next_cursor": "eyJvcmRlcmVkX2F0IjoiMjAyNi0wOS0x..."
}

The API checks that the token's subject maps to customer c-88, or that the caller is a support staff member allowed to view other customers, and reads from a serving table keyed by customer_id, refreshed from clean_orders every hour. The page loads in 40 milliseconds; the same query on the lake engine took two seconds and competed with reporting workloads.

Pitfalls#

  • Applications querying the lake directly. They couple to table structures and bring unpredictable load and latency.
  • Endpoints that mirror tables. Every table change then becomes an API change.
  • Trusting IDs in the request. The token, not the URL, decides whose data is returned.
  • Offset pagination on changing data. Records are skipped or repeated between pages.
  • One giant shared API for every client. Application-specific shaping belongs in a BFF.

Learn more#

Low-latency serving stores Multi-tenancy Reverse ETL and activation Agents and MCP BI and dashboards