Skip to content

Further reading#

The books, specifications and articles cited on the topic pages, grouped by section. Generated by build_reference.py from each page's Learn more list.

01 Foundations#

  • Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 3 (storage and retrieval). A 2nd edition appeared in 2026; chapter numbers differ. (from OLTP vs OLAP)
  • Ralph Kimball and Margy Ross, The Data Warehouse Toolkit, chapter 1. (from OLTP vs OLAP)
  • Armbrust et al., Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, CIDR 2021: https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf (from Data lake, warehouse and lakehouse)
  • Apache Iceberg, table spec overview: https://iceberg.apache.org/spec/ (from Data lake, warehouse and lakehouse)
  • Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 8 (queries, modeling and transformation). (from ETL, ELT and EtLT)
  • dbt Labs, Extract, load, transform: https://www.getdbt.com/blog/extract-load-transform (from ETL, ELT and EtLT)
  • Tyler Akidau, Streaming 101: https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/ (from Processing patterns: batch, micro-batch and streaming)
  • Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapters 10 and 11. A 2nd edition appeared in 2026; chapter numbers differ. (from Processing patterns: batch, micro-batch and streaming)
  • Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 2 (schema flexibility in the document model). A 2nd edition appeared in 2026; chapter numbers differ. (from Schema-on-write and schema-on-read)
  • Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 5 (data generation in source systems: schemas). (from Schema-on-write and schema-on-read)
  • Databricks, What is a medallion architecture: https://www.databricks.com/glossary/medallion-architecture (from Medallion layers: bronze, silver and gold)
  • Databricks documentation, medallion lakehouse architecture: https://docs.databricks.com/aws/lakehouse/medallion (from Medallion layers: bronze, silver and gold)
  • Zhamak Dehghani, Data Mesh Principles and Logical Architecture: https://martinfowler.com/articles/data-mesh-principles.html (from Data products and data mesh)
  • Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale (O'Reilly, 2022). (from Data products and data mesh)
  • Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 1, the data engineer and other roles. (from Data team roles)
  • dbt Labs, What is analytics engineering: https://www.getdbt.com/what-is-analytics-engineering (from Data team roles)

02 Storage#

03 Ingestion#

  • Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 7, ingestion. (from Ingestion patterns)
  • Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 11, change data capture. (from Ingestion patterns)
  • Apache Kafka documentation (design and Kafka Connect sections): https://kafka.apache.org/documentation/ (from Kafka and Kafka Connect)
  • Apache Kafka 4.0 release announcement: https://kafka.apache.org/blog/2025/03/18/apache-kafka-4.0.0-release-announcement/ (from Kafka and Kafka Connect)
  • Jay Kreps, The Log: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying (from Kafka and Kafka Connect)
  • Debezium documentation, PostgreSQL connector: https://debezium.io/documentation/reference/stable/connectors/postgresql.html (from Change data capture)
  • Debezium documentation, new record state extraction: https://debezium.io/documentation/reference/stable/transformations/event-flattening.html (from Change data capture)
  • Apache Kafka documentation (message delivery semantics section): https://kafka.apache.org/documentation/ (from Delivery guarantees and idempotency)
  • Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 11, fault tolerance and exactly-once. (from Delivery guarantees and idempotency)
  • Tyler Akidau, Streaming 102: https://www.oreilly.com/radar/the-world-beyond-batch-streaming-102/ (from Late and out-of-order data)
  • Apache Flink documentation, timely stream processing: https://nightlies.apache.org/flink/flink-docs-stable/docs/concepts/time/ (from Late and out-of-order data)
  • Confluent documentation, schema evolution and compatibility: https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html (from Schema evolution)
  • Apache Iceberg, schema evolution: https://iceberg.apache.org/docs/latest/evolution/ (from Schema evolution)
  • CloudEvents specification: https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md (from Event design)
  • Martin Fowler, What do you mean by "Event-Driven"?: https://martinfowler.com/articles/201701-event-driven.html (from Event design)
  • Apache Iceberg, Kafka Connect: https://iceberg.apache.org/docs/latest/kafka-connect/ (from Writing to Iceberg from Kafka)
  • Apache Iceberg, Flink writes: https://iceberg.apache.org/docs/latest/flink-writes/ (from Writing to Iceberg from Kafka)
  • Chris Richardson, Pattern: Transactional outbox: https://microservices.io/patterns/data/transactional-outbox.html (from The outbox pattern)
  • Debezium documentation, outbox event router: https://debezium.io/documentation/reference/stable/transformations/outbox-event-router.html (from The outbox pattern)
  • OASIS, MQTT version 5.0 specification: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html (from IoT and device data)
  • MQTT.org, MQTT overview: https://mqtt.org/ (from IoT and device data)
  • Apache Flink documentation, stateful stream processing: https://nightlies.apache.org/flink/flink-docs-stable/docs/concepts/stateful-stream-processing/ (from Stream processing)
  • Apache Flink documentation, window aggregation in SQL: https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/window-agg/ (from Stream processing)
  • Tyler Akidau, Slava Chernyak and Reuven Lax, Streaming Systems (O'Reilly, 2018). (from Stream processing)

04 Modelling and transformation#

  • Trino documentation, overview and concepts: https://trino.io/docs/current/overview/concepts.html (from SQL query engines)
  • Apache Spark SQL guide: https://spark.apache.org/docs/latest/sql-programming-guide.html (from SQL query engines)
  • DuckDB documentation: https://duckdb.org/docs/ (from SQL query engines)
  • Kimball Group, dimensional modeling techniques: https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/ (from Dimensional modelling)
  • Ralph Kimball and Margy Ross, The Data Warehouse Toolkit, 3rd edition (Wiley, 2013). (from Dimensional modelling)
  • Trino documentation, MERGE: https://trino.io/docs/current/sql/merge.html (from Deduplication and merge)
  • Apache Iceberg, Spark writes (MERGE INTO): https://iceberg.apache.org/docs/latest/spark-writes/ (from Deduplication and merge)
  • Kimball Group, slowly changing dimension techniques: https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/type-2/ (from History and slowly changing dimensions)
  • dbt documentation, snapshots: https://docs.getdbt.com/docs/build/snapshots (from History and slowly changing dimensions)
  • DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (Technics Publications, 2017), chapter 10, reference and master data. (from Multi-source conformance)
  • Kimball Group, dimensional modeling techniques (conformed dimensions): https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/ (from Multi-source conformance)
  • dbt documentation, about dbt models: https://docs.getdbt.com/docs/build/models (from dbt)
  • dbt documentation, incremental models: https://docs.getdbt.com/docs/build/incremental-models (from dbt)
  • dbt documentation, unit tests: https://docs.getdbt.com/docs/build/unit-tests (from dbt)
  • dbt documentation, data tests: https://docs.getdbt.com/docs/build/data-tests (from Testing data pipelines)
  • Apache Iceberg, branching and tagging: https://iceberg.apache.org/docs/latest/branching/ (from Write-audit-publish)
  • Apache Iceberg, Spark procedures (fast_forward): https://iceberg.apache.org/docs/latest/spark-procedures/ (from Write-audit-publish)
  • dbt documentation, microbatch incremental models: https://docs.getdbt.com/docs/build/incremental-microbatch (from Backfill and replay)
  • Maxime Beauchemin, Functional Data Engineering: https://maximebeauchemin.medium.com/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a (from Backfill and replay)
  • dbt documentation, how we structure our dbt projects: https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview (from Naming conventions)
  • SQLFluff documentation: https://docs.sqlfluff.com/ (from Naming conventions)
  • Dan Linstedt and Michael Olschimke, Building a Scalable Data Warehouse with Data Vault 2.0 (Morgan Kaufmann, 2015). (from Other modelling approaches: Data Vault, one big table and activity schema)
  • Activity Schema specification: https://github.com/ActivitySchema/ActivitySchema (from Other modelling approaches: Data Vault, one big table and activity schema)

05 Governance and security#

  • DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (Technics Publications, 2017), chapter 3, data governance. (from Governance roles)
  • Zhamak Dehghani, Data Mesh Principles and Logical Architecture: https://martinfowler.com/articles/data-mesh-principles.html (from Governance roles)
  • Bitol, Open Data Contract Standard: https://bitol-io.github.io/open-data-contract-standard/ (from Data contracts)
  • Data Contract CLI documentation: https://cli.datacontract.com/ (from Data contracts)
  • dbt documentation, model contracts: https://docs.getdbt.com/docs/mesh/govern/model-contracts (from Data contracts)
  • DAMA International, DAMA-DMBOK, 2nd edition (Technics Publications, 2017), chapter 13, data quality. (from Data quality dimensions)
  • Barr Moses, Lior Gavish and Molly Vorwerck, Data Quality Fundamentals (O'Reilly, 2022). (from Data quality dimensions)
  • OpenLineage, column lineage facet: https://openlineage.io/docs/spec/facets/dataset-facets/column_lineage_facet/ (from Catalog and lineage)
  • DataHub documentation: https://docs.datahub.com/ (from Catalog and lineage)
  • OpenMetadata documentation: https://docs.open-metadata.org/ (from Catalog and lineage)
  • Open Policy Agent documentation: https://www.openpolicyagent.org/docs/latest/ (from Governance as code)
  • Trino, Open Policy Agent access control: https://trino.io/docs/current/security/opa-access-control.html (from Governance as code)
  • OpenID Connect Core 1.0: https://openid.net/specs/openid-connect-core-1_0.html (from Authentication and authorization: OIDC and JWT)
  • RFC 7519, JSON Web Token: https://www.rfc-editor.org/rfc/rfc7519 (from Authentication and authorization: OIDC and JWT)
  • RFC 9700, OAuth 2.0 Security Best Current Practice: https://www.rfc-editor.org/rfc/rfc9700 (from Authentication and authorization: OIDC and JWT)
  • NIST SP 800-162, Guide to Attribute Based Access Control: https://csrc.nist.gov/pubs/sp/800/162/upd2/final (from RBAC and ABAC)
  • Zanzibar: Google's Consistent, Global Authorization System: https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/ (from RBAC and ABAC)
  • Trino, Open Policy Agent access control (row filters and column masks): https://trino.io/docs/current/security/opa-access-control.html (from Row-level security and masking)
  • Google Open Source Blog, Standardizing fine-grained access control in Apache Iceberg's REST catalog (October 2026): https://opensource.googleblog.com/2026/10/standardizing-fine-grained-access-control-in-apache-icebergs-rest-catalog.html (from Row-level security and masking)
  • AWS Well-Architected, SaaS Lens: tenant isolation: https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/tenant-isolation.html (from Multi-tenancy)
  • Trino documentation, resource groups: https://trino.io/docs/current/admin/resource-groups.html (from Multi-tenancy)
  • GDPR, Article 4 (definitions) and Recital 26: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from PII and data classification)
  • EDPB, Guidelines 01/2025 on pseudonymisation (version for public consultation): https://www.edpb.europa.eu/our-work-tools/documents/public-consultations/2025/guidelines-012025-pseudonymisation_en (from PII and data classification)
  • GDPR, Article 5(1)(e) storage limitation, Article 12(3) time limits and Article 17 right to erasure: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from Privacy lifecycle)
  • European Data Protection Board, right to erasure: https://www.edpb.europa.eu/sme-data-protection-guide/respect-individuals-rights_en (from Privacy lifecycle)
  • European Commission, international data transfers under the GDPR: https://commission.europa.eu/law/law-topic/data-protection/international-dimension-data-protection_en (from Residency and regions)
  • GDPR, Chapter V, transfers to third countries: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from Residency and regions)
  • Apache Iceberg, encryption: https://iceberg.apache.org/docs/latest/encryption/ (from Encryption and key management)
  • Apache Parquet, modular encryption: https://parquet.apache.org/docs/file-format/data-pages/encryption/ (from Encryption and key management)
  • NIST SP 800-57 Part 1, recommendation for key management: https://csrc.nist.gov/pubs/sp/800/57/pt1/r5/final (from Encryption and key management)
  • NIST SP 800-92, Guide to Computer Security Log Management: https://csrc.nist.gov/pubs/sp/800/92/final (from Audit logging)
  • Trino documentation, event listeners: https://trino.io/docs/current/develop/event-listener.html (from Audit logging)

06 Serving#

  • dbt documentation, about MetricFlow: https://docs.getdbt.com/docs/build/about-metricflow (from Semantic layer)
  • Cube documentation, semantic layer: https://cube.dev/docs/product/data-modeling/overview (from Semantic layer)
  • Apache Ossie (incubating), the new name for Open Semantic Interchange: https://ossie.apache.org/updates/ossie-enters-apache-incubator/ (from Semantic layer)
  • Sam Newman, Backends For Frontends pattern: https://samnewman.io/patterns/architectural/bff/ (from Data APIs and the backend-for-frontend pattern)
  • Microsoft Azure Architecture Center, Backends for Frontends pattern: https://learn.microsoft.com/en-us/azure/architecture/patterns/backends-for-frontends (from Data APIs and the backend-for-frontend pattern)
  • Hightouch, What is reverse ETL: https://hightouch.com/blog/reverse-etl (from Reverse ETL and activation)
  • Fivetran Activations documentation (formerly Census): https://fivetran.com/docs/activations (from Reverse ETL and activation)
  • Delta Sharing protocol: https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md (from Data sharing)
  • Databricks, OpenSharing announcement (June 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-announces-opensharing (from Data sharing)
  • ClickHouse documentation, introduction: https://clickhouse.com/docs/intro (from Low-latency serving stores)
  • Apache Pinot documentation: https://docs.pinot.apache.org/ (from Low-latency serving stores)
  • Martin Kleppmann and Chris Riccomini, Designing Data-Intensive Applications, 2nd edition (O'Reilly, 2026), the chapters on derived data and stream processing: https://dataintensive.net/ (from Low-latency serving stores)
  • Model Context Protocol specification: https://modelcontextprotocol.io/specification/ (from Agents and MCP)
  • OWASP, Top 10 for LLM applications (prompt injection): https://genai.owasp.org/llmrisk/llm01-prompt-injection/ (from Agents and MCP)
  • Stephen Few, Information Dashboard Design, 2nd edition (Analytics Press, 2013). (from BI and dashboards)
  • Apache Superset documentation: https://superset.apache.org/docs/intro (from BI and dashboards)
  • Microsoft Learn, DirectQuery in Power BI (compares live and import modes): https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-directquery-about (from BI and dashboards)

07 Operations#

  • Apache Airflow documentation, assets and data-aware scheduling: https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/asset-scheduling.html (from Orchestration)
  • Dagster documentation, assets: https://docs.dagster.io/guides/build/assets (from Orchestration)
  • dbt documentation, continuous integration: https://docs.getdbt.com/docs/deploy/continuous-integration (from Environments and CI)
  • dbt documentation, defer: https://docs.getdbt.com/reference/node-selection/defer (from Environments and CI)
  • Google, Site Reliability Engineering, chapter 4, service level objectives: https://sre.google/sre-book/service-level-objectives/ (from Observability and SLOs)
  • OpenTelemetry documentation, observability primer: https://opentelemetry.io/docs/concepts/observability-primer/ (from Observability and SLOs)
  • FinOps Foundation, FinOps framework: https://www.finops.org/framework/ (from Cost and FinOps)
  • Apache Iceberg, maintenance: https://iceberg.apache.org/docs/latest/maintenance/ (from Cost and FinOps)
  • Amazon S3, using versioning: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html (from Disaster recovery)
  • Apache Iceberg, Spark procedures (rollback and register_table): https://iceberg.apache.org/docs/latest/spark-procedures/ (from Disaster recovery)
  • OpenTofu documentation: https://opentofu.org/docs/ (from Infrastructure as code)
  • Terraform documentation: https://developer.hashicorp.com/terraform/docs (from Infrastructure as code)
  • Kief Morris, Infrastructure as Code, 3rd edition (O'Reilly, 2025). (from Infrastructure as code)
  • Martin Fowler, Strangler Fig Application: https://martinfowler.com/bliki/StranglerFigApplication.html (from Platform migration)
  • dbt documentation, migrating from stored procedures: https://docs.getdbt.com/guides/migrate-from-stored-procedures (from Platform migration)

08 Data science and ML#

  • Hadley Wickham, Mine Çetinkaya-Rundel and Garrett Grolemund, R for Data Science, 2nd edition, the whole game: https://r4ds.hadley.nz/whole-game (from The data science workflow)
  • Cassie Kozyrkov, What great data analysts do and why every organisation needs them: https://hbr.org/2018/12/what-great-data-analysts-do-and-why-every-organization-needs-them (from The data science workflow)
  • PyIceberg documentation: https://py.iceberg.apache.org/ (from Python on the lakehouse)
  • DuckDB documentation, Iceberg extension: https://duckdb.org/docs/stable/core_extensions/iceberg/overview (from Python on the lakehouse)
  • Polars user guide: https://docs.pola.rs/ (from Python on the lakehouse)
  • Ron Kohavi, Diane Tang and Ya Xu, Trustworthy Online Controlled Experiments (Cambridge University Press, 2020). (from Experimentation and A/B testing)
  • Fabijan et al., Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for Practitioners, KDD 2019: https://www.microsoft.com/en-us/research/?p=651990 (from Experimentation and A/B testing)
  • Feast documentation, concepts: https://docs.feast.dev/getting-started/concepts (from Feature engineering and feature stores)
  • Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022), chapter 5, feature engineering. (from Feature engineering and feature stores)
  • Gebru et al., Datasheets for Datasets: https://arxiv.org/abs/1803.09010 (from Training data management)
  • Apache Iceberg, branching and tagging: https://iceberg.apache.org/docs/latest/branching/ (from Training data management)
  • MLflow documentation: https://mlflow.org/docs/latest/ (from The ML lifecycle and MLOps)
  • Google Cloud, MLOps: continuous delivery and automation pipelines in machine learning: https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning (from The ML lifecycle and MLOps)
  • Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022). (from The ML lifecycle and MLOps)
  • Google, Rules of Machine Learning, monitoring and training-serving skew: https://developers.google.com/machine-learning/guides/rules-of-ml (from Model serving and monitoring)
  • Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022), chapters 7 and 8. (from Model serving and monitoring)

09 Generative AI#