# The data science workflow How a business question becomes a trustworthy answer: framing the question, exploring the data, testing explanations, and deciding whether the result is a one-off analysis, a recurring report or a model. > [!NOTE] > > **Before reading** > > [Dimensional modelling]() · [Semantic layer]() ## Why it matters Data science work starts loosely: a question from a manager, a hunch, a chart that looks odd. Without a workflow, it produces notebooks nobody can rerun, numbers that disagree with the official metrics, and conclusions that confuse correlation with cause. The platform shapes this work as much as the people do: where analysts can experiment, which data they can trust, and how results move into production. ## How it works The work goes through a loop of stages, often repeated. Framing turns a vague question into a precise one. "Are customers less loyal?" becomes "has the share of customers who order again within 90 days fallen, for which customers, and since when?". Framing decides the metric, the population, the time window and what decision the answer will inform. It is the step most often skipped and the one that most often decides whether the work is useful. Exploration looks at the data before testing anything: distributions, missing values, odd spikes, how the metric moved over time and across segments. Most surprises at this stage are data problems, such as a tracking change or a late load, not business changes. The quality status, lineage and definitions in the catalog save hours here. Hypotheses and analysis come next. Each candidate explanation, such as a price change, a new competitor or a website change, predicts something checkable in the data. Analysis compares groups, controls for confounders, such as product mix, and checks whether patterns hold across segments and time. Observational data can show association, not cause; when the cause matters and an intervention is possible, an experiment gives a stronger answer. Communication presents the answer with its uncertainty, assumptions and limits, in terms of the decision it informs. A good analysis states what would change the conclusion. The last step is deciding what the work becomes. Most analyses are one-off and end with a decision. Some become recurring: a metric worth tracking moves into the semantic layer and a dashboard. Some become models: a pattern that predicts something useful moves into the ML lifecycle. Each of these handovers means moving logic from a notebook into tested, owned pipelines. ``` flowchart LR Q[Business question] --> F[Frame:
metric, population,
window, decision] F --> E[Explore:
distributions, gaps,
data issues] E --> H[Hypotheses
and analysis] H --> C[Communicate:
answer, uncertainty,
limits] C --> D{What does it
become?} D -->|one-off| X[Decision made] D -->|recurring| M[Metric in semantic layer
and dashboard] D -->|predictive| ML[Model in the
ML lifecycle] H -.->|new questions| F ``` The platform supports this with a few things. Sandbox schemas, where data scientists can create their own tables from clean and gold data without affecting anyone else, with automatic cleanup after a period. Read access to clean and gold layers, governed by the same row and column rules as everywhere else. Governed metric definitions, so that an analysis of repeat rate starts from the same definition as the dashboard. And reproducibility: analyses that record which table snapshots and code versions they used can be rerun and checked later. ## Key terms - Framing Turning a question into a precise metric, population, window and decision. - Exploratory data analysis (EDA) Examining data to understand it before testing hypotheses. - Hypothesis A candidate explanation that predicts something checkable in the data. - Confounder A factor that influences both a cause and an effect, creating a false association. - Sandbox schema A personal or project area for experimental tables. - Reproducibility The ability to rerun an analysis on the same data and code and get the same result. ## Example In Q3 the head of sales says repeat purchases are down. - Framing: 90-day repeat rate, among customers whose first order was in each month, counting only cohorts whose 90-day window has closed (first orders up to June), compared with the same months last year. Later cohorts have not had 90 days to order again, so their rate looks low whatever happened. - Exploration: the drop is 6 points, all in customers acquired through one partner marketplace; other channels are flat. Data checks show no tracking change. - Hypotheses: that channel's customers shifted towards cheaper products, which have lower repeat rates anyway; or delivery times from that channel's warehouse got worse. The first explains a third of the drop; delivery times rose by two days from late June, matching the timing. - Communication: the repeat rate fell mainly for one channel; a third of the fall comes from cheaper products, and the rest coincides with slower delivery from late June. Confidence is moderate, since the analysis is observational. - Outcome: the repeat rate by channel becomes a metric in the semantic layer, and logistics tests faster shipping for that channel as an experiment. ## Pitfalls - Skipping framing. Precise answers to the wrong question. - Reading data problems as business changes. Check lineage and quality before explaining a spike. - Private metric definitions. An analysis that defines repeat rate differently from the dashboard starts an argument instead of a decision. - Correlation presented as cause. Say what the data can and cannot show. - Notebooks as production. A useful analysis that runs only on one laptop is lost when that person leaves. ## Learn more - [Hadley Wickham, Mine Çetinkaya-Rundel and Garrett Grolemund, R for Data Science, 2nd edition, the whole game]() - [Cassie Kozyrkov, What great data analysts do and why every organisation needs them]() ## Related [Python on the lakehouse]() [Experimentation and A/B testing]() [The ML lifecycle and MLOps]() [Catalog and lineage]()