🏛️ Westminster College, London | wcc.co.uk

📘 Data Science Intensive · Theory & Code Notes

🗓️ 6 Saturdays | 2pm–4pm 🎯 Beginner → Advanced 💻 Online · Live + Hands‑on

Comprehensive theory + applied snippets in SQL, Python (pandas, sklearn), R (dplyr, ggplot2), Tableau, and MLOps. Each concept is explained with intuition and practical context.

📌 Module 1 — SQL & Data Wrangling Week 1 · 2h

🗄️ Relational Databases & SQL Fundamentals

Why it matters: Most real-world data lives in relational databases (e.g., PostgreSQL, MySQL, Snowflake). Understanding normalization, primary/foreign keys ensures data integrity. SQL (Structured Query Language) lets you filter, aggregate, and join tables efficiently. A strong foundation in SQL is non‑negotiable for data extraction and initial exploration.

DDL vs DML: CREATE, ALTER, DROP (schema) vs SELECT, INSERT, UPDATE, DELETE (data).
Key clauses: SELECT, WHERE (row filtering), GROUP BY (aggregation), HAVING (filter groups), ORDER BY.
Aggregation functions: COUNT(*), SUM(sales), AVG(price), MIN/Max.

        -- Example: average order value per customer with at least 2 orders

SELECT c.customer_name, AVG(o.amount) AS avg_order

FROM customers c

JOIN orders o ON c.customer_id = o.customer_id

GROUP BY c.customer_name

HAVING COUNT(o.order_id) >= 2

ORDER BY avg_order DESC;

🔗 Joins: INNER, LEFT, RIGHT, FULL OUTER

Joins combine tables vertically/horizontally. INNER JOIN returns rows with matching keys in both tables. LEFT JOIN keeps all rows from the left table, filling NULL for missing matches on the right. RIGHT JOIN is symmetric, and FULL OUTER JOIN retains all records from both sides. This is essential for merging transaction data with customer profiles or product catalogs.

📈 Window Functions: ROW_NUMBER, RANK, DENSE_RANK, PARTITION BY

Window functions perform calculations across a set of rows related to the current row without collapsing them into a single output row. Partition by acts as a "group by" for the window. ROW_NUMBER gives a unique sequential number per partition. RANK leaves gaps when ties occur, while DENSE_RANK doesn’t. Use cases: ranking sales per region, computing running totals, or finding the top‑N records per category.

SELECT employee_id, department, salary,

       RANK() OVER (PARTITION BY department ORDER BY salary DESC) as rank_in_dept

FROM employees;

🧹 Data Cleaning & Transformation (pandas & dplyr)

Real data is messy: missing values, inconsistent formats, duplicates, outliers. pandas (Python) and dplyr (R) provide expressive, fast tools. Core operations: handling nulls (dropna, fillna), removing duplicates, type conversion, and creating new columns. Merging data frames (merge / join), reshaping (melt, pivot), and groupwise transformations are daily tasks.

🐍 Python (pandas)
df.isnull().sum(), df.fillna(df.median())
df.drop_duplicates()
pd.get_dummies(df['cat'])
Merge: pd.merge(df1, df2, on='id', how='left')

📦 R (dplyr + tidyr)
df %>% filter(!is.na(price))
mutate(log_price = log(price))
group_by(category) %>% summarise(avg = mean(value))
Joins: left_join(df1, df2, by='key')

💡 Pro tip: Always visualize missing data patterns (using missingno in Python or naniar in R) before imputing or dropping.

📈 Module 2 — Exploratory Data Analysis & Visualization Week 2 · 2h

📊 Understanding Distributions & Patterns

EDA is the art of summarizing data visually and statistically before modeling. It reveals outliers, missing values, relationships, and potential hypotheses. Univariate analysis examines single columns (histograms, boxplots). Bivariate analysis explores pairs (scatter plots, correlation matrices). Multivariate uses faceting, color, or dimension reduction (PCA). Key summary statistics: mean, median, standard deviation, IQR, skewness, kurtosis.

🎨 Matplotlib, Seaborn (Python) & ggplot2 (R)

Matplotlib provides low‑level control; Seaborn offers high‑level statistical plots (heatmaps, pair plots, violin plots). ggplot2 implements the Grammar of Graphics: layers, scales, themes, and facets. Both ecosystems support publication‑ready figures.

import seaborn as sns

sns.set_theme()

# correlation heatmap

sns.heatmap(df.corr(), annot=True, cmap='coolwarm')

# categorical plot

sns.boxplot(data=df, x='treatment', y='conversion')

library(ggplot2)

ggplot(df, aes(x=income, y=spending, color=segment)) +

  geom_point(alpha=0.6) +

  geom_smooth(method='lm') +

  facet_wrap(~region) +

  theme_minimal()

📊 Interactive Dashboards with Tableau

Tableau connects to databases and spreadsheets, enabling drag‑and‑drop visual analytics. Build dashboards with actions (filter, highlight, URL) and story points to guide stakeholders. Use LOD (Level of Detail) expressions for complex aggregations. Dashboards bridge data science results with business decisions.

🧪 Module 3 — Statistical Inference & Regression Week 3 · 2h

🎲 Probability, Hypothesis Testing & A/B Testing

Inferential statistics lets us draw conclusions about populations from samples. Hypothesis testing starts with a null hypothesis (no effect) and computes a p‑value. If p‑value < α (typically 0.05), we reject the null. A/B testing is a randomized experiment: split users into control vs treatment, measure a metric, and test for statistical significance. Confidence intervals provide a range of plausible effect sizes.

Common tests: t‑test (means), chi‑square (categorical), ANOVA (≥3 groups).
Power & sample size: ensures we can detect a meaningful effect.

📐 Linear & Logistic Regression Modelling

Linear regression models a continuous outcome: y = β₀ + β₁x₁ + … + ε. Coefficients represent expected change in y per unit change in x. Logistic regression models binary outcomes (yes/no) via log‑odds: log(p/(1-p)) = β₀ + β₁x. It outputs probabilities after sigmoid transformation. Model assumptions (linearity, independence, homoscedasticity, normality of residuals) must be checked. Diagnostic plots (residuals vs fitted, Q‑Q plot) and VIF for multicollinearity.

# Linear regression in Python (statsmodels)

import statsmodels.formula.api as smf

model = smf.ols('sales ~ advertising + price', data=df).fit()

print(model.summary())

# Logistic

logit_model = smf.logit('purchased ~ age + income', data=df).fit()

🌲 Module 4 — Machine Learning: Trees & Ensembles Week 4 · 2h

🌿 Decision Trees & Splitting Criteria

Decision trees partition the feature space into rectangles. For classification, splits minimize impurity (Gini, Entropy). For regression, splits minimize MSE/MAE. Trees are interpretable but prone to overfitting. Pruning (limiting depth, min samples leaf) controls complexity.

🌲 Random Forest, Gradient Boosting & XGBoost

Random Forest builds many decorrelated trees using bootstrap sampling and random feature subsets, then averages predictions → reduces variance. Gradient Boosting builds trees sequentially; each tree corrects previous errors using gradients. XGBoost is an optimized implementation with regularization, handling missing values, and parallel training. Feature importance (permutation or gain) helps interpretation.

import xgboost as xgb

model = xgb.XGBClassifier(n_estimators=100, learning_rate=0.1, max_depth=5)

model.fit(X_train, y_train)

importance = model.feature_importances_

📏 Metrics, Hyperparameter Tuning & Cross‑Validation

Classification metrics: Accuracy (overall correct), Precision (avoid false positives), Recall (capture true positives), F1 (harmonic mean), ROC‑AUC (area under curve). Regression: MAE (robust to outliers), MSE/RMSE (penalizes large errors), R² (variance explained). Cross‑validation (k‑fold) provides robust performance estimates. Hyperparameter tuning via Grid Search or Random Search finds optimal parameters.

⚠️ Always use a separate test set or nested CV to avoid overfitting during tuning.

🧠 Module 5 — Unsupervised Learning & Intro to Deep Learning Week 5 · 2h

✨ Clustering (K‑Means) & PCA

K‑Means partitions data into k clusters by minimizing within‑cluster sum of squares. Requires scaling. Choose k using elbow method (inertia) or silhouette score. PCA finds orthogonal axes of maximum variance, reducing dimensionality while preserving information. It’s used for visualization, noise reduction, and speeding up models.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=4, random_state=42)

labels = kmeans.fit_predict(X_scaled)

⚠️ Anomaly Detection Basics

Anomalies (outliers) can indicate fraud, network intrusion, or equipment failure. Statistical methods: Z‑score (>3), IQR. Isolation Forest isolates anomalies via random splits. DBSCAN clusters dense regions; points not assigned to a cluster are anomalies.

🤖 Introduction to Neural Networks & Deep Learning

Neural networks consist of layers of neurons with non‑linear activations (ReLU, sigmoid). Forward propagation computes outputs, backpropagation updates weights via gradient descent. Key concepts: loss functions (cross‑entropy for classification, MSE for regression), optimizers (Adam, SGD), batch size, epochs. Overfitting control: dropout, early stopping, regularization. Architectures: CNNs for images (convolution, pooling), RNNs/LSTMs for sequences (time series, text).

# Keras example

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense, Dropout

model = Sequential([

    Dense(128, activation='relu', input_shape=(20,)),

    Dropout(0.3),

    Dense(64, activation='relu'),

    Dense(1, activation='sigmoid')

])

model.compile(optimizer='adam', loss='binary_crossentropy')

🚀 Module 6 — MLOps & Capstone Project Week 6 · 2h + Final Presentation

⚙️ MLOps & Production Pipelines

MLOps (Machine Learning Operations) brings DevOps practices to ML: version control for data & models, automated testing, CI/CD pipelines, and monitoring. Model deployment via REST APIs (FastAPI, Flask) or batch inference. Streamlit builds interactive apps quickly. Model versioning using MLflow (track experiments, log parameters, metrics, models). Monitor data drift (input distribution changes) and concept drift (target relationship changes).

# FastAPI prediction endpoint

from fastapi import FastAPI

import joblib

app = FastAPI()

model = joblib.load("churn_model.pkl")

@app.post("/predict")

def predict(features: dict):

    pred = model.predict([list(features.values())])

    return {"prediction": int(pred[0])}

# Streamlit front‑end

import streamlit as st

st.title("Customer Churn Predictor")

tenure = st.slider("Tenure (months)", 0, 72)

monthly_charges = st.number_input("Monthly charges")

if st.button("Predict"):

    result = model.predict([[tenure, monthly_charges]])

    st.success(f"Churn risk: {'High' if result[0]==1 else 'Low'}")

🎓 End‑to‑End Capstone Project

Goal: solve a real‑world problem using the full data science lifecycle: data acquisition, cleaning, EDA, feature engineering, modeling, evaluation, deployment, and presentation. Example projects: house price prediction, customer churn, credit risk, sentiment analysis, recommendation systems, sales forecasting. Deliverables: GitHub repository, well‑commented code, a live demo (Streamlit/Cloud), and a 10‑min presentation covering business impact, methodology, and limitations.

🏆 Capstone checklist: ✔️ Problem framing ✔️ Data ingestion ✔️ EDA report ✔️ Baseline model ✔️ Hyperparameter tuning ✔️ Model interpretation ✔️ Deployment ✔️ Final slide deck