Data Science Report

Executive summary

The objective was to classify patients into risk categories. The agent identified this as a regression problem predicting PurchaseAmount, trained 6 models, and selected Ridge Regression, which scored 0.0462 R2 on data it had never seen.

Dataset overview

Data quality findings

Quality score: 93/100

• 7.24% of all cells are missing (398 cells).
• 1 identifier-like column(s) detected and excluded from modelling: Name.
• High-cardinality categorical column(s) that will be grouped before encoding: Email.

Cleaning performed

• Unified category casing. Merged case-variant categories (for example 'Male' and 'male') in 1 column(s).
• Parsed date columns. Converted 3 text column(s) into real datetime columns so that calendar features can be derived from them.
• Dropped identifier columns. Looks like a row identifier. Identifiers are unique per row and would let a model memorise rows instead of learning a pattern.

Remaining missing values are imputed inside the modelling pipeline (numeric columns with the median, categorical columns with the most frequent value). The imputers are fitted on the training split only, so no information from the test set leaks into training.

Problem type and target variable

• Problem type: Regression — The target is continuous with 449 distinct numeric values.
• Target variable: PurchaseAmount
• Confidence in the target: 100%
• Evaluation metric: R2 — R² reports the share of variation in the target the model explains, which is comparable across datasets in a way that RMSE is not.
• Split: Random 80/20 split with random_state=42. 360 training rows, 90 test rows.

Preprocessing

Preprocessing runs inside a scikit-learn pipeline fitted on the training split only, so no information from the test rows can influence it.

• Numeric columns: missing values filled with the training median, then scaled for the linear and distance-based models that need it.
• Categorical columns: missing values filled with the most frequent training value, then one-hot encoded with rare categories grouped together.
• Categories never seen during training are ignored at prediction time rather than causing an error.

Feature engineering

18 feature(s) were derived from the original columns.

Feature selection

19 feature(s) were kept and 4 removed.

Kept: CustomerID Currency LoyaltyPoints Region Email SubscriptionType DateOfBirth_year DateOfBirth_day DateOfBirth_weekday DateOfBirth_quarter SignupDate_day SignupDate_weekday SignupDate_quarter LastPurchaseDate_year LastPurchaseDate_day LastPurchaseDate_weekday LastPurchaseDate_quarter LoyaltyPoints_per_CustomerID LoyaltyPoints_minus_CustomerID

Models evaluated

Best model

• Model: Ridge Regression
• Headline score: 0.0462 (R2)
• Why this model: Ridge Regression achieved the best R2 of 0.0462, ahead of Linear Regression (0.0432) by 0.0030. R2 was chosen because it matches what the objective is asking for.
• Naive baseline: -0.0010 — the score to beat before a model means anything.
• Cross-validated: -4.7451 ± 9.3694 over 5 folds.

What drives the predictions

Absolute model coefficients: The size of each feature's coefficient after scaling. Larger means a one-unit change in that feature moves the prediction more.

Did the analysis meet the objective?

Model performance and objective fulfilment are separate questions. A model can score well and still fail the objective, so each check below was applied independently.

Key insights

• Cleaning removed 0 row(s) and 1 column(s) that could not contribute to a reliable model.
• The winning model's decisions are driven mainly by Currency_GBP (13%), LoyaltyPoints_minus_CustomerID (12%), SubscriptionType_Standard (10%).
• On average the prediction is off by 299.59 in the target's own units, which is the number to quote when someone asks how accurate it is.
• The objective was partially met. The model explains only 4.6% of the variation, which is too little to be useful. Success criterion: A model that clearly beats the naive baseline on F1 and behaves sensibly on the outcome the objective cares about.

Limitations

• Scores are measured on a single random hold-out split of this dataset. Performance on genuinely new data collected later can differ.
• No hyperparameter search was performed; every model used sensible defaults.
• The agent applies general statistical rules and has no domain knowledge about what these columns mean in your business.
• Gemini was not used for this run: google-genai package is not installed.
• The AI layer suggested classification, but 'PurchaseAmount' is the target is continuous with 449 distinct numeric values. The agent trusted the data and treated this as regression.
• 50 row(s) had no value for 'PurchaseAmount' and were removed. A missing label cannot be imputed without inventing the answer.

Recommended next steps

1. Collect more data. 500 rows is a small sample, and the score above could move considerably on a larger one.
2. Look for missing explanatory variables. When R² is low, the cause is usually that the drivers of the target were never recorded, not that the model is wrong.
3. Tune Ridge Regression with a grid or randomised search. This run used sensible defaults with no hyperparameter search, so there is headroom left.
4. Review the 4 removed feature(s) with a domain expert. Automated selection is statistical; it does not know which columns matter to the business.
5. Before deploying, re-test on data from a later time period than the training data. A random split cannot tell you whether the pattern holds next quarter.

AI analyst commentary

'SubscriptionType' scored highest as a target candidate because last column in the file; 3 distinct classes. The objective wording suggests a classification task. Identifier and constant columns were excluded because they cannot generalise. This recommendation was produced by the built-in rule-based analyst rather than Gemini.

Preprocessing suggested by the AI layer:

• Impute missing numeric values with the median and categorical values with the mode.
• One-hot encode categorical columns, grouping rare categories first.
• Scale numeric features for the distance-based and linear models only.

This report was produced automatically. Review the findings with someone who knows the data before making decisions based on them.