Data Science Report
Generated by Data Scientist Agent 2.0 on 14 November 2026, 15:07 · Analysis #8
Executive summary
The objective was to find which factors most strongly influence sales. The agent identified this as a
classification problem predicting
SubscriptionType, trained
6 models, and
selected Logistic Regression, which scored
0.4098 F1 on data it had
never seen.
Objective status: NOT FULFILLED
The objective was not met. F1 of 0.4098 is too low to rely on for decisions. Success criterion: A model that clearly beats the naive baseline on F1 and behaves sensibly on the outcome the objective cares about.
Dataset overview
| Rows | 500 |
| Columns | 11 |
| Numeric columns | 3 |
| Categorical columns | 5 |
| Date columns | 3 |
| Duplicate rows | 0 (0.0%) |
| Missing cells | 398 (7.24%) |
Data quality findings
Quality score: 93/100
- 7.24% of all cells are missing (398 cells).
- 1 identifier-like column(s) detected and excluded from modelling: Name.
- High-cardinality categorical column(s) that will be grouped before encoding: Email.
Cleaning performed
| Before | After | |
|---|---|---|
| Rows | 500 | 500 |
| Columns | 11 | 10 |
| Missing cells | 398 | 378 |
| Duplicate rows | 0 | 0 |
- Unified category casing. Merged case-variant categories (for example 'Male' and 'male') in 1 column(s).
- Parsed date columns. Converted 3 text column(s) into real datetime columns so that calendar features can be derived from them.
- Dropped identifier columns. Looks like a row identifier. Identifiers are unique per row and would let a model memorise rows instead of learning a pattern.
Remaining missing values are imputed inside the modelling pipeline (numeric columns with the median, categorical columns with the most frequent value). The imputers are fitted on the training split only, so no information from the test set leaks into training.
Problem type and target variable
- Problem type: Classification — The target is non-numeric with 3 distinct classes.
- Target variable:
SubscriptionType - Confidence in the target: 100%
- Evaluation metric: F1 — Using F1 as recommended by the AI analyst for this objective.
- Split: Stratified 80/20 split with random_state=42, so both splits keep the same class balance. 338 training rows, 85 test rows.
Preprocessing
Preprocessing runs inside a scikit-learn pipeline fitted on the training split only, so no information from the test rows can influence it.
- Numeric columns: missing values filled with the training median, then scaled for the linear and distance-based models that need it.
- Categorical columns: missing values filled with the most frequent training value, then one-hot encoded with rare categories grouped together.
- Categories never seen during training are ignored at prediction time rather than causing an error.
Feature engineering
22 feature(s) were derived from the original columns.
| Feature | Built from | Type | Why |
|---|---|---|---|
DateOfBirth_year | DateOfBirth | datetime part | A model cannot read a raw timestamp, so DateOfBirth was split into its calendar parts, which let it learn seasonal and trend patterns. |
DateOfBirth_month | DateOfBirth | datetime part | Calendar 'month' taken from DateOfBirth. |
DateOfBirth_day | DateOfBirth | datetime part | Calendar 'day' taken from DateOfBirth. |
DateOfBirth_weekday | DateOfBirth | datetime part | Calendar 'weekday' taken from DateOfBirth. |
DateOfBirth_quarter | DateOfBirth | datetime part | Calendar 'quarter' taken from DateOfBirth. |
SignupDate_year | SignupDate | datetime part | A model cannot read a raw timestamp, so SignupDate was split into its calendar parts, which let it learn seasonal and trend patterns. |
SignupDate_month | SignupDate | datetime part | Calendar 'month' taken from SignupDate. |
SignupDate_day | SignupDate | datetime part | Calendar 'day' taken from SignupDate. |
SignupDate_weekday | SignupDate | datetime part | Calendar 'weekday' taken from SignupDate. |
SignupDate_quarter | SignupDate | datetime part | Calendar 'quarter' taken from SignupDate. |
LastPurchaseDate_year | LastPurchaseDate | datetime part | A model cannot read a raw timestamp, so LastPurchaseDate was split into its calendar parts, which let it learn seasonal and trend patterns. |
LastPurchaseDate_month | LastPurchaseDate | datetime part | Calendar 'month' taken from LastPurchaseDate. |
LastPurchaseDate_day | LastPurchaseDate | datetime part | Calendar 'day' taken from LastPurchaseDate. |
LastPurchaseDate_weekday | LastPurchaseDate | datetime part | Calendar 'weekday' taken from LastPurchaseDate. |
LastPurchaseDate_quarter | LastPurchaseDate | datetime part | Calendar 'quarter' taken from LastPurchaseDate. |
CustomerID_per_PurchaseAmount | CustomerID, PurchaseAmount | ratio | Ratio of two of the most informative numeric columns. Ratios capture relative size, which raw values on their own cannot. |
CustomerID_minus_PurchaseAmount | CustomerID, PurchaseAmount | difference | Gap between CustomerID and PurchaseAmount; differences often matter more than either value alone. |
CustomerID_per_LoyaltyPoints | CustomerID, LoyaltyPoints | ratio | Ratio of two of the most informative numeric columns. Ratios capture relative size, which raw values on their own cannot. |
CustomerID_minus_LoyaltyPoints | CustomerID, LoyaltyPoints | difference | Gap between CustomerID and LoyaltyPoints; differences often matter more than either value alone. |
PurchaseAmount_per_LoyaltyPoints | PurchaseAmount, LoyaltyPoints | ratio | Ratio of two of the most informative numeric columns. Ratios capture relative size, which raw values on their own cannot. |
PurchaseAmount_minus_LoyaltyPoints | PurchaseAmount, LoyaltyPoints | difference | Gap between PurchaseAmount and LoyaltyPoints; differences often matter more than either value alone. |
Email (grouped) | rare category grouping | 290 rare category value(s) in Email were merged into 'Other'. Encoding each of them would create many near-empty columns that only add noise. |
Feature selection
22 feature(s) were kept and 5 removed.
Kept: CustomerID PurchaseAmount Currency LoyaltyPoints Region Email DateOfBirth_year DateOfBirth_month DateOfBirth_day DateOfBirth_weekday SignupDate_day SignupDate_weekday SignupDate_quarter LastPurchaseDate_month LastPurchaseDate_day LastPurchaseDate_weekday CustomerID_per_PurchaseAmount CustomerID_per_LoyaltyPoints PurchaseAmount_per_LoyaltyPoints CustomerID_minus_PurchaseAmount CustomerID_minus_LoyaltyPoints PurchaseAmount_minus_LoyaltyPoints
| Removed feature | Reason |
|---|---|
SignupDate_year | Only one distinct value, so it cannot explain any variation. |
LastPurchaseDate_year | Only one distinct value, so it cannot explain any variation. |
DateOfBirth_quarter | Correlated 0.972 with DateOfBirth_month, which carries the same information and scores higher against the target. |
SignupDate_month | Correlated 0.973 with SignupDate_quarter, which carries the same information and scores higher against the target. |
LastPurchaseDate_quarter | Correlated 0.975 with LastPurchaseDate_month, which carries the same information and scores higher against the target. |
Models evaluated
| Model | Accuracy | Precision | Recall | F1 | ROC-AUC |
|---|---|---|---|---|---|
| Logistic Regression | 0.4353 | 0.4305 | 0.4353 | 0.4098 | 0.5129 |
| Decision Tree Classifier | 0.4118 | 0.4012 | 0.4118 | 0.3951 | 0.4847 |
| Random Forest Classifier | 0.3765 | 0.3556 | 0.3765 | 0.3226 | 0.4306 |
| Extra Trees Classifier | 0.3647 | 0.2853 | 0.3647 | 0.3077 | 0.4213 |
| K-Nearest Neighbors Classifier | 0.3765 | 0.3408 | 0.3765 | 0.3432 | 0.4632 |
| Baseline (Most Frequent Class) (baseline) | 0.4706 | 0.2215 | 0.4706 | 0.3012 | 0.5000 |
Best model
- Model: Logistic Regression
- Headline score: 0.4098 (F1)
- Why this model: Logistic Regression achieved the best F1 of 0.4098, ahead of Decision Tree Classifier (0.3951) by 0.0147. F1 was chosen because it matches what the objective is asking for.
- Naive baseline: 0.3012 — the score to beat before a model means anything.
- Cross-validated: 0.4097 ± 0.0482 over 5 folds.
Performance on each class
| Class | Rows in test set | Correctly identified |
|---|---|---|
Basic | 40 | 65.0% |
Premium | 13 | 15.4% |
Standard | 32 | 28.1% |
What drives the predictions
Absolute model coefficients: The size of each feature's coefficient after scaling. Larger means a one-unit change in that feature moves the prediction more.
| Rank | Feature | Share of total importance |
|---|---|---|
| 1 | CustomerID_minus_LoyaltyPoints |
12.15% |
| 2 | LoyaltyPoints |
10.29% |
| 3 | PurchaseAmount |
7.39% |
| 4 | CustomerID_minus_PurchaseAmount |
5.99% |
| 5 | Region_east |
5.66% |
| 6 | CustomerID_per_LoyaltyPoints |
5.5% |
| 7 | CustomerID |
5.36% |
| 8 | Region_north |
4.02% |
| 9 | Currency_USD |
3.96% |
| 10 | LastPurchaseDate_weekday |
3.9% |
| 11 | Region_south |
3.6% |
| 12 | LastPurchaseDate_month |
3.48% |
| 13 | PurchaseAmount_minus_LoyaltyPoints |
3.02% |
| 14 | SignupDate_day |
3.0% |
| 15 | DateOfBirth_month |
2.7% |
Did the analysis meet the objective?
Model performance and objective fulfilment are separate questions. A model can score well and still fail the objective, so each check below was applied independently.
| Check | Result | Detail |
|---|---|---|
| Predictive quality | fail | F1 of 0.4098 is too low to rely on for decisions. |
| Beats the naive baseline | pass | The trivial baseline scores 0.3012 and the chosen model scores 0.4098 - an improvement of 0.1086. |
| Catches the minority class | fail | It identifies just 15.4% of the 'Premium' cases, so it is close to useless for the decision this objective implies. |
| Stable across folds | pass | 5-fold cross-validation gives 0.4097 (± 0.0482) against a test score of 0.4098, so the result is consistent. |
Key insights
- Cleaning removed 0 row(s) and 1 column(s) that could not contribute to a reliable model.
- The winning model's decisions are driven mainly by CustomerID_minus_LoyaltyPoints (12%), LoyaltyPoints (10%), PurchaseAmount (7%).
- The model is weakest on the 'Premium' class, catching only 15% of those cases. If that class is the one the business cares about, this is the number to improve.
- The objective was not met. F1 of 0.4098 is too low to rely on for decisions. Success criterion: A model that clearly beats the naive baseline on F1 and behaves sensibly on the outcome the objective cares about.
Limitations
- Scores are measured on a single random hold-out split of this dataset. Performance on genuinely new data collected later can differ.
- No hyperparameter search was performed; every model used sensible defaults.
- The agent applies general statistical rules and has no domain knowledge about what these columns mean in your business.
- Gemini was not used for this run: google-genai package is not installed.
- 77 row(s) had no value for 'SubscriptionType' and were removed. A missing label cannot be imputed without inventing the answer.
Recommended next steps
- Collect more data. 500 rows is a small sample, and the score above could move considerably on a larger one.
- Try adjusting the decision threshold rather than the model. Moving it away from 0.5 trades precision for recall and often fixes a model that is accurate but never flags the class you care about.
- If the classes are imbalanced, resampling the training data or class weighting is usually a bigger win than switching algorithms.
- Tune Logistic Regression with a grid or randomised search. This run used sensible defaults with no hyperparameter search, so there is headroom left.
- Review the 5 removed feature(s) with a domain expert. Automated selection is statistical; it does not know which columns matter to the business.
- Before deploying, re-test on data from a later time period than the training data. A random split cannot tell you whether the pattern holds next quarter.
AI analyst commentary
'SubscriptionType' scored highest as a target candidate because last column in the file; 3 distinct classes. The objective wording suggests a classification task. Identifier and constant columns were excluded because they cannot generalise. This recommendation was produced by the built-in rule-based analyst rather than Gemini.
Preprocessing suggested by the AI layer:
- Impute missing numeric values with the median and categorical values with the mode.
- One-hot encode categorical columns, grouping rare categories first.
- Scale numeric features for the distance-based and linear models only.
This report was produced automatically. Review the findings with someone who knows the data before making decisions based on them.