Superstore Dataset: Predicting Profit with Regression and Decision Tree Models

Introduction

The "Superstore Dataset" was created specifically for educational purposes, providing valuable insights into retail data. The dataset is widely used for data visualization and analysis tasks. In this project, we aim to explore patterns in sales and profit by building various machine learning models. Specifically, we predict profit using regression techniques and a decision tree classifier for categorical profit analysis. The dataset contains features like product category, customer segments, sales, and discounts, which are key drivers for predicting profitability.

The dataset can be found on Kaggle the full code is here You can find the report here .

Dataset Understanding

The dataset consists of 21 features and 9994 instances. It includes both categorical and numerical variables, such as:

• Categorical: Ship Mode, Customer ID, Segment, Country, City, State, Region, Category, Sub-Category
• Numerical: Sales, Quantity, Discount, Profit

We cleaned the dataset by handling missing values, removing outliers, and transforming some features for better model performance. We also performed data mining techniques, including association rule mining and clustering, to find hidden patterns in customer purchase behavior.

1. Regression Models

We applied multiple regression models (Linear, Lasso, Ridge, and ElasticNet) to predict the profit of sales transactions. The dataset was split into 80% training and 20% testing. For each model, we predicted the profit on the test set and evaluated the model's performance using various metrics such as R-squared, Mean Squared Error (MSE), and Root Mean Squared Error (RMSE).

The above scatter plot shows the actual vs predicted profit values for the regression models. Ideally, points should lie along the red dashed line, which represents perfect predictions. ElasticNet and Ridge models performed best in reducing error and avoiding overfitting.

Code Example:

            # Linear regression
            lm_model <- lm(Profit ~ ., data = X_train)
            lm_predictions <- predict(lm_model, newdata = X_test)
            lm_metrics <- calculate_metrics(y_test, lm_predictions)
            
            # Lasso regression
            lasso_model <- train(
                x = as.matrix(X_train),
                y = y_train,
                method = "glmnet",
                trControl = trainControl("cv"),
                tuneGrid = expand.grid(alpha = 1, lambda = seq(0.001, 1, by = 0.01))
            )
            lasso_predictions <- predict(lasso_model, newdata = as.matrix(X_test))
            lasso_metrics <- calculate_metrics(y_test, lasso_predictions)
        

2. Decision Tree Model

To further analyze profit prediction, we converted the profit into a categorical variable with three classes. We then built a decision tree classifier to predict which category each transaction falls into based on various features such as customer segment, product category, and region.

The decision tree model splits the data at important feature points to classify profit into different categories. As shown in the diagram, features like customer segment and product sub-category were the most significant in classifying profit into low, medium, and high categories.

Code Example:

            # Build decision tree
            profit_tree <- rpart(Profit_Category ~ ., data = df, method = "class")
            
            # Plot the tree
            rpart.plot(profit_tree, main = "Decision Tree for Profit Categories")
            
            # Evaluate model
            predictions <- predict(profit_tree, test_set, type = "class")
            cm <- confusionMatrix(predictions, test_set$Profit_Category)
            print(cm)
        

3. Clustering & Apriori Analysis

To gain deeper insights into customer behavior, we used K-means clustering to group customers based on their sales and profit. Additionally, we applied the Apriori algorithm to identify patterns in customer segments, regions, and product categories.

Code Example:

            # K-means clustering on sales and profit
            set.seed(123)
            kmeans_result <- kmeans(sales_profit_data, centers = 3)
            
            # Add cluster labels
            filtered_data$Cluster <- kmeans_result$cluster
            
            # Apriori algorithm for pattern discovery
            rules <- apriori(transactions, parameter = list(support = 0.2, confidence = 0.8))
            inspect(rules)
        

Evaluation Metrics

For regression models, we evaluated performance using:

• R-squared
• Mean Squared Error (MSE)
• Mean Absolute Error (MAE)
• Root Mean Squared Error (RMSE)

For the decision tree classifier, we computed:

• Accuracy
• Precision
• Recall