Project 1. Evaluating a Marketing Campaign with Ordinal Outcomes

Background

A small marketing team tested two campaign formats, email and social media, with 15 customers in each group. The outcome is a 1-5 satisfaction rating, so the primary analysis should respect the ordinal scale and the small sample size rather than relying only on a normal-theory mean comparison.

Research Question

The practical question is whether the email campaign tends to produce higher customer satisfaction than the social media campaign. Because the sample is small and the ratings contain ties, the Mann–Whitney rank-sum test is a defensible primary analysis, supported by descriptive summaries and a sensitivity check using the original score scale.

Descriptive Summary

Table P1.1

Satisfaction ratings by campaign

Campaign n Median IQR Mean SD
Email 15 4.0 1.0 3.87 0.99
Social 15 3.0 1.0 3.27 0.80

Note. Ratings are ordinal responses from 1 (low satisfaction) to 5 (high satisfaction).

Figure P1.1: Distribution of satisfaction ratings by campaign.

The email group has a higher median and mean rating, but the distributions overlap. That overlap is important: in a small ordinal study, the result should be reported as evidence of a tendency rather than as a definitive campaign ranking.

Primary Analysis

Table P1.2

Mann–Whitney rank-sum analysis for campaign satisfaction

Quantity Value
Wilcoxon rank-sum W 156
Two-sided p-value 0.057
Hodges–Lehmann shift 1.00
95% CI for shift -0.00 to 1.00
Cliff's delta 0.39

Note. The test uses a normal approximation because tied ordinal scores make the exact rank-sum distribution less straightforward.

The Hodges–Lehmann shift is about one rating point in favour of email, but the confidence interval reaches the null boundary. The result is better described as suggestive than conclusive. Cliff’s delta is positive, indicating that a randomly selected email respondent tends to rate satisfaction higher than a randomly selected social-media respondent, but the small sample limits precision.

Sensitivity and Subgroup Checks

Table P1.3

Primary and sensitivity analyses for the campaign comparison

Analysis Estimate 95% CI p-value
Rank-sum primary analysis Shift = 1.00 -0.00 to 1.00 0.057
Equal-variance t-test sensitivity Mean difference = 0.60 -0.07 to 1.27 0.078

Note. The t-test is a sensitivity analysis on the numeric rating scale, not the primary analysis for the ordinal outcome.

Table P1.4

Campaign satisfaction by prior-purchase history

Campaign Prior purchase n Median Mean
Email No 7 4.0 3.86
Email Yes 8 4.0 3.88
Social No 7 4.0 3.29
Social Yes 8 3.0 3.25

Note. Cells are small, so these subgroup summaries are descriptive only.

The sensitivity analysis points in the same direction as the rank-based analysis. The subgroup table should not be used to claim moderation because several cells contain only a few customers. Its role is diagnostic: it checks whether the campaign comparison is being driven by a visibly uneven customer mix.

Reporting Summary

In this mini-study, the email campaign produced higher observed satisfaction ratings than the social-media campaign. The rank-sum test was borderline, W = 156, p = 0.057, with an estimated location shift of about 1.00 rating point. A cautious report would say that the result is consistent with a modest email advantage, but that the sample is too small to treat the campaign difference as settled.

Extension Task

Re-run the analysis after treating satisfaction as a binary outcome, for example ratings of 4 or 5 versus lower ratings. Compare the Fisher exact test result with the rank-sum result and write two sentences explaining what information is lost when the ordinal scale is collapsed.