The company and program
| Client | A seed-stage AI homework-help app for high-school and college students aged 13 and over, with a free tier and a $9.99-a-month subscription. |
| Stage | Seed |
| Industry | Consumer AI apps |
| Timeline | One 90-day sprint, February 2 to May 3, 2026 |
| Channels | In-app onboarding, the answer screen, push notifications, weekly practice sets |
| Team | A Kiko growth lead plus agents: a retention-cohort agent, a notification-copy agent and an experiment write-up agent. The client's two engineers and a learning designer built the changes. |
The approach
- Hypothesis: students come back when the app knows what they're studying and when their next test is, and when the answer teaches the method instead of just ending the task.
- Comparison: each change ran against a 50/50 split of new installs. From week 5, a 10% holdout kept the old experience, to separate our changes from exam season. The final read compares installs from January 19 to February 1 with installs from March 16–29, on the new experience and in the holdout.
- Decision rule: ship if day-7 retention rises at least 2 points at 95% confidence and the share of answers students flag as wrong doesn't rise. Day-30 is read on the final cohort only.
What we changed
- Weeks 1–2: retention by habit. Cohorts by install day, subject and first problem, plus 15 student interviews. 64% of installs landed on Sunday to Wednesday evenings, the night an assignment was due.
- Weeks 2–4: test A, step-by-step by default. The answer screen walked through the method one step at a time, with the full answer one tap away. Day-7 retention rose from 24.3% to 29.8%.
- Weeks 3–5: test C, the notification ask. The permission prompt moved from first open to just after the first solved problem, with the reason stated. Opt-in rose from 29% to 52%.
- Weeks 4–6: test B, classes and test dates. Two taps in onboarding to add classes and the next test date, then a reminder two days before each test and a weekly practice set. Students active on 3 or more days in week one rose from 18.4% to 26.1%.
- Weeks 5–6: the holdout and tests D and E. The 10% holdout started. A streak counter and an earlier paywall were both killed.
- Weeks 7–13: the read and handover. Winners went to everyone outside the holdout on March 16. We read day-30 retention for March 16–29 installs on April 28 and handed over the cohort dashboard.
The results
| Metric | Before (Jan 19 to Feb 1 installs) | After (Mar 16–29 installs) | Change |
|---|---|---|---|
| Day-30 retention | 12% (7,020 of 58,500) | 21% (12,810 of 61,000) | +9 pts |
| Day-7 retention | 24% | 35% | +11 pts |
| Active on 3 or more days in week one | 18% | 31% | +13 pts |
| Notification opt-in | 29% | 52% | +23 pts |
| Paid within 30 days | 2.4% | 3.3% | +0.9 pts |
| Answers flagged wrong by students | 6.1% | 5.8% | −0.3 pts |
What didn't work
- A streak counter. Day-30 retention rose 0.4 points, which was not significant. Students don't study every day; they study before tests, and the reminders already matched that rhythm.
- An earlier paywall. Showing the paywall after the third problem added 0.6 points of paid conversion but cost 3 points of day-7 retention. We kept the paywall where it was.
- The calendar helped a little. Spring tests lifted everyone: the holdout went from 12% to 13% day-30 retention. Without the holdout we would have claimed all 9 points.
What we learned
- Run a holdout whenever the calendar moves your metric.
- Ask for notifications after the first win, and say what the reminder is for.
- Make the product teach, not just answer. That is what brought students back.
Frequently asked questions
How much does a retention sprint for a consumer AI app cost?
There was no creator or media spend, so the only cost was Kiko's 90-day sprint, priced from $10,000 a month. The client's two engineers and a learning designer built the changes; Kiko ran the diagnosis, the tests and the holdout read. Paid conversion within 30 days rose from 2.4% to 3.3%.
How long does it take to see a day-30 retention lift?
Winners were live for all new installs by March 16, week 7 of the sprint. Because day-30 needs 30 days, the final read came on April 28, 12 weeks after kickoff. Day-7 was the early signal.
Would this work for an AI companion or journaling app with no exam calendar?
The notification ask after the first win and the teach-not-answer default apply to any consumer AI app. The class and test-date setup only works when users have a real external deadline. Without one, skip test B.
Related case studies and guides
Same industry, other angles:
- Creator marketing: 48,000 installs in 9 weeks at a blended $1.42 CPI
- Paid ads: 2.1x 60-day ROAS on TikTok and Meta at $70K a month
- Full growth system: reached 250,000 monthly active users in 7 months