Question about Guideline 2.5.19 – A/B testing and remotely configured paywalls

App Store Distribution & Marketing App Review
mrmojo8 OP
Created
Replies 3
Boosts 0
Views 96
Participants 3

Hello,

I have a few questions about how App Review handles A/B testing and remote configuration, so I can make sure an app is compliant before submission.

  1. A/B testing: Is it allowed under Guideline 2.5.19 to A/B test paywall designs, subscription or consumable offers, and some content features with a server-side remote configuration service (e.g. Firebase Remote Config), so that different users see different variants?
  2. Review access: If this is allowed, what is the expected way to make all variants available to App Review? Is it enough to describe every variant in the Notes for Review, or should the reviewer also be able to switch between variants in the app?
  3. Changes after approval: If a new paywall variant or feature is enabled through remote configuration after a version has been approved, does it need a new submission? Or is it acceptable if it was described in the review notes of the approved version?
  4. Review-specific behavior: Is there any case where an app may behave differently for App Review than for regular users, or should the reviewed experience always be identical to what users get?
  5. DPLA Section 3.2(f): Could you give examples of what kind of app behavior is considered "interfering with" App Review or "evading the review process" under Section 3.2(f)?

Thank you for your help.

Best regards

Replies  3
Boosts 0
Views  96
Participants  3
Etresoft OP

A/B testing: Is it allowed under Guideline 2.5.19 to A/B test paywall designs, subscription or consumable offers, and some content features with a server-side remote configuration service (e.g. Firebase Remote Config), so that different users see different variants?

You might want to consider a "Meet with Apple" session to ask the App Review team directly. There's very little App Review participation in this forum.
Etresoft OP

Never mind. Screwed up.
Pokpong OP

I read 2.5.19 as being about hidden or undocumented functionality, rather than a blanket ban on A/B tests. If the variants are just different presentations of features already in the submitted app, that seems different from remotely switching on a feature that was not visible to App Review. The guidelines also say apps may not download code that adds or changes functionality.

I’d make the review path predictable: explain the variants in Review Notes and give the reviewer a way to see each one, such as a clearly documented setting or test account. Keep the reviewed experience representative of what users can get, and don’t use reviewer-specific behavior to conceal a variant. For digital subscriptions or consumables, keep the purchase flow within the applicable In-App Purchase rules.

I wouldn’t treat Review Notes as approval for a substantially new feature after release. If a remote change adds functionality, submit an update for review first. Apple’s guidelines don’t give a specific safe harbor for every paywall experiment, so for a business-critical setup I’d ask App Review directly, as the other reply suggests: https://developer.apple.com/app-store/review/guidelines/