Handle user IDs in Feature Experimentation

  • Updated
  • Optimizely Feature Experimentation

Optimizely Feature Experimentation identifies visitors with user IDs and determines which variation they see in an experiment (A/B test ormulti-armed bandit) or aflag delivery (targeted delivery). Feature Experimentation lets you define user IDs based on your testing strategy. For example, you can use a first-party cookie, a device ID for anonymous users, or a stored identifier for logged-in users.

Consistent user IDs prevent users from being potentially re-bucketed into different variations, ensuring reliable experiment results. If your tests span multiple devices or applications, you can use a UUID to maintain a consistent experience for each user. Alternatively, you can usebucketing IDs to provide a consistent experience across different identifiers.

Tips for user IDs

  • Ensure user IDs are unique – Feature Experimentation buckets users based on the user IDs you provide, so ensure these IDs are unique within your testing population.
  • Anonymize user IDs – Optimizely processes user IDs exactly as provided. Anonymize any personally identifiable information (PII), such as email addresses, according to your company's policies.
  • Use one namespace per project – Feature Experimentation assumes a single namespace for user IDs per project. Mixing different ID conventions within a project (for example, anonymous visitor IDs and UUIDs) can affectmutual exclusivity between tests.
  • Use logged-out or logged-in IDs – Optimizely does not alias logged-out IDs with logged-in IDs. Persist logged-out IDs for the duration of a test if you need to track users across authentication states. See theHandle multiple user IDs section.
  • iOS 14 ID for advertisers and disclosure updates – Feature Experimentation SDKs do not access user data or identifiers that you do not explicitly pass in through instrumented API calls. As a result, you should not depend on SDK changes planned to accommodate Apple's changes to itsprivacy disclosure and opt-in policy related to their ID for Advertisers (IDFA). You should review Apple's fullrequirements and guidance,other identifier options, and yourdata use carefully to determine the correct path forward for your application and usage.

Create a user context example

Feature Experimentation SDKs manage user IDs through user contexts. When you create a user context, you define a user ID and optional attributes for audience targeting. The SDKs consistently assign a user to the same variation as long as the traffic distribution is not modified. SeeWhy you should not change a running experiment.

For SDK-specific code examples, see theCreate a user context example section in the developer documentation. 

For information, see the SDK's user context documentation.

Handle multiple user IDs

If a user interacts with an experiment or targeted delivery while logged out and then logs in, Optimizely does not merge their logged-out identity with their logged-in identity. You should persist the logged-out ID and continue using it after login instead of generating a new one if you need to track a user's journey across login states (for example, seeing the same experiment variation before and after login).

Risks to changing a logged-out ID

  • Inconsistent user experience – Users may see different variations before and after logging in.
  • Corrupted experiment data – Optimizely may attribute conversion events to different IDs, inflating visitor counts and skewing results.
  • Billing inefficiencies – If you are on the Monthly Active User (MAU)-based pricing model, unnecessary user ID changes can increase your MAUs. See theusers and invoices section.

Best practices for multiple user IDs

  • Do not change user IDs in an existing user context, as it may alter variation assignments.
  • Maintain separate user context instances for anonymous and logged-in users.
  • Only use the logged-in user context where necessary, specifically in omnichannel experiments.
  • Ensure event tracking is performed separately for each user context to maintain accurate results.

Recommended approach for handling multiple user IDs

To maintain experiment consistency, create separate user context instances for anonymous and logged-in users. Never modify the user ID in an existing user context. The code sample demonstrating how to do this are available in the developer documentation.

Users and invoices

Starting in September 2020, Optimizely Feature Experimentation offersmonthly active user (MAU) pricing as an alternative to impressions pricing. Optimizely counts an MAU each time one of the following occurs for a unique user ID:

  • When you call a Decide method – A decision event (impression) is triggered.
  • When you call a Track Event method – A conversion event is triggered.

The MAU count includes unique anonymous IDs. If you useOptimizely Web Experimentation and Optimizely Feature Experimentation, you can override anonymous Optimizely Web Experimentation user IDs with known Feature Experimentation user IDs to avoid overcounting. SeeBring your own ID.

Article is closed for comments.

Articles in this section