> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cfosilvia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AB TEST VARIANT TRACKING FIX

# A/B Test Variant Tracking Bug Fix

> **Historical record — status as of 2026-09-23.** This describes a fix to an
> earlier landing-page test. The two landing components it names,
> `components/landing-page-client.tsx` and `components/landing-page-variant2.tsx`,
> were deleted in cleanup PR 3 (`docs/MODULUS-CLEANUP-PLAN.md`); both had been
> unreferenced for months. `components/variant-persistence.tsx` still exists but
> nothing mounts it. The live landing experiment today runs through
> `components/home-landing-router.tsx` and `hooks/use-landing-conversion-experiment.ts`
> on a different PostHog flag — read those, not this, for how it works now.

## Problem Summary

Users were being tracked in PostHog for the wrong variant during the sign-in/sign-up flow:

1. User visits landing page → sees **control** variant
2. User clicks sign in → enters email/code
3. **During redirect to onboarding**, page re-renders
4. **New variant (test) briefly flashes** on screen (\< 1 second)
5. PostHog captures pageview with **new variant** instead of control
6. Session recording shows user saw control, but PostHog attributes them to test variant

## Root Causes

### 1. Variant Assignment Instability

* `getBootstrapData()` called on every server render
* Variant could change between renders (especially after authentication)
* No persistence mechanism to lock variant assignment

### 2. Pageview Tracking During Redirects

* PostHog pageview tracking fired immediately on page load
* Captured brief variant flashes during authentication redirects
* No delay to filter out redirect flashes

### 3. Missing Experiment Exposure Tracking

* Only pageviews tracked, not explicit experiment exposure
* No session-based deduplication of exposure events

## Solution Implemented

### 1. Variant Persistence (Server + Client)

**Server Side** (`app/utils/getBootstrapData.js`):

* Check for persisted variant cookie before calling PostHog
* If variant exists and is fresh (\< 24 hours), use it instead of PostHog assignment
* This prevents variant switching during authentication

**Client Side** (`components/variant-persistence.tsx`):

* New component that persists variant in cookie on first page load
* Sets 24-hour expiration
* Includes timestamp for freshness check

### 2. Delayed Pageview Tracking

**PostHog Provider** (`app/_providers/posthog-provider.tsx`):

* Added 500ms delay for landing page (`/`) pageviews
* If page unmounts before 500ms (e.g., during redirect), pageview not tracked
* Prevents tracking brief flashes during authentication

### 3. Explicit Experiment Exposure Tracking

**Both Variants** (`components/landing-page-client.tsx`, `components/landing-page-variant2.tsx`):

* Added `useEffect` to track `$feature_flag_called` event on mount
* Uses sessionStorage to prevent duplicate tracking
* Tracks which variant user actually saw, not just pageviews

## Files Modified

1. **`/app/utils/getBootstrapData.js`**
   * Added variant cookie reading logic
   * Override PostHog variant with persisted variant if available

2. **`/components/variant-persistence.tsx`** (NEW)
   * Client component to persist variant in cookie

3. **`/app/(home)/page.tsx`**
   * Added `<VariantPersistence>` component to both variant branches

4. **`/components/landing-page-variant2.tsx`**
   * Added experiment exposure tracking on mount
   * SessionStorage-based deduplication

5. **`/components/landing-page-client.tsx`**
   * Added experiment exposure tracking on mount
   * SessionStorage-based deduplication

6. **`/app/_providers/posthog-provider.tsx`**
   * Added 500ms delay for landing page pageview tracking

## How It Works

### First Visit (No Cookie)

1. User visits `/` → `getBootstrapData()` called
2. No persisted variant found → PostHog assigns variant (e.g., "control")
3. Page renders control variant
4. `<VariantPersistence>` component sets cookie: `lp_experiment_variant={"variant":"control","timestamp":...}`
5. Control variant tracks experiment exposure via `$feature_flag_called`
6. After 500ms (if still on page), pageview tracked

### Sign-In Flow (Cookie Exists)

1. User clicks sign in → Clerk modal opens
2. User enters email/code → Clerk redirects
3. **During redirect**, page re-renders → `getBootstrapData()` called again
4. Cookie found with "control" variant → **Uses persisted variant, not new PostHog assignment**
5. Control variant renders (same as before)
6. Page unmounts before 500ms → **No pageview tracked** (prevents flash tracking)
7. User lands on `/onboarding`

### Result

* User consistently sees control variant throughout session
* PostHog correctly tracks user in control group
* No variant switching during authentication
* No false tracking from redirect flashes

## Testing Instructions

### Test 1: Variant Persistence

1. Open incognito window
2. Visit landing page
3. Open DevTools → Application → Cookies
4. Verify `lp_experiment_variant` cookie is set
5. Note the variant value (e.g., "control")
6. Refresh page multiple times
7. **Expected**: Same variant renders each time

### Test 2: Sign-In Flow

1. Clear cookies and start fresh
2. Visit landing page → note variant shown (e.g., control)
3. Click "Sign In" button
4. Enter credentials in Clerk modal
5. Watch carefully during redirect
6. **Expected**: If brief flash occurs, it should be same variant (control)
7. Check PostHog session recording
8. **Expected**: PostHog shows user assigned to correct variant

### Test 3: Experiment Exposure Tracking

1. Clear cookies and start fresh
2. Open DevTools → Console
3. Visit landing page
4. Look for log: `📊 Experiment exposure tracked: {experimentKey: "landing-page-conversion", variant: "control"}`
5. Refresh page
6. **Expected**: Log appears only once per session (sessionStorage prevents duplicates)

### Test 4: Pageview Delay

1. Visit landing page
2. Immediately navigate away (\< 500ms)
3. Check PostHog events
4. **Expected**: No `$pageview` event for landing page
5. Visit landing page again
6. Stay for > 500ms
7. **Expected**: `$pageview` event appears in PostHog

## PostHog Events to Monitor

### Correct Tracking (After Fix)

```
1. $feature_flag_called
   - $feature_flag: "landing-page-conversion"  
   - $feature_flag_response: "control"
   - Timestamp: [When user first sees page]

2. signup_button_clicked (if user clicks CTA)
   - source: "landing_page" or "landing_page_variant2"
   - variant: "control"
   - experiment: "landing-page-conversion"

3. $pageview (only if user stays > 500ms)
   - $current_url: "https://mycfosilvia.com/"
```

### Incorrect Tracking (Before Fix - Bug)

```
1. $pageview (during brief flash)
   - $current_url: "https://mycfosilvia.com/"
   - Variant: "new-test" (wrong - user saw control!)

2. signup_button_clicked
   - variant: "control" (correct - but inconsistent with pageview)
```

## Cookie Schema

### `lp_experiment_variant`

```json theme={null}
{
  "variant": "control",  // or "new-test"
  "timestamp": 1735945200000  // Unix timestamp
}
```

* **Path**: `/`
* **Expiration**: 24 hours
* **SameSite**: `Lax`
* **Secure**: Inherited from domain settings

## Session Storage Schema

### `experiment_exposure_landing-page-conversion`

* **Value**: `"true"`
* **Purpose**: Prevent duplicate exposure tracking within same session
* **Cleared**: When browser tab/window closes

## Rollback Instructions

If this fix causes issues:

1. **Remove variant persistence**:
   * Remove lines 36-55 from `/app/utils/getBootstrapData.js`
   * Remove lines 69-72 from same file

2. **Remove pageview delay**:
   * Revert `/app/_providers/posthog-provider.tsx` to always track immediately

3. **Remove exposure tracking**:
   * Remove `useEffect` blocks from both landing page components

4. **Remove cookie component**:
   * Remove `<VariantPersistence>` from `/app/(home)/page.tsx`
   * Delete `/components/variant-persistence.tsx`

## Future Improvements

1. **Server-side variant persistence** in database for logged-in users
2. **Middleware-based variant assignment** to prevent client-side delays
3. **A/B test framework** to handle multiple experiments consistently
4. **Automated testing** for variant stability across authentication flows

## Related Issues

* PostHog A/B test inconsistency
* Variant switching during authentication
* Session recording vs analytics mismatch
* Feature flag evaluation timing

## References

* PostHog Feature Flags: [https://posthog.com/docs/feature-flags](https://posthog.com/docs/feature-flags)
* PostHog Experiments: [https://posthog.com/docs/experiments](https://posthog.com/docs/experiments)
* Next.js Server Components: [https://nextjs.org/docs/app/building-your-application/rendering/server-components](https://nextjs.org/docs/app/building-your-application/rendering/server-components)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.