E2E Test Runner

1# Install agent-browser globally
2npm install -g agent-browser
3
4# Install Chromium (required)
5agent-browser install

1# Open a page and get a snapshot with interactive elements
2agent-browser open https://example.com
3agent-browser snapshot -i  # Returns elements with refs like [ref=e1]
4
5# Interact using element references from snapshot
6agent-browser click @e1                      # Click element by ref
7agent-browser fill @e2 "user@example.com"   # Fill input by ref
8agent-browser fill @e3 "password123"        # Fill password field
9agent-browser click @e4                      # Click submit button
10
11# Wait for conditions
12agent-browser wait visible @e5               # Wait for element
13agent-browser wait navigation                # Wait for page load
14
15# Take screenshots
16agent-browser screenshot after-login.png
17
18# Get text content
19agent-browser get text @e1

1import { execSync } from 'child_process'
2
3// Execute agent-browser commands
4const snapshot = execSync('agent-browser snapshot -i --json').toString()
5const elements = JSON.parse(snapshot)
6
7// Find element ref and interact
8execSync('agent-browser click @e1')
9execSync('agent-browser fill @e2 "test@example.com"')

1import { BrowserManager } from 'agent-browser'
2
3const browser = new BrowserManager()
4await browser.launch({ headless: true })
5await browser.navigate('https://example.com')
6
7// Low-level event injection
8await browser.injectMouseEvent({ type: 'mousePressed', x: 100, y: 200, button: 'left' })
9await browser.injectKeyboardEvent({ type: 'keyDown', key: 'Enter', code: 'Enter' })
10
11// Screencast for AI vision
12await browser.startScreencast()  // Stream viewport frames

1# Run all E2E tests
2npx playwright test
3
4# Run specific test file
5npx playwright test tests/markets.spec.ts
6
7# Run tests in headed mode (see browser)
8npx playwright test --headed
9
10# Debug test with inspector
11npx playwright test --debug
12
13# Generate test code from actions
14npx playwright codegen http://localhost:3000
15
16# Run tests with trace
17npx playwright test --trace on
18
19# Show HTML report
20npx playwright show-report
21
22# Update snapshots
23npx playwright test --update-snapshots
24
25# Run tests in specific browser
26npx playwright test --project=chromium
27npx playwright test --project=firefox
28npx playwright test --project=webkit

1a) Identify critical user journeys
2   - Authentication flows (login, logout, registration)
3   - Core features (market creation, trading, searching)
4   - Payment flows (deposits, withdrawals)
5   - Data integrity (CRUD operations)
6
7b) Define test scenarios
8   - Happy path (everything works)
9   - Edge cases (empty states, limits)
10   - Error cases (network failures, validation)
11
12c) Prioritize by risk
13   - HIGH: Financial transactions, authentication
14   - MEDIUM: Search, filtering, navigation
15   - LOW: UI polish, animations, styling

1For each user journey:
2
31. Write test in Playwright
4   - Use Page Object Model (POM) pattern
5   - Add meaningful test descriptions
6   - Include assertions at key steps
7   - Add screenshots at critical points
8
92. Make tests resilient
10   - Use proper locators (data-testid preferred)
11   - Add waits for dynamic content
12   - Handle race conditions
13   - Implement retry logic
14
153. Add artifact capture
16   - Screenshot on failure
17   - Video recording
18   - Trace for debugging
19   - Network logs if needed

1a) Run tests locally
2   - Verify all tests pass
3   - Check for flakiness (run 3-5 times)
4   - Review generated artifacts
5
6b) Quarantine flaky tests
7   - Mark unstable tests as @flaky
8   - Create issue to fix
9   - Remove from CI temporarily
10
11c) Run in CI/CD
12   - Execute on pull requests
13   - Upload artifacts to CI
14   - Report results in PR comments

1tests/
2├── e2e/                       # End-to-end user journeys
3│   ├── auth/                  # Authentication flows
4│   │   ├── login.spec.ts
5│   │   ├── logout.spec.ts
6│   │   └── register.spec.ts
7│   ├── markets/               # Market features
8│   │   ├── browse.spec.ts
9│   │   ├── search.spec.ts
10│   │   ├── create.spec.ts
11│   │   └── trade.spec.ts
12│   ├── wallet/                # Wallet operations
13│   │   ├── connect.spec.ts
14│   │   └── transactions.spec.ts
15│   └── api/                   # API endpoint tests
16│       ├── markets-api.spec.ts
17│       └── search-api.spec.ts
18├── fixtures/                  # Test data and helpers
19│   ├── auth.ts                # Auth fixtures
20│   ├── markets.ts             # Market test data
21│   └── wallets.ts             # Wallet fixtures
22└── playwright.config.ts       # Playwright configuration

1// pages/MarketsPage.ts
2import { Page, Locator } from '@playwright/test'
3
4export class MarketsPage {
5  readonly page: Page
6  readonly searchInput: Locator
7  readonly marketCards: Locator
8  readonly createMarketButton: Locator
9  readonly filterDropdown: Locator
10
11  constructor(page: Page) {
12    this.page = page
13    this.searchInput = page.locator('[data-testid="search-input"]')
14    this.marketCards = page.locator('[data-testid="market-card"]')
15    this.createMarketButton = page.locator('[data-testid="create-market-btn"]')
16    this.filterDropdown = page.locator('[data-testid="filter-dropdown"]')
17  }
18
19  async goto() {
20    await this.page.goto('/markets')
21    await this.page.waitForLoadState('networkidle')
22  }
23
24  async searchMarkets(query: string) {
25    await this.searchInput.fill(query)
26    await this.page.waitForResponse(resp => resp.url().includes('/api/markets/search'))
27    await this.page.waitForLoadState('networkidle')
28  }
29
30  async getMarketCount() {
31    return await this.marketCards.count()
32  }
33
34  async clickMarket(index: number) {
35    await this.marketCards.nth(index).click()
36  }
37
38  async filterByStatus(status: string) {
39    await this.filterDropdown.selectOption(status)
40    await this.page.waitForLoadState('networkidle')
41  }
42}

1// tests/e2e/markets/search.spec.ts
2import { test, expect } from '@playwright/test'
3import { MarketsPage } from '../../pages/MarketsPage'
4
5test.describe('Market Search', () => {
6  let marketsPage: MarketsPage
7
8  test.beforeEach(async ({ page }) => {
9    marketsPage = new MarketsPage(page)
10    await marketsPage.goto()
11  })
12
13  test('should search markets by keyword', async ({ page }) => {
14    // Arrange
15    await expect(page).toHaveTitle(/Markets/)
16
17    // Act
18    await marketsPage.searchMarkets('trump')
19
20    // Assert
21    const marketCount = await marketsPage.getMarketCount()
22    expect(marketCount).toBeGreaterThan(0)
23
24    // Verify first result contains search term
25    const firstMarket = marketsPage.marketCards.first()
26    await expect(firstMarket).toContainText(/trump/i)
27
28    // Take screenshot for verification
29    await page.screenshot({ path: 'artifacts/search-results.png' })
30  })
31
32  test('should handle no results gracefully', async ({ page }) => {
33    // Act
34    await marketsPage.searchMarkets('xyznonexistentmarket123')
35
36    // Assert
37    await expect(page.locator('[data-testid="no-results"]')).toBeVisible()
38    const marketCount = await marketsPage.getMarketCount()
39    expect(marketCount).toBe(0)
40  })
41
42  test('should clear search results', async ({ page }) => {
43    // Arrange - perform search first
44    await marketsPage.searchMarkets('trump')
45    await expect(marketsPage.marketCards.first()).toBeVisible()
46
47    // Act - clear search
48    await marketsPage.searchInput.clear()
49    await page.waitForLoadState('networkidle')
50
51    // Assert - all markets shown again
52    const marketCount = await marketsPage.getMarketCount()
53    expect(marketCount).toBeGreaterThan(10) // Should show all markets
54  })
55})

1test('user can browse and view markets', async ({ page }) => {
2  // 1. Navigate to markets page
3  await page.goto('/markets')
4  await expect(page.locator('h1')).toContainText('Markets')
5
6  // 2. Verify markets are loaded
7  const marketCards = page.locator('[data-testid="market-card"]')
8  await expect(marketCards.first()).toBeVisible()
9
10  // 3. Click on a market
11  await marketCards.first().click()
12
13  // 4. Verify market details page
14  await expect(page).toHaveURL(/\/markets\/[a-z0-9-]+/)
15  await expect(page.locator('[data-testid="market-name"]')).toBeVisible()
16
17  // 5. Verify chart loads
18  await expect(page.locator('[data-testid="price-chart"]')).toBeVisible()
19})

1test('semantic search returns relevant results', async ({ page }) => {
2  // 1. Navigate to markets
3  await page.goto('/markets')
4
5  // 2. Enter search query
6  const searchInput = page.locator('[data-testid="search-input"]')
7  await searchInput.fill('election')
8
9  // 3. Wait for API call
10  await page.waitForResponse(resp =>
11    resp.url().includes('/api/markets/search') && resp.status() === 200
12  )
13
14  // 4. Verify results contain relevant markets
15  const results = page.locator('[data-testid="market-card"]')
16  await expect(results).not.toHaveCount(0)
17
18  // 5. Verify semantic relevance (not just substring match)
19  const firstResult = results.first()
20  const text = await firstResult.textContent()
21  expect(text?.toLowerCase()).toMatch(/election|trump|biden|president|vote/)
22})

1test('user can connect wallet', async ({ page, context }) => {
2  // Setup: Mock Privy wallet extension
3  await context.addInitScript(() => {
4    // @ts-ignore
5    window.ethereum = {
6      isMetaMask: true,
7      request: async ({ method }) => {
8        if (method === 'eth_requestAccounts') {
9          return ['0x1234567890123456789012345678901234567890']
10        }
11        if (method === 'eth_chainId') {
12          return '0x1'
13        }
14      }
15    }
16  })
17
18  // 1. Navigate to site
19  await page.goto('/')
20
21  // 2. Click connect wallet
22  await page.locator('[data-testid="connect-wallet"]').click()
23
24  // 3. Verify wallet modal appears
25  await expect(page.locator('[data-testid="wallet-modal"]')).toBeVisible()
26
27  // 4. Select wallet provider
28  await page.locator('[data-testid="wallet-provider-metamask"]').click()
29
30  // 5. Verify connection successful
31  await expect(page.locator('[data-testid="wallet-address"]')).toBeVisible()
32  await expect(page.locator('[data-testid="wallet-address"]')).toContainText('0x1234')
33})

1test('authenticated user can create market', async ({ page }) => {
2  // Prerequisites: User must be authenticated
3  await page.goto('/creator-dashboard')
4
5  // Verify auth (or skip test if not authenticated)
6  const isAuthenticated = await page.locator('[data-testid="user-menu"]').isVisible()
7  test.skip(!isAuthenticated, 'User not authenticated')
8
9  // 1. Click create market button
10  await page.locator('[data-testid="create-market"]').click()
11
12  // 2. Fill market form
13  await page.locator('[data-testid="market-name"]').fill('Test Market')
14  await page.locator('[data-testid="market-description"]').fill('This is a test market')
15  await page.locator('[data-testid="market-end-date"]').fill('2025-12-31')
16
17  // 3. Submit form
18  await page.locator('[data-testid="submit-market"]').click()
19
20  // 4. Verify success
21  await expect(page.locator('[data-testid="success-message"]')).toBeVisible()
22
23  // 5. Verify redirect to new market
24  await expect(page).toHaveURL(/\/markets\/test-market/)
25})

1test('user can place trade with sufficient balance', async ({ page }) => {
2  // WARNING: This test involves real money - use testnet/staging only!
3  test.skip(process.env.NODE_ENV === 'production', 'Skip on production')
4
5  // 1. Navigate to market
6  await page.goto('/markets/test-market')
7
8  // 2. Connect wallet (with test funds)
9  await page.locator('[data-testid="connect-wallet"]').click()
10  // ... wallet connection flow
11
12  // 3. Select position (Yes/No)
13  await page.locator('[data-testid="position-yes"]').click()
14
15  // 4. Enter trade amount
16  await page.locator('[data-testid="trade-amount"]').fill('1.0')
17
18  // 5. Verify trade preview
19  const preview = page.locator('[data-testid="trade-preview"]')
20  await expect(preview).toContainText('1.0 SOL')
21  await expect(preview).toContainText('Est. shares:')
22
23  // 6. Confirm trade
24  await page.locator('[data-testid="confirm-trade"]').click()
25
26  // 7. Wait for blockchain transaction
27  await page.waitForResponse(resp =>
28    resp.url().includes('/api/trade') && resp.status() === 200,
29    { timeout: 30000 } // Blockchain can be slow
30  )
31
32  // 8. Verify success
33  await expect(page.locator('[data-testid="trade-success"]')).toBeVisible()
34
35  // 9. Verify balance updated
36  const balance = page.locator('[data-testid="wallet-balance"]')
37  await expect(balance).not.toContainText('--')
38})

1// playwright.config.ts
2import { defineConfig, devices } from '@playwright/test'
3
4export default defineConfig({
5  testDir: './tests/e2e',
6  fullyParallel: true,
7  forbidOnly: !!process.env.CI,
8  retries: process.env.CI ? 2 : 0,
9  workers: process.env.CI ? 1 : undefined,
10  reporter: [
11    ['html', { outputFolder: 'playwright-report' }],
12    ['junit', { outputFile: 'playwright-results.xml' }],
13    ['json', { outputFile: 'playwright-results.json' }]
14  ],
15  use: {
16    baseURL: process.env.BASE_URL || 'http://localhost:3000',
17    trace: 'on-first-retry',
18    screenshot: 'only-on-failure',
19    video: 'retain-on-failure',
20    actionTimeout: 10000,
21    navigationTimeout: 30000,
22  },
23  projects: [
24    {
25      name: 'chromium',
26      use: { ...devices['Desktop Chrome'] },
27    },
28    {
29      name: 'firefox',
30      use: { ...devices['Desktop Firefox'] },
31    },
32    {
33      name: 'webkit',
34      use: { ...devices['Desktop Safari'] },
35    },
36    {
37      name: 'mobile-chrome',
38      use: { ...devices['Pixel 5'] },
39    },
40  ],
41  webServer: {
42    command: 'npm run dev',
43    url: 'http://localhost:3000',
44    reuseExistingServer: !process.env.CI,
45    timeout: 120000,
46  },
47})

1# Run test multiple times to check stability
2npx playwright test tests/markets/search.spec.ts --repeat-each=10
3
4# Run specific test with retries
5npx playwright test tests/markets/search.spec.ts --retries=3

1// Mark flaky test for quarantine
2test('flaky: market search with complex query', async ({ page }) => {
3  test.fixme(true, 'Test is flaky - Issue #123')
4
5  // Test code here...
6})
7
8// Or use conditional skip
9test('market search with complex query', async ({ page }) => {
10  test.skip(process.env.CI, 'Test is flaky in CI - Issue #123')
11
12  // Test code here...
13})

1// ❌ FLAKY: Don't assume element is ready
2await page.click('[data-testid="button"]')
3
4// ✅ STABLE: Wait for element to be ready
5await page.locator('[data-testid="button"]').click() // Built-in auto-wait

1// ❌ FLAKY: Arbitrary timeout
2await page.waitForTimeout(5000)
3
4// ✅ STABLE: Wait for specific condition
5await page.waitForResponse(resp => resp.url().includes('/api/markets'))

1// ❌ FLAKY: Click during animation
2await page.click('[data-testid="menu-item"]')
3
4// ✅ STABLE: Wait for animation to complete
5await page.locator('[data-testid="menu-item"]').waitFor({ state: 'visible' })
6await page.waitForLoadState('networkidle')
7await page.click('[data-testid="menu-item"]')

1// Take screenshot at key points
2await page.screenshot({ path: 'artifacts/after-login.png' })
3
4// Full page screenshot
5await page.screenshot({ path: 'artifacts/full-page.png', fullPage: true })
6
7// Element screenshot
8await page.locator('[data-testid="chart"]').screenshot({
9  path: 'artifacts/chart.png'
10})

1// Start trace
2await browser.startTracing(page, {
3  path: 'artifacts/trace.json',
4  screenshots: true,
5  snapshots: true,
6})
7
8// ... test actions ...
9
10// Stop trace
11await browser.stopTracing()

1// Configured in playwright.config.ts
2use: {
3  video: 'retain-on-failure', // Only save video if test fails
4  videosPath: 'artifacts/videos/'
5}

1# .github/workflows/e2e.yml
2name: E2E Tests
3
4on: [push, pull_request]
5
6jobs:
7  test:
8    runs-on: ubuntu-latest
9    steps:
10      - uses: actions/checkout@v3
11
12      - uses: actions/setup-node@v3
13        with:
14          node-version: 18
15
16      - name: Install dependencies
17        run: npm ci
18
19      - name: Install Playwright browsers
20        run: npx playwright install --with-deps
21
22      - name: Run E2E tests
23        run: npx playwright test
24        env:
25          BASE_URL: https://staging.pmx.trade
26
27      - name: Upload artifacts
28        if: always()
29        uses: actions/upload-artifact@v3
30        with:
31          name: playwright-report
32          path: playwright-report/
33          retention-days: 30
34
35      - name: Upload test results
36        if: always()
37        uses: actions/upload-artifact@v3
38        with:
39          name: playwright-results
40          path: playwright-results.xml

1# E2E Test Report
2
3**Date:** YYYY-MM-DD HH:MM
4**Duration:** Xm Ys
5**Status:** ✅ PASSING / ❌ FAILING
6
7## Summary
8
9- **Total Tests:** X
10- **Passed:** Y (Z%)
11- **Failed:** A
12- **Flaky:** B
13- **Skipped:** C
14
15## Test Results by Suite
16
17### Markets - Browse & Search
18- ✅ user can browse markets (2.3s)
19- ✅ semantic search returns relevant results (1.8s)
20- ✅ search handles no results (1.2s)
21- ❌ search with special characters (0.9s)
22
23### Wallet - Connection
24- ✅ user can connect MetaMask (3.1s)
25- ⚠️  user can connect Phantom (2.8s) - FLAKY
26- ✅ user can disconnect wallet (1.5s)
27
28### Trading - Core Flows
29- ✅ user can place buy order (5.2s)
30- ❌ user can place sell order (4.8s)
31- ✅ insufficient balance shows error (1.9s)
32
33## Failed Tests
34
35### 1. search with special characters
36**File:** `tests/e2e/markets/search.spec.ts:45`
37**Error:** Expected element to be visible, but was not found
38**Screenshot:** artifacts/search-special-chars-failed.png
39**Trace:** artifacts/trace-123.zip
40
41**Steps to Reproduce:**
421. Navigate to /markets
432. Enter search query with special chars: "trump & biden"
443. Verify results
45
46**Recommended Fix:** Escape special characters in search query
47
48---
49
50### 2. user can place sell order
51**File:** `tests/e2e/trading/sell.spec.ts:28`
52**Error:** Timeout waiting for API response /api/trade
53**Video:** artifacts/videos/sell-order-failed.webm
54
55**Possible Causes:**
56- Blockchain network slow
57- Insufficient gas
58- Transaction reverted
59
60**Recommended Fix:** Increase timeout or check blockchain logs
61
62## Artifacts
63
64- HTML Report: playwright-report/index.html
65- Screenshots: artifacts/*.png (12 files)
66- Videos: artifacts/videos/*.webm (2 files)
67- Traces: artifacts/*.zip (2 files)
68- JUnit XML: playwright-results.xml
69
70## Next Steps
71
72- [ ] Fix 2 failing tests
73- [ ] Investigate 1 flaky test
74- [ ] Review and merge if all green