# Tracer v3.6.1 — public footprint intelligence

Tracer is a Cloudflare Pages application for limited, consent-based research using public platform APIs, DNS records and optional third-party lookups. It does not access private accounts, retrieve passwords, identify phone subscribers, or establish that two profiles belong to the same person.

## Current v3 implementation status

### Included
- Passwordless email-link registration and sign-in using one-time, 20-minute links; successful link click verifies email and creates a 30-day HttpOnly session.
- Guest allowance: 2 searches per day per hashed IP; verified accounts: 25 searches per day. Quota is enforced server-side when D1 is configured.
- Disposable-domain screening against a small starter denylist stored in D1; maintain the list as needed.
- Existing v2 UI and public-source integrations retained.
- Username lookups: GitHub, Reddit, GitLab, Hacker News, Bluesky and Mastodon (mastodon.social instance only).
- Additional username URL patterns are shown as **Unverified URL leads**, not account-existence checks.
- Email domain DNS: MX, SPF and DMARC.
- Optional Have I Been Pwned email breach lookup (requires an authorized API key and plan).
- Optional Brave public-web search for usernames and explicitly opted-in exact email queries.
- Phone E.164 syntax check and optional Twilio Lookup v2 (paid line-type feature may incur charges).
- `/api/config` publishes only the Turnstile site key and non-secret feature state.
- Server-side Turnstile Siteverify when `TURNSTILE_SECRET_KEY` is configured. The browser widget is rendered when `TURNSTILE_SITE_KEY` is present.
- D1-backed per-IP hashed request throttling when the `DB` binding is configured (15 username requests/minute; 8 email or phone requests/minute).
- Request body size and content-type checks; `no-store` API responses; baseline browser security headers.
- D1 schema for rate limits, users, one-time email links, sessions, daily usage and disposable-domain records.

### Still required before public launch
- Configure Resend (`RESEND_API_KEY`, `MAIL_FROM`) and test delivery using a verified sender domain.
- Configure Cloudflare Turnstile; the challenge becomes mandatory when the secret is set.
- Populate and maintain the disposable-domain list in D1 from a reputable source whose licence permits this use.
- Add production monitoring, alerting, provider-specific budgets and broader automated integration tests.
- Test registration, link expiry, one-time use, cookie/session expiry, logout, guest quota and account quota against a non-production D1 database.

Do not advertise the app as production-ready until those items are completed and the live Cloudflare configuration is tested.

## Environment variables / secrets

Configure under Cloudflare Pages → Settings → Variables and Secrets. Add only those you intend to use:

| Name | Purpose |
|---|---|
| `TURNSTILE_SITE_KEY` | Public Turnstile site key, used by the frontend widget |
| `TURNSTILE_SECRET_KEY` | Secret Turnstile key; enables mandatory server-side token verification |
| `RATE_LIMIT_SALT` | Long random secret used to hash client IPs before storing rate-limit keys |
| `RESEND_API_KEY` | Resend email API key used to send sign-in/verification links |
| `MAIL_FROM` | Verified sender, e.g. `Tracer <auth@yourdomain.com>` |
| `BRAVE_SEARCH_API_KEY` | Optional Brave Search API integration |
| `HIBP_API_KEY` | Optional Have I Been Pwned API integration |
| `HIBP_USER_AGENT` | Optional HIBP request identification string |
| `TWILIO_ACCOUNT_SID` | Optional Twilio Lookup account SID |
| `TWILIO_AUTH_TOKEN` | Optional Twilio Lookup secret |

Never put secret keys in `app.js`, HTML, GitHub, or a public issue. Never send secrets in chat. Configure both Turnstile keys together; if the secret is configured without a site key, searches will be blocked because visitors cannot obtain a token.

## D1 database setup

1. Install Node.js LTS on your computer.
2. Open a terminal in this project folder.
3. Run `npm install`.
4. Run `npx wrangler login` and approve access to your Cloudflare account.
5. Create the database: `npx wrangler d1 create tracer-db`.
6. Copy the returned database ID into `wrangler.toml`, replacing `REPLACE_WITH_YOUR_D1_DATABASE_ID`.
7. Apply the schema: `npx wrangler d1 migrations apply tracer-db --remote`.
8. In Cloudflare Pages project settings, ensure the D1 binding is named `DB` and points to `tracer-db` for production (and preview if you use preview deployments).

The D1 schema stores hashed IP-derived rate-limit keys, not raw IP addresses. `RATE_LIMIT_SALT` should be a long random secret before public launch. D1 `users`, `email_links`, `sessions`, `daily_usage`, `rate_limits`, and `disposable_domains` support account and quota routes. Guest searches are limited to 2 per day per hashed IP; verified users are limited to 25 per day.

## Email-link account setup

1. Create a Resend account and verify a domain you control.
2. Create an API key in Resend.
3. In Cloudflare Pages → Settings → Variables and Secrets, add `RESEND_API_KEY` as a secret and `MAIL_FROM` as a variable using an address on the verified domain.
4. Apply the D1 migration before enabling accounts.
5. Test a new email address, verify that the email arrives, click the link, and confirm `/api/auth/me` reports an authenticated user.
6. Confirm the link cannot be reused and expires after 20 minutes.

The app uses passwordless email links: new addresses are registered and verified when the link is clicked; verified addresses receive a sign-in link. Sessions use a 30-day HttpOnly, Secure, SameSite=Lax cookie.

## Turnstile setup

1. In Cloudflare Dashboard, open Turnstile and create a widget for your intended hostname(s).
2. Add the returned site key as `TURNSTILE_SITE_KEY` and secret key as `TURNSTILE_SECRET_KEY` in Pages Variables and Secrets.
3. Redeploy after setting both values.
4. Test a real challenge in the browser and confirm the API rejects a missing, invalid, and replayed token.

The server-side Siteverify call is essential: rendering the browser widget alone does not protect an endpoint.

## Deploy through GitHub

1. Push the **contents** of this folder to a GitHub repository (not an enclosing folder containing the project).
2. Cloudflare Dashboard → Workers & Pages → Create application → Pages → Connect to Git.
3. Select the repository. Framework preset: None. Build command: leave blank. Build output directory: `.`. Root directory: `/`.
4. Add the D1 binding and environment variables/secrets described above.
5. Deploy, then open `https://YOUR-PROJECT.pages.dev/api/health` and `https://YOUR-PROJECT.pages.dev/api/config`.
6. Verify that `databaseConfigured`, `turnstileConfigured`, and `emailDeliveryConfigured` in the health response reflect the settings you intended. Test `/api/auth/me` while signed out and signed in.

Cloudflare Pages Functions are in `/functions` at the project root. Do not use the Pages dashboard Direct Upload workflow for this repository because the project relies on Pages Functions.

## Local checks

```bash
npm install
npm run check
npx wrangler pages dev .
```

For local D1 development, use Wrangler's local D1 database and apply migrations locally. Do not point local tests at production data.

## Evidence semantics

- **Verified public profile**: the platform's public API returned a matching record. This does not verify the real-world identity or account owner.
- **Potential public profile**: a provider returned a candidate requiring manual confirmation.
- **Public web mention**: an indexed page result, not proof of ownership or association.
- **Unverified URL lead**: a URL pattern only; it must not be interpreted as an existing account.
- **No exact account**: no record was returned by that specific provider and query; it is not proof that the handle is unused elsewhere.
- **No breach found**: HIBP returned no matching breach records at query time; it does not prove an email has never been exposed.

## Privacy and responsible use

Searches are processed on demand. The current frontend stores up to ten recent queries in browser localStorage; users can clear this local history. API responses are marked `no-store`. Search terms are sent to the platform/provider being queried. Exact email search, HIBP and Twilio are opt-in. Do not use Tracer for stalking, harassment, doxxing, credential access, or consequential decisions. Respect source terms, rate limits, privacy law and the rights of people whose public information is queried.


## Build 3.2 development updates

The account dialog now shows a server-derived daily usage meter. Search history remains local to the browser in this build; no server-side investigation history is stored.

- Added `GET /api/account/usage` to display the current UTC-day search allowance for guests and signed-in users.
- Account email-domain checks now match blocked disposable domains and their subdomains. The denylist remains a small starter list and must be maintained from a reputable source before public launch.
- Email verification tokens are consumed atomically with a conditional `DELETE ... RETURNING`, preventing simultaneous reuse.
- `RATE_LIMIT_SALT` is now required when D1-backed request limiting or quota tracking is enabled. Configure a random secret of at least 32 characters in Cloudflare Pages secrets. Do not reuse the example or commit it.
- Added Node test coverage for email syntax and disposable-domain matching. Run `npm test` and `npm run check`.

### Remaining release blockers

Live email delivery, Cloudflare D1 binding/migrations, Turnstile, provider API calls, concurrent quota behavior on D1, account flows in a deployed preview, and operational monitoring still require staging tests. This build is not production-approved.

## Build 3.3 development updates

- Replaced the prior increment/read/rollback daily quota flow with one conditional SQLite UPSERT. The database only increments a counter while it is below the applicable limit; a zero-row change means the allowance is exhausted. This avoids a read-then-write race under concurrent requests. The app fails closed if D1 or the rate-limit salt is unavailable.
- Added `POST /api/account/delete`. It requires a same-origin `Origin` header and an authenticated session, then deletes the user's pending email links, email-link rate-limit key, daily usage row, active sessions and user record in a D1 batch. It clears the session cookie on success.
- Added account controls to clear browser-local search history and delete the signed-in account. Account deletion also clears the current browser's local search history. The UI asks for confirmation before the irreversible operation.
- Email addresses and phone numbers are now masked in newly saved local search history; existing local history entries are masked on the next history render. Full search inputs are still sent to the selected provider(s) as required to perform a search; this change only reduces what is retained in browser history.
- Added automated tests for quota grant/deny/fail-closed behavior and account-deletion origin/authentication checks.

### Build 3.3 verification

- `npm run check`: passed.
- `npm test`: 8 tests passed.
- SQLite migration and quota boundary exercise: passed; a guest counter accepted exactly two increments and rejected subsequent increments without exceeding two.
- Not performed: live D1 concurrency testing, Resend delivery, Turnstile Siteverify, provider API live calls, or Cloudflare preview deployment. Build 3.3 is a development build, not a production approval.

## Build 3.4 operations and cost visibility

This development build adds a private operations page at `/admin.html` and two admin-only endpoints:

- `GET /api/admin/metrics` — aggregate account totals, daily search totals, pending email-link count, and provider usage for the last 30 days.
- `POST /api/admin/retention` — removes expired rate-limit rows, expired email links and sessions, daily usage older than 90 days, provider aggregates older than 180 days, and operational events older than 30 days.

### Additional D1 migration

Apply `migrations/0002_operations.sql` after `0001_initial.sql`. With Wrangler, use the appropriate local or remote D1 migration command for your target environment. Do not run remote migrations until you intentionally configure the production D1 database.

### Additional secrets and optional cost estimates

Set `ADMIN_API_TOKEN` to a randomly generated secret of at least 32 characters. Keep it private; enter it only in the `/admin.html` page when you intentionally view metrics. The page does not persist the token to localStorage.

Optional per-call cost estimates are configured as integer micro-USD amounts (1 USD = 1,000,000 micro-USD):

- `BRAVE_COST_MICROUSD`
- `HIBP_COST_MICROUSD`
- `TWILIO_COST_MICROUSD`

These values default to zero. Set them from the provider's current pricing and your actual billing plan. The dashboard estimates usage cost from configured per-call amounts; it is not a billing ledger and does not include taxes, minimums, credits, tiers or other charges.

Provider telemetry records aggregate call/error counts only. It does not store the username, email, phone, IP address, query string or provider response. Telemetry write failures do not interrupt a search.

### Retention and privacy

The admin retention endpoint is manually triggered; it is not a scheduled job. Before public launch, configure a scheduled Worker/Cron Trigger or another operational process to run cleanup regularly. Daily usage is retained for 90 days and provider aggregate counts for 180 days. Expired auth/rate-limit rows are removed when cleanup runs. This policy should be reviewed against applicable legal requirements before launch.

The admin token is a single shared operator secret, not a multi-user admin identity system. Rotate it if exposed. Do not expose `/admin.html` as a public-facing feature or share the token with users.

### Build 3.4 updates

- Added aggregate provider call/error/cost telemetry for Brave, HIBP and Twilio requests.
- Added admin metrics endpoint and private admin dashboard.
- Added manually triggered data-retention cleanup endpoint.
- Added migration `0002_operations.sql` for aggregate telemetry tables.
- Added tests for admin bearer authentication and telemetry behavior.

This is still a development build. Live Cloudflare D1, secrets, admin endpoint and provider billing must be verified in a staging deployment before production use.


## Build 3.5 provider budget guardrails

Paid/usage-billed optional integrations (Brave Search, Have I Been Pwned and Twilio Lookup) now require a pre-call D1 budget reservation. A provider request is not sent unless its daily call cap and daily estimated-cost cap are configured and a single conditional D1 UPSERT reserves capacity. If D1 is unavailable, a limit is missing/invalid, or the cap is exhausted, the integration is skipped and the UI reports the reason. This is deliberately fail-closed.

Configure these integer environment variables from your current provider plan and pricing (costs are micro-USD; 1 USD = 1,000,000 micro-USD):

- `BRAVE_DAILY_CALL_LIMIT`, `BRAVE_DAILY_BUDGET_MICROUSD`, `BRAVE_COST_MICROUSD`
- `HIBP_DAILY_CALL_LIMIT`, `HIBP_DAILY_BUDGET_MICROUSD`, `HIBP_COST_MICROUSD`
- `TWILIO_DAILY_CALL_LIMIT`, `TWILIO_DAILY_BUDGET_MICROUSD`, `TWILIO_COST_MICROUSD`

The per-call estimate must be explicitly configured; use `0` only when your provider plan has no marginal per-call charge for that operation. The configured estimate is reserved before the provider call, so failed provider calls still consume the reserved estimate and count toward the call cap. Provider errors increment a separate aggregate counter. Set cost estimates conservatively and verify current provider billing terms. The dashboard is an operational estimate, not an invoice or hard financial guarantee: external providers can bill differently due to pricing tiers, taxes, credits, retries, rounding or product changes. The daily cap is a software guardrail, not a substitute for provider-side spending limits.

### Build 3.5 verification

- Paid-provider calls fail closed when D1 or explicit daily limits are absent.
- Provider budgets are reserved with a single conditional UPSERT before sending paid API requests.
- Added tests for missing configuration, atomic reservation, exhausted limits, per-call budget rejection and aggregate error recording.
- Live provider billing and concurrent D1 reservation behavior still require a staging deployment and provider test accounts.

## Build 3.6.1 — staging readiness

### Health endpoint improvements

`GET /api/health` now distinguishes a configured D1 binding from a reachable database and confirms that the expected tables from both migrations exist. It also reports whether Turnstile and email delivery are configured, plus boolean availability flags for optional providers. It never returns secret values. A healthy database requires `databaseConnected: true` and `migrationsReady: true`.

### Safe preview deployment workflow

Use a Git branch named `staging` and connect the repository to Cloudflare Pages using Git integration. Cloudflare creates preview deployments for non-production branches; keep `main` as the production branch and do not merge staging to `main` until the acceptance checklist passes. Cloudflare documents that Git integration supports preview deployments and that the dashboard Direct Upload workflow does not support uploading a Pages `/functions` directory.

Recommended Pages build settings for this repository:

- Framework preset: None
- Root directory: `/`
- Build command: leave blank
- Build output directory: `.`
- Production branch: `main`
- Preview branch: `staging`

Do not create a public production project as part of staging. Use a separate D1 database for staging. Never point a preview deployment at production data.

### Staging database

1. Create a separate D1 database named `tracer-staging-db`.
2. Copy its actual database ID into a staging-specific Wrangler configuration or configure the `DB` binding in the Pages Preview environment.
3. Apply both migrations to the staging database, in order: `0001_initial.sql`, then `0002_operations.sql`.
4. Confirm the Pages Preview binding name is exactly `DB` and points to `tracer-staging-db`.
5. Configure `RATE_LIMIT_SALT` as an encrypted secret (32+ random characters), `RESEND_API_KEY` as an encrypted secret, `MAIL_FROM` as a verified sender, and both Turnstile keys in the Preview environment.
6. Keep all optional paid provider keys absent for the first staging test. The provider integrations should remain disabled.
7. Set `ADMIN_API_TOKEN` as an encrypted secret for the Preview environment only.

### Read-only staging smoke test

After the preview deployment is available, run from the project folder:

```bash
TRACER_BASE_URL=https://YOUR-PREVIEW-DEPLOYMENT.pages.dev npm run smoke:staging
```

This checks the homepage, frontend script, health/migration state, public configuration, signed-out account endpoint, and unauthenticated admin rejection. It does not submit a search, send an email, or call a paid provider.

After those checks pass, test manually with a dedicated test email address that you control:

- Request an email sign-in link and confirm delivery.
- Confirm the link expires after 20 minutes and cannot be reused.
- Confirm the session survives a page reload and logout invalidates it.
- Confirm guest allowance stops at 2 searches and signed-in allowance stops at 25 per UTC day.
- Confirm Turnstile rejects missing/invalid/replayed tokens.
- Confirm disposable test domains are blocked; do not use a real person's address.
- Confirm account deletion removes the account and invalidates its session.
- Confirm admin metrics reject missing/wrong tokens and work only with the secret.
- Confirm no paid-provider calls are possible while their keys are absent.

Optional stricter smoke mode also requires Turnstile and email delivery flags:

```bash
TRACER_BASE_URL=https://YOUR-PREVIEW-DEPLOYMENT.pages.dev npm run smoke:staging -- --require-integrations
```

This build has not been deployed to Cloudflare. The staging smoke test must be run against an actual Preview URL before any live integration can be called verified.
