Import documentation: CONTINUE.md
Import the existing local project snapshot. This commit organizes existing files; it does not represent newly developed functionality.
This commit is contained in:
arsam
+972
@@ -0,0 +1,972 @@
|
||||
# CShare — Full Production Roadmap
|
||||
|
||||
This document covers everything built so far and everything that needs to be implemented to take CShare from a frontend prototype to a production-grade file-sharing service. Read it top to bottom before starting any backend work.
|
||||
|
||||
---
|
||||
|
||||
## What Is Already Built (Frontend)
|
||||
|
||||
### Pages
|
||||
- `/` — Landing page (Playfair italic hero, split layout, live upload preview, features grid, how-it-works)
|
||||
- `/upload` — Drag-and-drop upload zone with simulated progress, share link generation, "Share All"
|
||||
- `/dashboard` — File table + mobile card layout, search, stat cards, delete with confirm, stats modal
|
||||
- `/preview` — File view/download page (recipient side), countdown expiry timer
|
||||
- `/pricing` — 3-tier pricing page (Free / Pro / Team)
|
||||
- `/docs` — Documentation with sticky sidebar and API reference
|
||||
- `/about`, `/blog`, `/careers`, `/contact` — Company pages
|
||||
- `/privacy`, `/terms`, `/security`, `/gdpr` — Legal pages
|
||||
|
||||
### Components & System
|
||||
- `Navbar` — sticky, mobile hamburger, React Router wired
|
||||
- `Footer` — all links routed
|
||||
- `Toast` — context-based success/info/error notifications
|
||||
- `Icons` — 17 SVG icons
|
||||
- Design system in `index.css` — CSS variables, button/card/input classes, keyframes
|
||||
- Lenis smooth scrolling
|
||||
- React Router v7 with real URLs
|
||||
|
||||
### Stack
|
||||
- React 19, Vite 8, Tailwind CSS 4
|
||||
- Outfit (body) + Syne (headings) + Playfair Display (hero) — self-hosted TTFs
|
||||
- Electric blue accent (`#4A90FF`), dark charcoal background (`#0D0D0F`)
|
||||
|
||||
---
|
||||
|
||||
## What Needs to Be Built
|
||||
|
||||
---
|
||||
|
||||
## 1. Backend — Node.js / Express API
|
||||
|
||||
### Stack Decision
|
||||
```
|
||||
Runtime: Node.js 20 LTS
|
||||
Framework: Express 5 (or Fastify for higher throughput)
|
||||
Language: TypeScript (strict mode)
|
||||
Process mgr: PM2 (production) / tsx watch (development)
|
||||
```
|
||||
|
||||
### Directory Structure
|
||||
```
|
||||
/backend
|
||||
/src
|
||||
/config # env, constants, feature flags
|
||||
/db # Drizzle ORM schema + migrations
|
||||
/middleware # auth, rate-limit, error handler, logger
|
||||
/routes
|
||||
files.ts # upload, download, delete, list
|
||||
auth.ts # register, login, logout, refresh
|
||||
links.ts # generate, resolve, validate
|
||||
analytics.ts # view/download events, stats
|
||||
admin.ts # internal admin endpoints
|
||||
/services
|
||||
storage.ts # S3/R2 abstraction
|
||||
expiry.ts # cron-based file expiry worker
|
||||
email.ts # Nodemailer / Resend wrapper
|
||||
analytics.ts # event recording + aggregation
|
||||
/workers
|
||||
expiry.worker.ts # scheduled deletion
|
||||
cleanup.worker.ts # orphaned object cleanup
|
||||
/utils
|
||||
crypto.ts # link ID generation, hash helpers
|
||||
validate.ts # Zod schemas for all request bodies
|
||||
mime.ts # content-type detection + sanitization
|
||||
server.ts # Express app bootstrap
|
||||
worker.ts # standalone worker entry (BullMQ)
|
||||
/migrations # SQL migration files (Drizzle Kit)
|
||||
package.json
|
||||
tsconfig.json
|
||||
.env.example
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Server
|
||||
PORT=3001
|
||||
NODE_ENV=production
|
||||
|
||||
# Database
|
||||
DATABASE_URL=postgresql://user:pass@localhost:5432/cshare
|
||||
|
||||
# Redis
|
||||
REDIS_URL=redis://localhost:6379
|
||||
|
||||
# Storage (Cloudflare R2 or AWS S3)
|
||||
STORAGE_ENDPOINT=https://<account>.r2.cloudflarestorage.com
|
||||
STORAGE_BUCKET=cshare-files
|
||||
STORAGE_ACCESS_KEY=
|
||||
STORAGE_SECRET_KEY=
|
||||
STORAGE_PUBLIC_URL=https://files.asabbagh.com
|
||||
|
||||
# Auth
|
||||
JWT_ACCESS_SECRET=<64-char random hex>
|
||||
JWT_REFRESH_SECRET=<64-char random hex>
|
||||
ACCESS_TOKEN_TTL=15m
|
||||
REFRESH_TOKEN_TTL=30d
|
||||
|
||||
# Email
|
||||
EMAIL_FROM=noreply@cshare.app
|
||||
RESEND_API_KEY=
|
||||
|
||||
# Security
|
||||
BCRYPT_ROUNDS=12
|
||||
MAX_FILE_SIZE_FREE=104857600 # 100 MB
|
||||
MAX_FILE_SIZE_PRO=524288000 # 500 MB
|
||||
FREE_UPLOAD_MONTHLY_LIMIT=5
|
||||
|
||||
# Rate limiting
|
||||
RATE_LIMIT_WINDOW_MS=60000
|
||||
RATE_LIMIT_MAX_UPLOAD=10
|
||||
RATE_LIMIT_MAX_DOWNLOAD=60
|
||||
|
||||
# Frontend URL (CORS)
|
||||
FRONTEND_URL=https://cshare.app
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Database — PostgreSQL
|
||||
|
||||
### Why PostgreSQL
|
||||
- Transactional integrity for file + link creation (atomically or not at all)
|
||||
- Row-level expiry queries with proper index support
|
||||
- JSONB for extensible metadata without schema churn
|
||||
- Native UUID support
|
||||
- Strong enum types for plan, status
|
||||
|
||||
### Schema (Drizzle ORM)
|
||||
|
||||
```typescript
|
||||
// db/schema.ts
|
||||
|
||||
import { pgTable, uuid, text, integer, bigint, boolean, timestamp, pgEnum, jsonb, index } from 'drizzle-orm/pg-core';
|
||||
|
||||
// ── Enums ──────────────────────────────────────────────────────────
|
||||
export const planEnum = pgEnum('plan', ['free', 'pro', 'team']);
|
||||
export const fileStatusEnum = pgEnum('file_status', ['pending', 'uploaded', 'deleted', 'expired']);
|
||||
|
||||
// ── Users ──────────────────────────────────────────────────────────
|
||||
export const users = pgTable('users', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
email: text('email').notNull().unique(),
|
||||
passwordHash: text('password_hash').notNull(),
|
||||
plan: planEnum('plan').notNull().default('free'),
|
||||
verified: boolean('verified').notNull().default(false),
|
||||
uploadCount: integer('upload_count').notNull().default(0), // rolling monthly counter
|
||||
uploadResetAt: timestamp('upload_reset_at').notNull().defaultNow(),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
updatedAt: timestamp('updated_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
emailIdx: index('users_email_idx').on(t.email),
|
||||
}));
|
||||
|
||||
// ── Refresh tokens ─────────────────────────────────────────────────
|
||||
export const refreshTokens = pgTable('refresh_tokens', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
userId: uuid('user_id').notNull().references(() => users.id, { onDelete: 'cascade' }),
|
||||
tokenHash: text('token_hash').notNull().unique(), // SHA-256 of the raw token
|
||||
expiresAt: timestamp('expires_at').notNull(),
|
||||
revokedAt: timestamp('revoked_at'),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
userIdx: index('refresh_tokens_user_idx').on(t.userId),
|
||||
hashIdx: index('refresh_tokens_hash_idx').on(t.tokenHash),
|
||||
}));
|
||||
|
||||
// ── Files ──────────────────────────────────────────────────────────
|
||||
export const files = pgTable('files', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
userId: uuid('user_id').references(() => users.id, { onDelete: 'set null' }), // null = anonymous
|
||||
storageKey: text('storage_key').notNull().unique(), // e.g. "uploads/2026/05/abc123.pdf"
|
||||
originalName: text('original_name').notNull(),
|
||||
mimeType: text('mime_type').notNull(),
|
||||
sizeBytes: bigint('size_bytes', { mode: 'number' }).notNull(),
|
||||
status: fileStatusEnum('file_status').notNull().default('pending'),
|
||||
passwordHash: text('password_hash'), // null = no password
|
||||
expiresAt: timestamp('expires_at').notNull(),
|
||||
deletedAt: timestamp('deleted_at'),
|
||||
metadata: jsonb('metadata').default({}), // extensible bag
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
userIdx: index('files_user_idx').on(t.userId),
|
||||
statusIdx: index('files_status_idx').on(t.status),
|
||||
expiryIdx: index('files_expires_at_idx').on(t.expiresAt), // for expiry worker queries
|
||||
}));
|
||||
|
||||
// ── Share links ────────────────────────────────────────────────────
|
||||
export const links = pgTable('links', {
|
||||
id: text('id').primaryKey(), // short random ID, e.g. "x7k2m9"
|
||||
fileId: uuid('file_id').notNull().references(() => files.id, { onDelete: 'cascade' }),
|
||||
active: boolean('active').notNull().default(true),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
fileIdx: index('links_file_idx').on(t.fileId),
|
||||
}));
|
||||
|
||||
// ── Analytics events ───────────────────────────────────────────────
|
||||
export const events = pgTable('events', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
linkId: text('link_id').notNull().references(() => links.id, { onDelete: 'cascade' }),
|
||||
type: text('type').notNull(), // 'view' | 'download'
|
||||
ip: text('ip'), // hashed, not raw
|
||||
userAgent: text('user_agent'),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
linkIdx: index('events_link_idx').on(t.linkId),
|
||||
typeIdx: index('events_type_idx').on(t.type),
|
||||
timeIdx: index('events_created_at_idx').on(t.createdAt),
|
||||
}));
|
||||
|
||||
// ── Email verification tokens ──────────────────────────────────────
|
||||
export const verificationTokens = pgTable('verification_tokens', {
|
||||
token: text('token').primaryKey(), // random UUID stored as-is (not sensitive long-term)
|
||||
userId: uuid('user_id').notNull().references(() => users.id, { onDelete: 'cascade' }),
|
||||
expiresAt: timestamp('expires_at').notNull(),
|
||||
usedAt: timestamp('used_at'),
|
||||
});
|
||||
|
||||
// ── Password reset tokens ──────────────────────────────────────────
|
||||
export const passwordResetTokens = pgTable('password_reset_tokens', {
|
||||
tokenHash: text('token_hash').primaryKey(), // SHA-256
|
||||
userId: uuid('user_id').notNull().references(() => users.id, { onDelete: 'cascade' }),
|
||||
expiresAt: timestamp('expires_at').notNull(),
|
||||
usedAt: timestamp('used_at'),
|
||||
});
|
||||
|
||||
// ── API keys (Pro/Team) ────────────────────────────────────────────
|
||||
export const apiKeys = pgTable('api_keys', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
userId: uuid('user_id').notNull().references(() => users.id, { onDelete: 'cascade' }),
|
||||
name: text('name').notNull(),
|
||||
keyHash: text('key_hash').notNull().unique(), // SHA-256 of raw key
|
||||
prefix: text('prefix').notNull(), // first 8 chars for display, e.g. "csh_abc1"
|
||||
lastUsedAt: timestamp('last_used_at'),
|
||||
expiresAt: timestamp('expires_at'),
|
||||
revokedAt: timestamp('revoked_at'),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
}, (t) => ({
|
||||
userIdx: index('api_keys_user_idx').on(t.userId),
|
||||
hashIdx: index('api_keys_hash_idx').on(t.keyHash),
|
||||
}));
|
||||
```
|
||||
|
||||
### Migrations Strategy
|
||||
- Use **Drizzle Kit** (`drizzle-kit generate`, `drizzle-kit migrate`)
|
||||
- Migration files committed to git and run in CI before deploy
|
||||
- Never edit past migrations — always add new ones
|
||||
- Production: run migrations as a pre-start step, not in the application boot path
|
||||
|
||||
```bash
|
||||
# Generate migration after schema change
|
||||
npx drizzle-kit generate
|
||||
|
||||
# Run pending migrations
|
||||
npx drizzle-kit migrate
|
||||
|
||||
# Inspect current DB state
|
||||
npx drizzle-kit studio
|
||||
```
|
||||
|
||||
### Key Indexes to Add Post-Launch (monitor query plans first)
|
||||
- Partial index on `files` where `status = 'uploaded'` and `expires_at < now()` — used by the expiry worker every minute
|
||||
- Partial index on `events` for last-30-days window queries — used by analytics
|
||||
|
||||
---
|
||||
|
||||
## 3. Redis
|
||||
|
||||
### Why Redis
|
||||
- Rate limiting (sliding window counters, per-IP and per-user)
|
||||
- Upload session state (presigned URL → file ID mapping while upload is in flight)
|
||||
- Short-lived caches (file metadata for hot links, avoids DB hit on every download)
|
||||
- BullMQ job queue (expiry worker, email queue, cleanup worker)
|
||||
- Session invalidation list (jti blocklist for logged-out JWTs before they expire)
|
||||
|
||||
### Key Namespaces
|
||||
```
|
||||
# Rate limiting
|
||||
rl:upload:{ip} # sliding window, 10 uploads/min per IP
|
||||
rl:download:{ip} # sliding window, 60 downloads/min per IP
|
||||
rl:auth:{ip} # 10 attempts/15 min, then lockout
|
||||
|
||||
# Upload sessions (TTL: 30 min)
|
||||
upload_session:{sessionId} -> { fileId, userId, expectedSize, mimeType }
|
||||
|
||||
# File metadata cache (TTL: 5 min)
|
||||
file_meta:{fileId} -> JSON of file row (invalidated on delete)
|
||||
link_meta:{linkId} -> JSON of link + file join
|
||||
|
||||
# JWT blocklist (jti → expiry timestamp)
|
||||
jwt_blocklist:{jti} -> "1" # TTL = token's remaining lifetime
|
||||
|
||||
# Monthly upload counter per free user (TTL: rolling 31 days)
|
||||
upload_count:{userId}:{YYYY-MM} -> integer
|
||||
|
||||
# BullMQ queues (managed by BullMQ, don't write to these directly)
|
||||
bull:expiry # file expiry jobs
|
||||
bull:email # transactional email jobs
|
||||
bull:cleanup # orphan storage cleanup jobs
|
||||
```
|
||||
|
||||
### Rate Limiting Implementation
|
||||
```typescript
|
||||
// middleware/rateLimit.ts
|
||||
import { RateLimiterRedis } from 'rate-limiter-flexible';
|
||||
|
||||
export const uploadLimiter = new RateLimiterRedis({
|
||||
storeClient: redis,
|
||||
keyPrefix: 'rl:upload',
|
||||
points: 10, // 10 uploads
|
||||
duration: 60, // per 60 seconds
|
||||
blockDuration: 300, // block for 5 min if exceeded
|
||||
});
|
||||
|
||||
export const downloadLimiter = new RateLimiterRedis({
|
||||
storeClient: redis,
|
||||
keyPrefix: 'rl:download',
|
||||
points: 60,
|
||||
duration: 60,
|
||||
});
|
||||
|
||||
export const authLimiter = new RateLimiterRedis({
|
||||
storeClient: redis,
|
||||
keyPrefix: 'rl:auth',
|
||||
points: 10,
|
||||
duration: 900, // 15 minutes
|
||||
blockDuration: 3600, // 1 hour lockout
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. File Storage — Cloudflare R2
|
||||
|
||||
### Why R2 over S3
|
||||
- Zero egress fees (biggest cost driver for a file-sharing service)
|
||||
- S3-compatible API — drop-in replacement
|
||||
- Built-in CDN via Cloudflare network
|
||||
- Cheaper storage per GB
|
||||
|
||||
### Upload Flow
|
||||
```
|
||||
Client Backend R2
|
||||
| | |
|
||||
|-- POST /api/upload -->| |
|
||||
| (file stream) | |
|
||||
| |-- PutObject ----->|
|
||||
| | (stream body) |
|
||||
| |<-- ETag ----------|
|
||||
| | |
|
||||
| |-- INSERT file row |
|
||||
| |-- INSERT link row |
|
||||
|<-- { linkId, url } --| |
|
||||
```
|
||||
|
||||
**Alternative for large files — presigned URLs:**
|
||||
```
|
||||
Client Backend R2
|
||||
| | |
|
||||
|-- POST /api/upload/init -->| |
|
||||
| |-- createPresign ->|
|
||||
| |<-- presignedUrl --|
|
||||
|<-- { uploadUrl, sessionId } --| |
|
||||
| | |
|
||||
|---- PUT presignedUrl ------------------>|
|
||||
|<---- 200 --------------------------------|
|
||||
| | |
|
||||
|-- POST /api/upload/complete -->| |
|
||||
| { sessionId } | |
|
||||
| |-- HeadObject ---->|
|
||||
| |<-- metadata ------|
|
||||
| |-- INSERT file/link|
|
||||
|<-- { linkId, url } --| |
|
||||
```
|
||||
|
||||
### Storage Key Format
|
||||
```
|
||||
uploads/{year}/{month}/{uuidv4}/{sanitized-filename}
|
||||
|
||||
Example:
|
||||
uploads/2026/05/f47ac10b-58cc-4372-a567-0e02b2c3d479/Q4-report.pdf
|
||||
```
|
||||
|
||||
- UUID in path prevents enumeration
|
||||
- Year/month prefix enables efficient lifecycle rules
|
||||
- Sanitized filename strips path traversal characters
|
||||
|
||||
### File Deletion
|
||||
```typescript
|
||||
// services/storage.ts
|
||||
async function deleteFile(storageKey: string): Promise<void> {
|
||||
await s3.send(new DeleteObjectCommand({
|
||||
Bucket: process.env.STORAGE_BUCKET,
|
||||
Key: storageKey,
|
||||
}));
|
||||
}
|
||||
```
|
||||
|
||||
Always delete from storage AND update DB status in the same operation. If storage delete fails, mark the file as `pending_deletion` and retry via the cleanup worker.
|
||||
|
||||
### Content-Type Handling
|
||||
- Extract MIME type from actual file bytes (use `file-type` library), not from the browser's reported Content-Type
|
||||
- Serve files with `Content-Disposition: attachment` to prevent browser execution of uploaded HTML/JS
|
||||
- Never serve with `Content-Type: text/html` regardless of what was uploaded
|
||||
|
||||
---
|
||||
|
||||
## 5. Authentication & Authorization
|
||||
|
||||
### Strategy
|
||||
- **JWT access tokens** (short-lived, 15 min) — stored in memory on the client, never localStorage
|
||||
- **HTTP-only refresh tokens** (30 days) — cookie, not accessible to JavaScript
|
||||
- **API keys** for programmatic access (Pro/Team) — hashed in DB, `csh_` prefix
|
||||
|
||||
### Auth Endpoints
|
||||
```
|
||||
POST /api/auth/register { email, password }
|
||||
POST /api/auth/login { email, password } -> sets refresh cookie
|
||||
POST /api/auth/refresh (reads refresh cookie) -> new access token
|
||||
POST /api/auth/logout revokes refresh token, clears cookie
|
||||
POST /api/auth/verify-email { token }
|
||||
POST /api/auth/forgot-password { email }
|
||||
POST /api/auth/reset-password { token, newPassword }
|
||||
GET /api/auth/me returns current user
|
||||
```
|
||||
|
||||
### Token Lifecycle
|
||||
```typescript
|
||||
// Access token payload
|
||||
{
|
||||
sub: userId, // UUID
|
||||
email: user.email,
|
||||
plan: 'pro',
|
||||
jti: randomUUID(), // for blocklist
|
||||
iat: now,
|
||||
exp: now + 15min,
|
||||
}
|
||||
|
||||
// Refresh token (stored in DB as SHA-256 hash)
|
||||
// Raw token sent as HTTP-only cookie, never logged
|
||||
```
|
||||
|
||||
### Password Hashing
|
||||
```typescript
|
||||
import bcrypt from 'bcrypt';
|
||||
const ROUNDS = parseInt(process.env.BCRYPT_ROUNDS); // 12
|
||||
|
||||
const hash = await bcrypt.hash(password, ROUNDS);
|
||||
const valid = await bcrypt.compare(password, hash);
|
||||
```
|
||||
|
||||
### Plan-based Authorization Middleware
|
||||
```typescript
|
||||
// middleware/auth.ts
|
||||
export const requirePlan = (minPlan: 'free' | 'pro' | 'team') =>
|
||||
(req, res, next) => {
|
||||
const PLAN_RANK = { free: 0, pro: 1, team: 2 };
|
||||
if (PLAN_RANK[req.user.plan] < PLAN_RANK[minPlan]) {
|
||||
return res.status(403).json({ error: 'Plan upgrade required' });
|
||||
}
|
||||
next();
|
||||
};
|
||||
|
||||
// Usage
|
||||
router.post('/upload', requireAuth, requirePlan('pro'), uploadHandler);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. API Routes
|
||||
|
||||
### Files
|
||||
```
|
||||
POST /api/v1/files/upload Multipart upload (stream to R2)
|
||||
POST /api/v1/files/upload/init Init presigned upload (returns URL + sessionId)
|
||||
POST /api/v1/files/upload/complete Finalize presigned upload
|
||||
GET /api/v1/files List user's files (paginated)
|
||||
GET /api/v1/files/:id Get file metadata
|
||||
DELETE /api/v1/files/:id Delete file + storage object + invalidate links
|
||||
PATCH /api/v1/files/:id Update expiry, password, name
|
||||
```
|
||||
|
||||
### Links
|
||||
```
|
||||
GET /api/v1/links/:linkId Resolve link -> file metadata (recipient endpoint)
|
||||
GET /api/v1/links/:linkId/download Redirect to signed R2 URL (or stream)
|
||||
POST /api/v1/links/:linkId/verify Verify password for password-protected links
|
||||
```
|
||||
|
||||
### Analytics
|
||||
```
|
||||
POST /api/v1/events Record view/download event (called from link resolution)
|
||||
GET /api/v1/files/:id/analytics View + download counts, timeline
|
||||
GET /api/v1/dashboard/stats Aggregate stats for the current user
|
||||
```
|
||||
|
||||
### Auth (see above)
|
||||
|
||||
### Response Format
|
||||
```typescript
|
||||
// Success
|
||||
{ data: T, meta?: { page, limit, total } }
|
||||
|
||||
// Error
|
||||
{ error: string, code: string, details?: Record<string, string> }
|
||||
|
||||
// HTTP status codes used consistently:
|
||||
// 200 — OK
|
||||
// 201 — Created
|
||||
// 204 — No Content (delete success)
|
||||
// 400 — Validation error
|
||||
// 401 — Unauthenticated
|
||||
// 403 — Forbidden (plan limit or wrong owner)
|
||||
// 404 — Not found (or intentionally opaque for security)
|
||||
// 409 — Conflict (e.g., email already registered)
|
||||
// 413 — Payload too large
|
||||
// 422 — Unprocessable entity
|
||||
// 429 — Rate limited
|
||||
// 500 — Internal server error (never expose stack traces)
|
||||
```
|
||||
|
||||
### Request Validation (Zod)
|
||||
```typescript
|
||||
// routes/files.ts
|
||||
const uploadSchema = z.object({
|
||||
expiresIn: z.number().int().min(3600).max(7776000).default(604800), // 1h to 90d
|
||||
password: z.string().min(4).max(128).optional(),
|
||||
});
|
||||
|
||||
router.post('/upload', requireAuth, upload.single('file'), async (req, res) => {
|
||||
const body = uploadSchema.safeParse(req.body);
|
||||
if (!body.success) return res.status(400).json({ error: 'Validation failed', details: body.error.flatten() });
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Background Workers
|
||||
|
||||
### Expiry Worker (runs every 60 seconds)
|
||||
```typescript
|
||||
// workers/expiry.worker.ts
|
||||
import { CronJob } from 'cron';
|
||||
|
||||
async function runExpiry() {
|
||||
const expired = await db
|
||||
.select({ id: files.id, storageKey: files.storageKey })
|
||||
.from(files)
|
||||
.where(
|
||||
and(
|
||||
eq(files.status, 'uploaded'),
|
||||
lt(files.expiresAt, new Date()),
|
||||
)
|
||||
)
|
||||
.limit(100); // process in batches
|
||||
|
||||
for (const file of expired) {
|
||||
try {
|
||||
await deleteFromStorage(file.storageKey);
|
||||
await db.update(files)
|
||||
.set({ status: 'expired', deletedAt: new Date() })
|
||||
.where(eq(files.id, file.id));
|
||||
await db.update(links).set({ active: false }).where(eq(links.fileId, file.id));
|
||||
await redis.del(`file_meta:${file.id}`);
|
||||
} catch (err) {
|
||||
logger.error({ fileId: file.id, err }, 'Expiry failed — will retry next tick');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
new CronJob('* * * * *', runExpiry).start();
|
||||
```
|
||||
|
||||
### Cleanup Worker (runs every 6 hours)
|
||||
- List all objects in R2 bucket
|
||||
- Cross-reference with DB — any storage key not in DB is an orphan
|
||||
- Delete orphans (they're files that failed mid-upload and never got a DB row)
|
||||
|
||||
### Email Worker (BullMQ)
|
||||
```typescript
|
||||
// Queue jobs for:
|
||||
// - Email verification on register
|
||||
// - Password reset
|
||||
// - File expiry warning (24h before, Pro plan)
|
||||
// - Usage limit warning (at 80% of monthly Free quota)
|
||||
|
||||
const emailQueue = new Queue('email', { connection: redis });
|
||||
|
||||
// Producer
|
||||
await emailQueue.add('verify-email', {
|
||||
to: user.email,
|
||||
userId: user.id,
|
||||
token: verificationToken,
|
||||
});
|
||||
|
||||
// Consumer
|
||||
const worker = new Worker('email', async (job) => {
|
||||
if (job.name === 'verify-email') await sendVerificationEmail(job.data);
|
||||
if (job.name === 'reset-password') await sendResetEmail(job.data);
|
||||
}, { connection: redis });
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Infrastructure & Deployment
|
||||
|
||||
### Architecture Overview
|
||||
```
|
||||
Internet
|
||||
│
|
||||
▼
|
||||
Cloudflare (DNS + DDoS + CDN for static assets)
|
||||
│
|
||||
├──> Nginx (reverse proxy, port 443 → 3000 frontend, 3001 backend)
|
||||
│ SSL termination (Let's Encrypt via Certbot)
|
||||
│
|
||||
├──> Node.js API (PM2, port 3001)
|
||||
│ └── connects to PostgreSQL (port 5432, local socket)
|
||||
│ └── connects to Redis (port 6379, local socket)
|
||||
│
|
||||
├──> PostgreSQL 16 (local, not exposed to internet)
|
||||
│
|
||||
├──> Redis 7 (local, not exposed to internet)
|
||||
│
|
||||
└──> Cloudflare R2 (file storage, via HTTPS API)
|
||||
```
|
||||
|
||||
### Nginx Config for API
|
||||
```nginx
|
||||
# /etc/nginx/sites-enabled/cshare-api
|
||||
server {
|
||||
listen 3001;
|
||||
server_name 188.245.39.229;
|
||||
|
||||
# Max upload size (must match backend limit)
|
||||
client_max_body_size 520M;
|
||||
client_body_buffer_size 10M;
|
||||
|
||||
# Timeouts for large file uploads
|
||||
proxy_read_timeout 600s;
|
||||
proxy_send_timeout 600s;
|
||||
proxy_connect_timeout 10s;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:4000; # internal port
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection 'upgrade';
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### PM2 Ecosystem File
|
||||
```javascript
|
||||
// ecosystem.config.js
|
||||
module.exports = {
|
||||
apps: [
|
||||
{
|
||||
name: 'cshare-api',
|
||||
script: 'dist/server.js',
|
||||
instances: 'max', // one per CPU core
|
||||
exec_mode: 'cluster',
|
||||
watch: false,
|
||||
env_production: {
|
||||
NODE_ENV: 'production',
|
||||
},
|
||||
error_file: '/var/log/cshare/api-error.log',
|
||||
out_file: '/var/log/cshare/api-out.log',
|
||||
merge_logs: true,
|
||||
log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
|
||||
},
|
||||
{
|
||||
name: 'cshare-worker',
|
||||
script: 'dist/worker.js',
|
||||
instances: 1, // single worker instance
|
||||
exec_mode: 'fork',
|
||||
watch: false,
|
||||
env_production: {
|
||||
NODE_ENV: 'production',
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### PostgreSQL Hardening
|
||||
```sql
|
||||
-- Create dedicated app user, never use superuser
|
||||
CREATE USER cshare WITH PASSWORD 'strong-random-password';
|
||||
CREATE DATABASE cshare OWNER cshare;
|
||||
GRANT ALL PRIVILEGES ON DATABASE cshare TO cshare;
|
||||
|
||||
-- In postgresql.conf
|
||||
max_connections = 100
|
||||
shared_buffers = 256MB -- 25% of RAM on a 1GB server
|
||||
effective_cache_size = 768MB
|
||||
work_mem = 4MB
|
||||
maintenance_work_mem = 64MB
|
||||
wal_level = replica -- enables point-in-time recovery
|
||||
archive_mode = on
|
||||
log_min_duration_statement = 500ms -- log slow queries
|
||||
```
|
||||
|
||||
### Redis Hardening
|
||||
```conf
|
||||
# /etc/redis/redis.conf
|
||||
bind 127.0.0.1 # local only, never expose to internet
|
||||
requirepass strong-random-password
|
||||
maxmemory 256mb
|
||||
maxmemory-policy allkeys-lru
|
||||
save 900 1 # RDB snapshot
|
||||
save 300 10
|
||||
appendonly yes # AOF for durability
|
||||
appendfsync everysec
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Security Checklist
|
||||
|
||||
### Input/Output
|
||||
- [ ] All file sizes validated server-side before writing to storage
|
||||
- [ ] MIME type detected from bytes, not from Content-Type header
|
||||
- [ ] All file names sanitized (strip `../`, null bytes, control characters)
|
||||
- [ ] Download served with `Content-Disposition: attachment; filename="..."` (prevents XSS via HTML files)
|
||||
- [ ] `X-Content-Type-Options: nosniff` on all responses
|
||||
- [ ] All HTML rendered by the frontend — API is JSON only, no server-side HTML
|
||||
|
||||
### Auth
|
||||
- [ ] Passwords hashed with bcrypt (rounds ≥ 12)
|
||||
- [ ] Refresh tokens stored as SHA-256 hash in DB (never raw)
|
||||
- [ ] JWT `jti` blocklist in Redis for immediate invalidation on logout
|
||||
- [ ] API keys stored as SHA-256 hash (never recoverable after generation)
|
||||
- [ ] Email verification required before upload on registered accounts
|
||||
- [ ] Rate limiting on auth endpoints (10 attempts / 15 min per IP)
|
||||
|
||||
### Infrastructure
|
||||
- [ ] PostgreSQL not accessible from internet (bind to `127.0.0.1` or Unix socket)
|
||||
- [ ] Redis not accessible from internet
|
||||
- [ ] All secrets in `.env`, never committed to git
|
||||
- [ ] `.env.example` committed with placeholder values only
|
||||
- [ ] UFW firewall — only 22, 80, 443, 3000 exposed externally
|
||||
- [ ] Fail2ban on SSH
|
||||
- [ ] Non-root user to run Node processes
|
||||
- [ ] Let's Encrypt TLS certificate auto-renewed via Certbot systemd timer
|
||||
|
||||
### Headers (set via Helmet.js)
|
||||
```typescript
|
||||
app.use(helmet({
|
||||
contentSecurityPolicy: {
|
||||
directives: {
|
||||
defaultSrc: ["'self'"],
|
||||
scriptSrc: ["'self'"],
|
||||
styleSrc: ["'self'", "'unsafe-inline'"], // Tailwind needs this
|
||||
imgSrc: ["'self'", "data:"],
|
||||
connectSrc: ["'self'"],
|
||||
fontSrc: ["'self'"],
|
||||
objectSrc: ["'none'"],
|
||||
upgradeInsecureRequests: [],
|
||||
},
|
||||
},
|
||||
crossOriginEmbedderPolicy: false, // needed for some browser download behaviors
|
||||
}));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Frontend Integration Points
|
||||
|
||||
When the backend is ready, each of these frontend mocks needs to be replaced:
|
||||
|
||||
### UploadPage.jsx
|
||||
- Replace `simulateProgress` with a real `XMLHttpRequest` or `fetch` with `ReadableStream` to track upload progress
|
||||
- Replace `generateShareLink` with `POST /api/v1/files/upload`
|
||||
- On success, store the real link returned by the API
|
||||
|
||||
### DashboardPage.jsx
|
||||
- Replace `mockFiles` import with `GET /api/v1/files` (paginated)
|
||||
- Replace `deleteFile` local state mutation with `DELETE /api/v1/files/:id`
|
||||
- Replace static `totalViews` / `totalDownloads` with `GET /api/v1/dashboard/stats`
|
||||
|
||||
### FileViewPage.jsx
|
||||
- Replace `sharedFile` import with `GET /api/v1/links/:linkId`
|
||||
- Replace fake download with `GET /api/v1/links/:linkId/download` (redirect to signed R2 URL)
|
||||
- Handle password-protected links (prompt UI → `POST /api/v1/links/:linkId/verify`)
|
||||
|
||||
### Navbar + auth state
|
||||
- Add a `useAuth` context that holds the decoded access token payload
|
||||
- Conditionally show "Log in" / "Sign up" vs. user avatar dropdown
|
||||
- Refresh access token silently using the HTTP-only cookie before it expires
|
||||
|
||||
### New pages needed on the frontend
|
||||
- `/login` — email + password form
|
||||
- `/register` — email + password + plan selection
|
||||
- `/account` — manage subscription, change password, generate API keys, delete account
|
||||
- `/verify-email` — handles the `?token=` query param from the verification email
|
||||
- `/reset-password` — handles the `?token=` query param
|
||||
- `404.jsx` — for expired links and unknown routes
|
||||
|
||||
---
|
||||
|
||||
## 11. Payments — Stripe
|
||||
|
||||
### Integration Points
|
||||
```
|
||||
POST /api/v1/billing/checkout Create Stripe Checkout session (upgrade to Pro/Team)
|
||||
POST /api/v1/billing/portal Create Stripe Customer Portal session (manage subscription)
|
||||
POST /api/v1/billing/webhook Stripe webhook handler (subscription events)
|
||||
GET /api/v1/billing/status Current subscription status for logged-in user
|
||||
```
|
||||
|
||||
### Webhook Events to Handle
|
||||
```
|
||||
checkout.session.completed -> upgrade user plan in DB
|
||||
customer.subscription.updated -> sync plan changes
|
||||
customer.subscription.deleted -> downgrade to free, enforce limits immediately
|
||||
invoice.payment_failed -> email user, flag account
|
||||
```
|
||||
|
||||
### Plan Enforcement
|
||||
- On upload: check `user.plan` + `user.uploadCount` for Free users
|
||||
- Monthly counter (`upload_count:{userId}:{YYYY-MM}` in Redis) resets on the first of each month via a cron job
|
||||
- On downgrade: do not delete existing files, but block new uploads until user is under the Free limit
|
||||
|
||||
---
|
||||
|
||||
## 12. Observability
|
||||
|
||||
### Logging (Pino)
|
||||
```typescript
|
||||
import pino from 'pino';
|
||||
const logger = pino({ level: process.env.LOG_LEVEL || 'info' });
|
||||
|
||||
// Log shape for every request
|
||||
logger.info({ method, path, statusCode, durationMs, userId, fileId });
|
||||
// Never log: passwords, tokens, file contents, full IP addresses
|
||||
```
|
||||
|
||||
### Metrics (to export to Prometheus or Datadog)
|
||||
```
|
||||
cshare_uploads_total counter, labels: plan, status
|
||||
cshare_upload_bytes_total counter
|
||||
cshare_downloads_total counter
|
||||
cshare_active_files_gauge gauge (refreshed every 5 min)
|
||||
cshare_expired_files_total counter
|
||||
cshare_api_request_duration_ms histogram, labels: route, method, status
|
||||
cshare_db_query_duration_ms histogram
|
||||
cshare_redis_hit_ratio gauge
|
||||
```
|
||||
|
||||
### Error Tracking
|
||||
- Integrate Sentry (`@sentry/node`) for unhandled exceptions and slow transactions
|
||||
- Set up a Sentry project, add `SENTRY_DSN` to env
|
||||
- Sample rate: 100% errors, 10% performance traces in production
|
||||
|
||||
### Health Check Endpoint
|
||||
```typescript
|
||||
// GET /health
|
||||
{
|
||||
status: 'ok',
|
||||
db: 'connected', // result of `SELECT 1`
|
||||
redis: 'connected', // result of PING
|
||||
storage: 'reachable', // result of HeadBucket
|
||||
uptime: 12345, // process.uptime()
|
||||
version: '1.0.0',
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Implementation Order
|
||||
|
||||
Work in this exact sequence to always have something deployable:
|
||||
|
||||
```
|
||||
Phase 1 — Data Layer (no network, just local)
|
||||
[ ] PostgreSQL installed + hardened
|
||||
[ ] Redis installed + hardened
|
||||
[ ] Drizzle schema written + first migration run
|
||||
[ ] Test: can create a user, a file row, and query it
|
||||
|
||||
Phase 2 — API Core (no auth, no storage)
|
||||
[ ] Express app with Helmet, CORS, logger, error handler
|
||||
[ ] POST /api/v1/files/upload (local disk for now, fake link response)
|
||||
[ ] GET /api/v1/links/:linkId (returns hardcoded metadata)
|
||||
[ ] Frontend: wire UploadPage to the real endpoint, replace simulateProgress
|
||||
[ ] Frontend: wire FileViewPage to the real endpoint
|
||||
[ ] Confirm end-to-end works with local disk storage
|
||||
|
||||
Phase 3 — Storage
|
||||
[ ] Cloudflare R2 bucket created + access keys
|
||||
[ ] storage.ts service (upload, delete, signed URL generation)
|
||||
[ ] Replace local disk with R2 in upload handler
|
||||
[ ] Test: upload → get link → download file → file is actually the right bytes
|
||||
|
||||
Phase 4 — Auth
|
||||
[ ] POST /api/auth/register + bcrypt
|
||||
[ ] POST /api/auth/login + JWT issuance + refresh cookie
|
||||
[ ] POST /api/auth/refresh + GET /api/auth/me
|
||||
[ ] POST /api/auth/logout + blocklist jti in Redis
|
||||
[ ] requireAuth middleware on upload/list/delete endpoints
|
||||
[ ] Frontend: /login and /register pages + useAuth context
|
||||
|
||||
Phase 5 — Rate Limiting + Plan Enforcement
|
||||
[ ] rate-limiter-flexible on upload, download, auth routes
|
||||
[ ] Free monthly upload counter in Redis
|
||||
[ ] Plan checks in upload handler (size limit, count limit)
|
||||
|
||||
Phase 6 — Workers
|
||||
[ ] Expiry worker (cron every 60s)
|
||||
[ ] BullMQ email queue + Resend integration
|
||||
[ ] Email verification flow (register → email → /verify-email)
|
||||
[ ] Password reset flow
|
||||
|
||||
Phase 7 — Dashboard & Analytics
|
||||
[ ] GET /api/v1/files (list with pagination)
|
||||
[ ] DELETE /api/v1/files/:id
|
||||
[ ] POST /api/v1/events (record view/download)
|
||||
[ ] GET /api/v1/dashboard/stats
|
||||
[ ] Frontend: wire DashboardPage to real data
|
||||
|
||||
Phase 8 — Payments
|
||||
[ ] Stripe account + products (Pro, Team)
|
||||
[ ] POST /api/v1/billing/checkout + portal
|
||||
[ ] POST /api/v1/billing/webhook
|
||||
[ ] Frontend: /pricing Buy buttons → Stripe Checkout
|
||||
|
||||
Phase 9 — Production hardening
|
||||
[ ] Sentry integration (backend + frontend)
|
||||
[ ] Pino structured logging
|
||||
[ ] PM2 cluster mode
|
||||
[ ] Nginx tuned for large file uploads
|
||||
[ ] UFW + Fail2ban
|
||||
[ ] Let's Encrypt TLS
|
||||
[ ] Cloudflare proxying for cshare.app domain
|
||||
[ ] Uptime monitoring (UptimeRobot or Better Uptime)
|
||||
[ ] Weekly DB backups to a separate R2 bucket (pg_dump → upload)
|
||||
[ ] Runbook: how to roll back a migration, restart workers, rotate secrets
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 14. Estimated Effort
|
||||
|
||||
| Phase | Complexity | Notes |
|
||||
|---|---|---|
|
||||
| Data Layer | Low | Schema is defined above — mostly setup |
|
||||
| API Core | Medium | Express boilerplate + multipart upload |
|
||||
| Storage | Low | S3 SDK is straightforward |
|
||||
| Auth | High | Many edge cases: token refresh, blocklist, verification |
|
||||
| Rate Limiting | Low | `rate-limiter-flexible` handles the hard parts |
|
||||
| Workers | Medium | Expiry is simple; BullMQ setup takes time |
|
||||
| Dashboard | Medium | Pagination + analytics aggregation |
|
||||
| Payments | High | Stripe webhook reliability is tricky |
|
||||
| Hardening | Medium | Mostly config, but easy to get wrong |
|
||||
|
||||
Total realistic estimate for a solo developer: **6–10 weeks** for Phases 1–8 at a sustainable pace.
|
||||
Reference in New Issue
Block a user