diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..462329c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,125 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +A single-file Telegram bot (`bot.py`) that receives files (and URLs) and uploads them to Google Drive. Uses **Pyrogram** (MTProto, v2+) for direct Telegram communication and **rclone** (with `gdrive:` remote) for Drive uploads. All async, all in one file. + +## Environment setup + +```bash +source venv/bin/activate +pip install -r requirements.txt +``` + +Required `.env` variables: `BOT_TOKEN`, `API_ID`, `API_HASH`. The rclone remote (`gdrive:`) must be authenticated separately via `rclone config`. + +## Running the bot + +**Production: runs as a systemd service** (`/etc/systemd/system/tg-gdrive-bot.service`, enabled on boot, `Restart=always` replaces the old `run_bot.sh` watchdog). Manage it with: + +```bash +systemctl status tg-gdrive-bot # state +systemctl restart tg-gdrive-bot # restart (or use the in-bot /restart) +systemctl stop tg-gdrive-bot # stop (do this before any manual run, or the session conflicts) +journalctl -u tg-gdrive-bot -f # follow logs (also appended to bot.log for the in-bot /logs) +``` + +The unit uses the venv interpreter (`venv/bin/python -u bot.py`) and a PATH that includes `/usr/local/bin` (for `deno`). stdout/stderr append to `bot.log` so the in-bot `/logs` command keeps working. `run_bot.sh` is the legacy nohup watchdog — superseded by systemd; don't run both. + +Manual run (debugging only — stop the service first): + +```bash +systemctl stop tg-gdrive-bot +source venv/bin/activate +python bot.py +``` + +No real tests exist beyond the stub `test_local_api.py`. + +## Bot commands + +| Command | Auth required | Description | +|---------|---------------|-------------| +| `/start` | No | Upload instructions + system info | +| `/help` | No | Full command reference | +| `/list` | Yes | Last 10 files in Drive | +| `/listall` | Yes | All files (paginated) | +| `/search ` | Yes | Search files by name | +| `/share ` | Yes | Make file public + share link | +| `/status` | Yes | Uptime, RAM, CPU, disk | +| `/logs` | Yes | Last 50 log lines | +| `/adduser @user` | Yes | Grant full access | +| `/removeuser @user` | Yes | Revoke full access | +| `/setquota @user ` | Yes | Set daily upload quota | +| `/clearquota @user` | Yes | Remove quota | +| `/quotas` | Yes | Overview all quotas | +| `/myquota` | No | Check own quota | +| `/restart` | Yes | Graceful restart via `os.execv` | + +## Architecture + +All logic in `bot.py` (~1368 lines). Key components: + +- **`handle_file`** (`@app.on_message(filters.media)`) — receives any media type (document, photo, video, audio, voice, sticker, video_note, animation). Routes to one of two upload strategies based on file size: + - **`_download_then_upload`** (< 2 GB) — downloads to temp dir via `client.download_media` with retry logic (3 attempts, 5/15/30s backoff) and live progress updates, then `rclone copy` to Drive. Cleans up temp files on success. + - **`_stream_to_drive`** (≥ 2 GB) — streams directly through RAM via `client.stream_media` → `rclone rcat` stdin, with retry logic and speed tracking. Zero disk usage. +- **`handle_url`** (`@app.on_message(filters.text)`) — detects URLs in text messages, downloads via `rclone copyurl` (RAM-only, no disk), saves to Drive. Pulse animation during transfer. +- **`classify_to_folder`** — routes files into Drive subfolders by extension/MIME (Videos, Music, Photos, Software, Archives, Models, Docs, etc.). +- **`resolve_filename`** — uses caption as filename override if provided; falls back to `file_name` attribute or generates one with appropriate extension. +- **`send_links`** — after upload, queries Drive via `rclone lsjson` for the file ID, then sends direct-download + wget links. + +### Auth system + +- **`AUTHORIZED_USERS`** set persisted in `authorized_users.json`. Anyone in this set has unlimited upload access. +- **Quota system** in `quotas.json` schema: `{"quotas": {"username": bytes}, "usage": {"username": {"YYYY-MM-DD": bytes}}}`. Non-auth users can upload if they have a quota with remaining bytes today. Quotas auto-reset at midnight UTC. + +### Upload methods + +| Size | Method | Disk usage | CLI tool | +|------|--------|------------|----------| +| < 2 GB | Download then upload | Temp file on disk | `rclone copy` | +| ≥ 2 GB | RAM stream | None | `rclone rcat` | +| URLs (any size) | RAM stream | None | `rclone copyurl` | +| Media-site URLs | yt-dlp to temp dir, then upload | Temp file on disk | `yt-dlp` + `rclone copy` | + +The 2 GB threshold is `STREAM_THRESHOLD` at line 53. + +### yt-dlp media downloads + +URLs whose domain matches `YTDLP_DOMAINS` (YouTube, Instagram, TikTok, Twitter/X, Facebook, Reddit, Pinterest, Tumblr, LinkedIn, Aparat) or `NSFW_DOMAINS` (PornHub, XVideos, xHamster, YouPorn, RedTube, SpankBang) are routed by `_process_url` to `_ytdlp_download` instead of `rclone copyurl`. Files are downloaded with yt-dlp (venv package; ffmpeg merges streams) into a temp dir, then uploaded to `gdrive:MEDIA-DLS//` — category from `classify_to_folder` (Videos, Music, …), NSFW domains always go to `MEDIA-DLS/NSFW`. Adding the word `mp3` or `audio` after the link extracts audio (`-x --audio-format mp3`). Playlists are capped at 50 items; runtime capped at 3 h (livestreams). If `cookies.txt` exists in the bot dir it is passed via `--cookies` — required for Instagram and for YouTube (this server's IP is bot-flagged by YouTube). `curl_cffi` is installed for browser impersonation (needed by e.g. SpankBang). Keep yt-dlp updated (`venv/bin/pip install -U yt-dlp`) — extractors break when sites change. + +### Inline file-action buttons + +Every `send_links` message carries an inline keyboard (🌍 Share · ℹ️ Info · 📂 Move · 🗑 Delete with confirm step · 📥 Send to Telegram). Buttons reference an 8-hex-char token stored in `FILE_ACTIONS` (persisted to `file_actions.json`, capped at 500 entries, survives restarts). `handle_buttons` (`@app.on_callback_query()`) dispatches on the `action:token[:dest]` callback-data format and runs rclone (`link`, `moveto`, `deletefile`, `lsjson`). Authorized users can act on any file; quota users only on files they uploaded (`owner` field). + +### Torrents (aria2c) + +Magnet links (detected in `handle_url` via `MAGNET_RE`) and `.torrent` files (detected in `handle_file` by extension/mime) route to `process_torrent`. A minimal bencode decoder (`_bdecode` / `torrent_files`) reads the file list **without** sanitizing names (they must match what aria2 writes to disk and are used verbatim as Drive paths). Magnets first fetch metadata only (`_aria2_fetch_metadata`). Download strategy by total size: **< 8 GB** downloads the whole torrent then uploads each file; **≥ 8 GB** (`TORRENT_STREAM_THRESHOLD`) uses per-file mode (`--select-file=N`, download→upload→delete per file) to keep peak disk low. There is no RAM-stream path — BitTorrent is out-of-order, so torrents always hit disk. Work dir is `dl_tmp/` on `/dev/sda1` (NOT `/tmp`, which is RAM-backed tmpfs, only 3.9 GB). Pre-checks: free disk (`shutil.disk_usage`, needs biggest-file or total + 2 GB headroom) and quota. Files upload to `gdrive:Torrents//…` preserving structure. `aria2c` and the seeding-disabled flags live in `ARIA2_COMMON`; progress parsed by `ARIA2_PROGRESS_RE`. + +### Telegram delivery (Drive → Telegram) + +`/get ` and the 📥 button both call `deliver_to_telegram`: look up the file (`get_drive_entry` / `_find_drive_file`), refuse if over `TG_SEND_LIMIT` (~2 GB Telegram bot cap, offers `/share` instead), rclone-copy to `dl_tmp/` with progress (parsed from `--stats-one-line`), then `send_video`/`send_audio`/`send_document` by extension with an upload-progress callback. This is the only path that sends files *out* of Drive. + +### yt-dlp auto-updater + +`ytdlp_update_loop` (started in `_main()` alongside `app.start()`/`idle()`) runs `pip install -U yt-dlp` in the venv every 24 h and logs version changes. Subprocesses pick up the new version immediately. + +Spotify links (`SPOTIFY_DOMAINS`) are routed to `_spotify_download`, which runs **spotdl** (Spotify metadata → audio downloaded from YouTube Music) and uploads results to `MEDIA-DLS/Music`. Because the audio comes from YouTube, this also requires `cookies.txt` on this server. Deno is installed at `/root/.config/spotdl/deno` (symlinked to `/usr/local/bin/deno`) — yt-dlp needs it for YouTube JS challenges. Both yt-dlp and spotdl uploads share `_upload_media_files`. + +### Key tech stack + +- **Pyrogram** v2+ — async MTProto client (`Client`), `filters.media` / `filters.command()` handlers +- **rclone** — Drive operations: `copy`, `rcat`, `copyurl`, `lsjson`, `link` +- **`asyncio.create_subprocess_exec`** — all rclone calls are non-blocking subprocesses +- **`psutil`** — RAM/CPU/disk stats for `/status` +- **`python-dotenv`** — loads `.env` + +### Key constraints + +- All handlers are `async def` — Pyrogram is fully async. +- rclone subprocesses use `asyncio.create_subprocess_exec` — do not replace with `subprocess.run`. +- Temp files only deleted on successful upload (manual retry on failure). +- Filenames cleaned via `sanitize_filename` (NFKC normalization, emoji removal, illegal char stripping). +- The bot can also handle files sent by non-auth users if they have a daily quota assigned. \ No newline at end of file