v0.13.1

README.md

@@ -16,11 +16,6 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
 
 
-## breaking changes \o/
-
-this is the readme for v0.12 which has a different expression for volume permissions (`-v`); see [the v0.11.x readme](https://github.com/9001/copyparty/tree/15b59822112dda56cee576df30f331252fc62628#readme) for stuff regarding the [current stable release](https://github.com/9001/copyparty/releases/tag/v0.11.47)
-
-
 ## readme toc
 
 * top
@@ -41,11 +36,13 @@ this is the readme for v0.12 which has a different expression for volume permiss
     * [uploading](#uploading)
         * [file-search](#file-search)
     * [file manager](#file-manager)
+    * [batch rename](#batch-rename)
     * [markdown viewer](#markdown-viewer)
     * [other tricks](#other-tricks)
-* [searching](#searching)
-    * [search configuration](#search-configuration)
-    * [database location](#database-location)
+    * [searching](#searching)
+* [server config](#server-config)
+    * [upload rules](#upload-rules)
+    * [database location](#database-location)upload rules
     * [metadata from audio files](#metadata-from-audio-files)
     * [file parser plugins](#file-parser-plugins)
     * [complete examples](#complete-examples)
@@ -125,30 +122,31 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ [accounts](#accounts-and-volumes)
 * upload
   * ☑ basic: plain multipart, ie6 support
-  * ☑ up2k: js, resumable, multithreaded
+  * ☑ [up2k](#uploading): js, resumable, multithreaded
   * ☑ stash: simple PUT filedropper
   * ☑ unpost: undo/delete accidental uploads
   * ☑ symlink/discard existing files (content-matching)
 * download
   * ☑ single files in browser
-  * ☑ folders as zip / tar files
+  * ☑ [folders as zip / tar files](#zip-downloads)
   * ☑ FUSE client (read-only)
 * browser
   * ☑ navpane (directory tree sidebar)
+  * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
   * ☑ audio player (with OS media controls)
-  * ☑ thumbnails
+  * ☑ image gallery with webm player
+  * ☑ [thumbnails](#thumbnails)
     * ☑ ...of images using Pillow
     * ☑ ...of videos using FFmpeg
     * ☑ cache eviction (max-age; maybe max-size eventually)
-  * ☑ image gallery with webm player
   * ☑ SPA (browse while uploading)
     * if you use the navpane to navigate, not folders in the file list
 * server indexing
-  * ☑ locate files by contents
+  * ☑ [locate files by contents](#file-search)
   * ☑ search by name/path/date/size
-  * ☑ search by ID3-tags etc.
+  * ☑ [search by ID3-tags etc.](#searching)
 * markdown
-  * ☑ viewer
+  * ☑ [viewer](#markdown-viewer)
   * ☑ editor (sure why not)
 
 
@@ -181,7 +179,7 @@ small collection of user feedback
   * this is an msys2 bug, the regular windows edition of python is fine
 
 * VirtualBox: sqlite throws `Disk I/O Error` when running in a VM and the up2k database is in a vboxsf
-  * use `--hist` or the `hist` volflag (`-v [...]:chist=/tmp/foo`) to place the db inside the vm instead
+  * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db inside the vm instead
 
 
 # accounts and volumes
@@ -373,6 +371,53 @@ if you have the required permissions, you can cut/paste, rename, and delete file
 you can move files across browser tabs (cut in one tab, paste in another)
 
 
+## batch rename
+
+![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
+
+select some files and press F2 to bring up the rename UI
+
+quick explanation of the buttons,  
+* `[✅ apply rename]` confirms and begins renaming
+* `[❌ cancel]` aborts and closes the rename window
+* `[↺ reset]` reverts any filename changes back to the original name
+* `[decode]` does a URL-decode on the filename, fixing stuff like `&` and `%20`
+* `[advanced]` toggles advanced mode
+
+advanced mode: rename files based on rules to decide the new names, based on the original name (regex), or based on the tags collected from the file (artist/title/...), or a mix of both
+
+in advanced mode,  
+* `[case]` toggles case-sensitive regex
+* `regex` is the regex pattern to apply to the original filename; any files which don't match will be skipped
+* `format` is the new filename, taking values from regex capturing groups and/or from file tags
+  * very loosely based on foobar2000 syntax
+* `presets` lets you save rename rules for later
+
+available functions:
+* `$lpad(text, length, pad_char)`
+* `$rpad(text, length, pad_char)`
+
+so,
+
+say you have a file named [`meganeko - Eclipse - 07 Sirius A.mp3`](https://www.youtube.com/watch?v=-dtb0vDPruI) (absolutely fantastic album btw) and the tags are: `Album:Eclipse`, `Artist:meganeko`, `Title:Sirius A`, `tn:7`
+
+you could use just regex to rename it:
+* `regex` = `(.*) - (.*) - ([0-9]{2}) (.*)`
+* `format` = `(3). (1) - (4)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+or you could use just tags:
+* `format` = `$lpad((tn),2,0). (artist) - (title).(ext)`
+* `output` = `7. meganeko - Sirius A.mp3`
+
+or a mix of both:
+* `regex` = ` - ([0-9]{2}) `
+* `format` = `(1). (artist) - (title).(ext)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
+
+
 ## markdown viewer
 
 ![copyparty-md-read-fs8](https://user-images.githubusercontent.com/241032/115978057-66419080-a57d-11eb-8539-d2be843991aa.png)
@@ -387,7 +432,7 @@ you can move files across browser tabs (cut in one tab, paste in another)
 * if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
 
 
-# searching
+## searching
 
 ![copyparty-search-fs8](https://user-images.githubusercontent.com/241032/115978060-6772bd80-a57d-11eb-81d3-174e869b72c3.png)
 
@@ -399,12 +444,12 @@ path/name queries are space-separated, AND'ed together, and words are negated wi
 * path: `shibayan -bossa` finds all files where one of the folders contain `shibayan` but filters out any results where `bossa` exists somewhere in the path
 * name: `demetori styx` gives you [good stuff](https://www.youtube.com/watch?v=zGh0g14ZJ8I&list=PL3A147BD151EE5218&index=9)
 
-add `-e2ts` to also scan/index tags from music files:
+add the argument `-e2ts` to also scan/index tags from music files, which brings us over to:
 
 
-## search configuration
+# server config
 
-searching relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
+file indexing relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
 
 through arguments:
 * `-e2d` enables file indexing on upload
@@ -415,20 +460,60 @@ through arguments:
 * `-e2tsr` deletes all existing tags, does a full reindex
 
 the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
-* `-v ~/music::r:ce2dsa:ce2tsr` does a full reindex of everything on startup
-* `-v ~/music::r:cd2d` disables **all** indexing, even if any `-e2*` are on
-* `-v ~/music::r:cd2t` disables all `-e2t*` (tags), does not affect `-e2d*`
+* `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
+* `-v ~/music::r:c,d2d` disables **all** indexing, even if any `-e2*` are on
+* `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
 
 note:
 * `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
 * the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
 
-you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `cdhash`, this has the following consequences:
+you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
 * initial indexing is way faster, especially when the volume is on a network disk
 * makes it impossible to [file-search](#file-search)
 * if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
 
-if you set `--no-hash`, you can enable hashing for specific volumes using flag `cehash`
+if you set `--no-hash`, you can enable hashing for specific volumes using flag `:c,ehash`
+
+
+## upload rules
+
+you can set upload rules using volume flags, some examples:
+
+* `:c,sz=1k-3m` sets allowed filesize between 1 KiB and 3 MiB inclusive (suffixes: b, k, m, g)
+* `:c,nosub` disallow uploading into subdirectories; goes well with `rotn` and `rotf`:
+* `:c,rotn=1000,2` moves uploads into subfolders, up to 1000 files in each folder before making a new one, two levels deep (must be at least 1)
+* `:c,rotf=%Y/%m/%d/%H` enforces files to be uploaded into a structure of subfolders according to that date format
+  * if someone uploads to `/foo/bar` the path would be rewritten to `/foo/bar/2021/08/06/23` for example
+  * but the actual value is not verified, just the structure, so the uploader can choose any values which conform to the format string
+    * just to avoid additional complexity in up2k which is enough of a mess already
+* `:c,lifetime=300` delete uploaded files when they become 5 minutes old
+
+you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be off, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to
+
+* `:c,maxn=250,3600` allows 250 files over 1 hour from each IP (tracked per-volume)
+* `:c,maxb=1g,300` allows 1 GiB total over 5 minutes from each IP (tracked per-volume)
+
+
+## compress uploads
+
+files can be autocompressed on upload, either on user-request (if config allows) or forced by server-config
+
+* volume flag `gz` allows gz compression
+* volume flag `xz` allows lzma compression
+* volume flag `pk` **forces** compression on all files
+* url parameter `pk` requests compression with server-default algorithm
+* url parameter `gz` or `xz` requests compression with a specific algorithm
+* url parameter `xz` requests xz compression
+
+things to note,
+* the `gz` and `xz` arguments take a single optional argument, the compression level (range 0 to 9)
+* the `pk` volume flag takes the optional argument `ALGORITHM,LEVEL` which will then be forced for all uploads, for example `gz,9` or `xz,0`
+* default compression is gzip level 9
+* all upload methods except up2k are supported
+* the files will be indexed after compression, so dupe-detection and file-search will not work as expected
+
+some examples,
 
 
 ## database location
@@ -436,7 +521,7 @@ if you set `--no-hash`, you can enable hashing for specific volumes using flag `
 copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
 
 this can instead be kept in a single place using the `--hist` argument, or the `hist=` volume flag, or a mix of both:
-* `--hist ~/.cache/copyparty -v ~/music::r:chist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
+* `--hist ~/.cache/copyparty -v ~/music::r:c,hist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
 
 note:
 * markdown edits are always stored in a local `.hist` subdirectory
@@ -447,10 +532,12 @@ note:
 ## metadata from audio files
 
 `-mte` decides which tags to index and display in the browser (and also the display order), this can be changed per-volume:
-* `-v ~/music::r:cmte=title,artist` indexes and displays *title* followed by *artist*
+* `-v ~/music::r:c,mte=title,artist` indexes and displays *title* followed by *artist*
 
 if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
 
+but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
+
 `-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
 
 tags that start with a `.` such as `.bpm` and `.dur`(ation) indicate numeric value
@@ -471,7 +558,7 @@ copyparty can invoke external programs to collect additional metadata for files
 
 * `-mtp .bpm=~/bin/audio-bpm.py` will execute `~/bin/audio-bpm.py` with the audio file as argument 1 to provide the `.bpm` tag, if that does not exist in the audio metadata
 * `-mtp key=f,t5,~/bin/audio-key.py` uses `~/bin/audio-key.py` to get the `key` tag, replacing any existing metadata tag (`f,`), aborting if it takes longer than 5sec (`t5,`)
-* `-v ~/music::r:cmtp=.bpm=~/bin/audio-bpm.py:cmtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
+* `-v ~/music::r:c,mtp=.bpm=~/bin/audio-bpm.py:c,mtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
 
 *but wait, there's more!* `-mtp` can be used for non-audio files as well using the `a` flag: `ay` only do audio files, `an` only do non-audio files, or `ad` do all files (d as in dontcare) 
 
@@ -535,11 +622,11 @@ quick summary of more eccentric web-browsers trying to view a directory index:
   * `var xhr = new XMLHttpRequest(); xhr.open('POST', 'https://127.0.0.1:3923/msgs?raw'); xhr.send('foo');`
 
 * curl/wget: upload some files (post=file, chunk=stdin)
-  * `post(){ curl -b cppwd=wark http://127.0.0.1:3923/ -F act=bput -F f=@"$1";}`  
+  * `post(){ curl -b cppwd=wark -F act=bput -F f=@"$1" http://127.0.0.1:3923/;}`  
     `post movie.mkv`
-  * `post(){ wget --header='Cookie: cppwd=wark' http://127.0.0.1:3923/?raw --post-file="$1" -O-;}`  
+  * `post(){ wget --header='Cookie: cppwd=wark' --post-file="$1" -O- http://127.0.0.1:3923/?raw;}`  
     `post movie.mkv`
-  * `chunk(){ curl -b cppwd=wark http://127.0.0.1:3923/ -T-;}`  
+  * `chunk(){ curl -b cppwd=wark -T- http://127.0.0.1:3923/;}`  
     `chunk <movie.mkv`
 
 * FUSE: mount a copyparty server as a local filesystem

bin/README.md

@@ -61,3 +61,8 @@ cd /mnt/nas/music/.hist
 ~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy key
 ~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy .bpm -vac
 ```
+
+
+# [`prisonparty.sh`](prisonparty.sh)
+* run copyparty in a chroot, preventing any accidental file access
+* creates bindmounts for /bin, /lib, and so on, see `sysdirs=`

bin/mtag/README.md

@@ -4,6 +4,7 @@ some of these rely on libraries which are not MIT-compatible
 
 * [audio-bpm.py](./audio-bpm.py) detects the BPM of music using the BeatRoot Vamp Plugin; imports GPL2
 * [audio-key.py](./audio-key.py) detects the melodic key of music using the Mixxx fork of keyfinder; imports GPL3
+* [media-hash.py](./media-hash.py) generates checksums for audio and video streams; uses FFmpeg (LGPL or GPL)
 
 
 # dependencies
@@ -18,7 +19,10 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 
 # usage from copyparty
 
-`copyparty -e2dsa -e2ts -mtp key=f,audio-key.py -mtp .bpm=f,audio-bpm.py`
+`copyparty -e2dsa -e2ts` followed by any combination of these:
+* `-mtp key=f,audio-key.py`
+* `-mtp .bpm=f,audio-bpm.py`
+* `-mtp ahash,vhash=f,media-hash.py`
 
 * `f,` makes the detected value replace any existing values
 * the `.` in `.bpm` indicates numeric value
@@ -29,6 +33,9 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 ## usage with volume-flags
 
 instead of affecting all volumes, you can set the options for just one volume like so:
-```
-copyparty -v /mnt/nas/music:/music:r:cmtp=key=f,audio-key.py:cmtp=.bpm=f,audio-bpm.py:ce2dsa:ce2ts
-```
+
+`copyparty -v /mnt/nas/music:/music:r:c,e2dsa:c,e2ts` immediately followed by any combination of these:
+
+* `:c,mtp=key=f,audio-key.py`
+* `:c,mtp=.bpm=f,audio-bpm.py`
+* `:c,mtp=ahash,vhash=f,media-hash.py`

bin/mtag/media-hash.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python
+
+import re
+import sys
+import json
+import time
+import base64
+import hashlib
+import subprocess as sp
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p
+
+
+"""
+dep: ffmpeg
+"""
+
+
+def det():
+    # fmt: off
+    cmd = [
+        "ffmpeg",
+        "-nostdin",
+        "-hide_banner",
+        "-v", "fatal",
+        "-i", fsenc(sys.argv[1]),
+        "-f", "framemd5",
+        "-"
+    ]
+    # fmt: on
+
+    p = sp.Popen(cmd, stdout=sp.PIPE)
+    # ps = io.TextIOWrapper(p.stdout, encoding="utf-8")
+    ps = p.stdout
+
+    chans = {}
+    for ln in ps:
+        if ln.startswith(b"#stream#"):
+            break
+
+        m = re.match(r"^#media_type ([0-9]): ([a-zA-Z])", ln.decode("utf-8"))
+        if m:
+            chans[m.group(1)] = m.group(2)
+
+    hashers = [hashlib.sha512(), hashlib.sha512()]
+    for ln in ps:
+        n = int(ln[:1])
+        v = ln.rsplit(b",", 1)[-1].strip()
+        hashers[n].update(v)
+
+    r = {}
+    for k, v in chans.items():
+        dg = hashers[int(k)].digest()[:12]
+        dg = base64.urlsafe_b64encode(dg).decode("ascii")
+        r[v[0].lower() + "hash"] = dg
+
+    print(json.dumps(r, indent=4))
+
+
+def main():
+    try:
+        det()
+    except:
+        pass  # mute
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/res/yt-ipr.conf

@@ -0,0 +1,39 @@
+# example config file to use copyparty as a youtube manifest collector,
+# use with copyparty like:  python copyparty.py -c yt-ipr.conf
+#
+# see docs/example.conf for a better explanation of the syntax, but
+# newlines are block separators, so adding blank lines inside a volume definition is bad
+# (use comments as separators instead)
+
+
+# create user ed, password wark
+u ed:wark
+
+
+# create a volume at /ytm which stores files at ./srv/ytm
+./srv/ytm
+/ytm
+# write-only, but read-write for user ed
+w
+rw ed
+# rescan the volume on startup
+c e2dsa
+# collect tags from all new files since last scan
+c e2ts
+# optionally enable compression to make the files 50% smaller
+c pk
+# only allow uploads which are between 16k and 1m large
+c sz=16k-1m
+# allow up to 10 uploads over 5 minutes from each ip
+c maxn=10,300
+# move uploads into subfolders: YEAR-MONTH / DAY-HOUR / <upload>
+c rotf=%Y-%m/%d-%H
+# delete uploads when they are 24 hours old
+c lifetime=86400
+# add the parser and tell copyparty what tags it can expect from it
+c mtp=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires=bin/mtag/yt-ipr.py
+# decide which tags we want to index and in what order
+c mte=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires
+
+
+# create any other volumes you'd like down here, or merge this with an existing config file

bin/mtag/res/yt-ipr.user.js

@@ -0,0 +1,48 @@
+// ==UserScript==
+// @name    youtube-playerdata-hub
+// @match   https://youtube.com/*
+// @match   https://*.youtube.com/*
+// @version 1.0
+// @grant   GM_addStyle
+// ==/UserScript==
+
+function main() {
+    var server = 'https://127.0.0.1:3923/ytm?pw=wark',
+        interval = 60; // sec
+
+    var sent = {};
+    function send(txt, mf_url, desc) {
+        if (sent[mf_url])
+            return;
+
+        fetch(server + '&_=' + Date.now(), { method: "PUT", body: txt });
+        console.log('[yt-pdh] yeet %d bytes, %s', txt.length, desc);
+        sent[mf_url] = 1;
+    }
+
+    function collect() {
+        setTimeout(collect, interval * 1000);
+        try {
+            var pd = document.querySelector('ytd-watch-flexy');
+            if (!pd)
+                return console.log('[yt-pdh] no video found');
+
+            pd = pd.playerData;
+            var mu = pd.streamingData.dashManifestUrl || pd.streamingData.hlsManifestUrl;
+            if (!mu || !mu.length)
+                return console.log('[yt-pdh] no manifest found');
+
+            var desc = pd.videoDetails.videoId + ', ' + pd.videoDetails.title;
+            send(JSON.stringify(pd), mu, desc);
+        }
+        catch (ex) {
+            console.log("[yt-pdh]", ex);
+        }
+    }
+    collect();
+}
+
+var scr = document.createElement('script');
+scr.textContent = '(' + main.toString() + ')();';
+(document.head || document.getElementsByTagName('head')[0]).appendChild(scr);
+console.log('[yt-pdh] a');

bin/mtag/yt-ipr.py

@@ -0,0 +1,198 @@
+#!/usr/bin/env python
+
+import re
+import os
+import sys
+import gzip
+import json
+import base64
+import string
+import urllib.request
+from datetime import datetime
+
+"""
+youtube initial player response
+
+it's probably best to use this through a config file; see res/yt-ipr.conf
+
+but if you want to use plain arguments instead then:
+  -v srv/ytm:ytm:w:rw,ed
+       :c,e2ts:c,e2dsa
+       :c,sz=16k-1m:c,maxn=10,300:c,rotf=%Y-%m/%d-%H
+       :c,mtp=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires=bin/mtag/yt-ipr.py
+       :c,mte=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires
+
+see res/yt-ipr.user.js for the example userscript to go with this
+"""
+
+
+def main():
+    try:
+        with gzip.open(sys.argv[1], "rt", encoding="utf-8", errors="replace") as f:
+            txt = f.read()
+    except:
+        with open(sys.argv[1], "r", encoding="utf-8", errors="replace") as f:
+            txt = f.read()
+
+    txt = "{" + txt.split("{", 1)[1]
+
+    try:
+        pd = json.loads(txt)
+    except json.decoder.JSONDecodeError as ex:
+        pd = json.loads(txt[: ex.pos])
+
+    # print(json.dumps(pd, indent=2))
+
+    if "videoDetails" in pd:
+        parse_youtube(pd)
+    else:
+        parse_freg(pd)
+
+
+def get_expiration(url):
+    et = re.search(r"[?&]expire=([0-9]+)", url).group(1)
+    et = datetime.utcfromtimestamp(int(et))
+    return et.strftime("%Y-%m-%d, %H:%M")
+
+
+def parse_youtube(pd):
+    vd = pd["videoDetails"]
+    sd = pd["streamingData"]
+
+    et = sd["adaptiveFormats"][0]["url"]
+    et = get_expiration(et)
+
+    mf = []
+    if "dashManifestUrl" in sd:
+        mf.append("dash")
+    if "hlsManifestUrl" in sd:
+        mf.append("hls")
+
+    r = {
+        "yt-id": vd["videoId"],
+        "yt-title": vd["title"],
+        "yt-author": vd["author"],
+        "yt-channel": vd["channelId"],
+        "yt-views": vd["viewCount"],
+        "yt-private": vd["isPrivate"],
+        # "yt-expires": sd["expiresInSeconds"],
+        "yt-manifest": ",".join(mf),
+        "yt-expires": et,
+    }
+    print(json.dumps(r))
+
+    freg_conv(pd)
+
+
+def parse_freg(pd):
+    md = pd["metadata"]
+    r = {
+        "yt-id": md["id"],
+        "yt-title": md["title"],
+        "yt-author": md["channelName"],
+        "yt-channel": md["channelURL"].strip("/").split("/")[-1],
+        "yt-expires": get_expiration(list(pd["video"].values())[0]),
+    }
+    print(json.dumps(r))
+
+
+def freg_conv(pd):
+    # based on getURLs.js v1.5 (2021-08-07)
+    # fmt: off
+    priority = {
+        "video": [
+            337, 315, 266, 138,  # 2160p60
+            313, 336,  # 2160p
+            308,  # 1440p60
+            271, 264,  # 1440p
+            335, 303, 299,  # 1080p60
+            248, 169, 137,  # 1080p
+            334, 302, 298,  # 720p60
+            247, 136  # 720p
+        ],
+        "audio": [
+            251, 141, 171, 140, 250, 249, 139
+        ]
+    }
+
+    vid_id = pd["videoDetails"]["videoId"]
+    chan_id = pd["videoDetails"]["channelId"]
+
+    try:
+        thumb_url = pd["microformat"]["playerMicroformatRenderer"]["thumbnail"]["thumbnails"][0]["url"]
+        start_ts = pd["microformat"]["playerMicroformatRenderer"]["liveBroadcastDetails"]["startTimestamp"]
+    except:
+        thumb_url = f"https://img.youtube.com/vi/{vid_id}/maxresdefault.jpg"
+        start_ts = ""
+
+    # fmt: on
+
+    metadata = {
+        "title": pd["videoDetails"]["title"],
+        "id": vid_id,
+        "channelName": pd["videoDetails"]["author"],
+        "channelURL": "https://www.youtube.com/channel/" + chan_id,
+        "description": pd["videoDetails"]["shortDescription"],
+        "thumbnailUrl": thumb_url,
+        "startTimestamp": start_ts,
+    }
+
+    if [x for x in vid_id if x not in string.ascii_letters + string.digits + "_-"]:
+        print(f"malicious json", file=sys.stderr)
+        return
+
+    basepath = os.path.dirname(sys.argv[1])
+
+    thumb_fn = f"{basepath}/{vid_id}.jpg"
+    tmp_fn = f"{thumb_fn}.{os.getpid()}"
+    if not os.path.exists(thumb_fn) and (
+        thumb_url.startswith("https://img.youtube.com/vi/")
+        or thumb_url.startswith("https://i.ytimg.com/vi/")
+    ):
+        try:
+            with urllib.request.urlopen(thumb_url) as fi:
+                with open(tmp_fn, "wb") as fo:
+                    fo.write(fi.read())
+
+            os.rename(tmp_fn, thumb_fn)
+        except:
+            if os.path.exists(tmp_fn):
+                os.unlink(tmp_fn)
+
+    try:
+        with open(thumb_fn, "rb") as f:
+            thumb = base64.b64encode(f.read()).decode("ascii")
+    except:
+        thumb = "/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAMCAgICAgMCAgIDAwMDBAYEBAQEBAgGBgUGCQgKCgkICQkKDA8MCgsOCwkJDRENDg8QEBEQCgwSExIQEw8QEBD/yQALCAABAAEBAREA/8wABgAQEAX/2gAIAQEAAD8A0s8g/9k="
+
+    metadata["thumbnail"] = "data:image/jpeg;base64," + thumb
+
+    ret = {
+        "metadata": metadata,
+        "version": "1.5",
+        "createTime": datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ"),
+    }
+
+    for stream, itags in priority.items():
+        for itag in itags:
+            url = None
+            for afmt in pd["streamingData"]["adaptiveFormats"]:
+                if itag == afmt["itag"]:
+                    url = afmt["url"]
+                    break
+
+            if url:
+                ret[stream] = {itag: url}
+                break
+
+    fn = f"{basepath}/{vid_id}.urls.json"
+    with open(fn, "w", encoding="utf-8", errors="replace") as f:
+        f.write(json.dumps(ret, indent=4))
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except:
+        # raise
+        pass

bin/prisonparty.sh

@@ -0,0 +1,94 @@
+#!/bin/bash
+set -e
+
+# runs copyparty (or any other python script really) in a chroot
+#
+# assumption: these directories, and everything within, are owned by root
+sysdirs=(bin lib lib32 lib64 sbin usr)
+
+
+# error-handler
+help() { cat <<'EOF'
+
+usage:
+  ./prisonparty.sh <ROOTDIR> <UID> <GID> [VOLDIR [VOLDIR...]] -- copyparty-sfx.py [...]"
+
+example:
+  ./prisonparty.sh /var/jail 1000 1000 /mnt/nas/music -- copyparty-sfx.py -v /mnt/nas/music::rwmd"
+
+EOF
+exit 1
+}
+
+
+# read arguments
+trap help EXIT
+jail="$1"; shift
+uid="$1"; shift
+gid="$1"; shift
+
+vols=()
+while true; do
+	v="$1"; shift
+	[ "$v" = -- ] && break  # end of volumes
+	[ "$#" -eq 0 ] && break  # invalid usage
+	vols+=("$v")
+done
+cpp="$1"; shift
+cpp="$(realpath "$cpp")"
+cppdir="$(dirname "$cpp")"
+trap - EXIT
+
+
+# debug/vis
+echo
+echo "chroot-dir = $jail"
+echo "user:group = $uid:$gid"
+echo " copyparty = $cpp"
+echo
+printf '\033[33m%s\033[0m\n' "copyparty can access these folders and all their subdirectories:"
+for v in "${vols[@]}"; do
+	printf '\033[36m ├─\033[0m %s \033[36m ── added by (You)\033[0m\n' "$v"
+done
+printf '\033[36m ├─\033[0m %s \033[36m ── where the copyparty binary is\033[0m\n' "$cppdir"
+printf '\033[36m ╰─\033[0m %s \033[36m ── the folder you are currently in\033[0m\n' "$PWD"
+vols+=("$cppdir" "$PWD")
+echo
+
+
+# resolve and remove trailing slash
+jail="$(realpath "$jail")"
+jail="${jail%/}"
+
+
+# bind-mount system directories and volumes
+printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | LC_ALL=C sort |
+while IFS= read -r v; do
+	[ -e "/$v" ] || {
+		# printf '\033[1;31mfolder does not exist:\033[0m %s\n' "$v"
+		continue
+	}
+	mkdir -p "$jail/$v"
+	mount | grep -qF " on $jail/$v " ||
+		mount --bind /$v "$jail/$v"
+done
+
+
+# create a tmp
+mkdir -p "$jail/tmp"
+chown -R "$uid:$gid" "$jail/tmp"
+
+
+# run copyparty
+/sbin/chroot --userspec=$uid:$gid "$jail" "$(which python3)" "$cpp" "$@" && rv=0 || rv=$?
+
+
+# cleanup if not in use
+lsof "$jail" | grep -qF "$jail" &&
+	echo "chroot is in use, will not cleanup" ||
+{
+	mount | grep -F " on $jail" |
+	awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' |
+	LC_ALL=C sort -r  | tr '\n' '\0' | xargs -r0 umount
+}
+exit $rv

contrib/systemd/copyparty.service

@@ -15,8 +15,11 @@
 #   https://user-images.githubusercontent.com/241032/126040249-cb535cc7-c599-4931-a796-a5d9af691bad.png
 #
 # enable line-buffering for realtime logging (slight performance cost):
-#   modify ExecStart and prefix it with `/bin/stdbuf -oL` like so:
-#   ExecStart=/bin/stdbuf -oL /usr/bin/python3 [...]
+#   modify ExecStart and prefix it with `/usr/bin/stdbuf -oL` like so:
+#   ExecStart=/usr/bin/stdbuf -oL /usr/bin/python3 [...]
+# but some systemd versions require this instead (higher performance cost):
+#   inside the [Service] block, add the following line:
+#   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
 Description=copyparty file server

copyparty/__init__.py

@@ -25,6 +25,28 @@ ANYWIN = WINDOWS or sys.platform in ["msys"]
 MACOS = platform.system() == "Darwin"
 
 
+def get_unix_home():
+    try:
+        v = os.environ["XDG_CONFIG_HOME"]
+        if not v:
+            raise Exception()
+        ret = os.path.normpath(v)
+        os.listdir(ret)
+        return ret
+    except:
+        pass
+
+    try:
+        v = os.path.expanduser("~/.config")
+        if v.startswith("~"):
+            raise Exception()
+        ret = os.path.normpath(v)
+        os.listdir(ret)
+        return ret
+    except:
+        return "/tmp"
+
+
 class EnvParams(object):
     def __init__(self):
         self.t0 = time.time()
@@ -37,10 +59,7 @@ class EnvParams(object):
         elif sys.platform == "darwin":
             self.cfg = os.path.expanduser("~/Library/Preferences/copyparty")
         else:
-            self.cfg = os.path.normpath(
-                os.getenv("XDG_CONFIG_HOME", os.path.expanduser("~/.config"))
-                + "/copyparty"
-            )
+            self.cfg = get_unix_home() + "/copyparty"
 
         self.cfg = self.cfg.replace("\\", "/")
         try:

copyparty/__main__.py

@@ -23,7 +23,8 @@ from textwrap import dedent
 from .__init__ import E, WINDOWS, VT100, PY2, unicode
 from .__version__ import S_VERSION, S_BUILD_DT, CODENAME
 from .svchub import SvcHub
-from .util import py_desc, align_tab, IMPLICATIONS
+from .util import py_desc, align_tab, IMPLICATIONS, ansi_re
+from .authsrv import re_vol
 
 HAVE_SSL = True
 try:
@@ -66,8 +67,12 @@ class Dodge11874(RiceFormatter):
 def lprint(*a, **ka):
     global printed
 
-    printed += " ".join(unicode(x) for x in a) + ka.get("end", "\n")
-    print(*a, **ka)
+    txt = " ".join(unicode(x) for x in a) + ka.get("end", "\n")
+    printed += txt
+    if not VT100:
+        txt = ansi_re.sub("", txt)
+
+    print(txt, **ka)
 
 
 def warn(msg):
@@ -196,12 +201,18 @@ def run_argparse(argv, formatter):
         formatter_class=formatter,
         prog="copyparty",
         description="http file sharing hub v{} ({})".format(S_VERSION, S_BUILD_DT),
-        epilog=dedent(
-            """
+    )
+
+    sects = [
+        [
+            "accounts",
+            "accounts and volumes",
+            dedent(
+                """
             -a takes username:password,
-            -v takes src:dst:perm1:perm2:permN:cflag1:cflag2:cflagN:...
+            -v takes src:dst:perm1:perm2:permN:volflag1:volflag2:volflagN:...
                where "perm" is "accesslevels,username1,username2,..."
-               and "cflag" is config flags to set on this volume
+               and "volflag" is config flags to set on this volume
             
             list of accesslevels:
               "r" (read):   list folder contents, download files
@@ -209,11 +220,7 @@ def run_argparse(argv, formatter):
               "m" (move):   move files and folders; need "w" at destination
               "d" (delete): permanently delete files and folders
 
-            list of cflags:
-              "c,nodupe" rejects existing files (instead of symlinking them)
-              "c,e2d" sets -e2d (all -e2* args can be set using ce2* cflags)
-              "c,d2t" disables metadata collection, overrides -e2t*
-              "c,d2d" disables all database stuff, overrides -e2*
+            too many volflags to list here, see the other sections
 
             example:\033[35m
               -a ed:hunter2 -v .::r:rw,ed -v ../inc:dump:w:rw,ed:c,nodupe  \033[36m
@@ -230,29 +237,86 @@ def run_argparse(argv, formatter):
 
             consider the config file for more flexible account/volume management,
             including dynamic reload at runtime (and being more readable w)
+            """
+            ),
+        ],
+        [
+            "flags",
+            "list of volflags",
+            dedent(
+                """
+            volflags are appended to volume definitions, for example,
+            to create a write-only volume with the \033[33mnodupe\033[0m and \033[32mnosub\033[0m flags:
+              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub
 
+            \033[0muploads, general:
+              \033[36mnodupe\033[35m rejects existing files (instead of symlinking them)
+              \033[36mnosub\033[35m forces all uploads into the top folder of the vfs
+              \033[36mgz\033[35m allows server-side gzip of uploads with ?gz (also c,xz)
+              \033[36mpk\033[35m forces server-side compression, optional arg: xz,9
+            
+            \033[0mupload rules:
+              \033[36mmaxn=250,600\033[35m max 250 uploads over 15min
+              \033[36mmaxb=1g,300\033[35m max 1 GiB over 5min (suffixes: b, k, m, g)
+              \033[36msz=1k-3m\033[35m allow filesizes between 1 KiB and 3MiB
+            
+            \033[0mupload rotation:
+            (moves all uploads into the specified folder structure)
+              \033[36mrotn=100,3\033[35m 3 levels of subfolders with 100 entries in each
+              \033[36mrotf=%Y-%m/%d-%H\033[35m date-formatted organizing
+              \033[36mlifetime=3600\033[35m uploads are deleted after 1 hour
+            
+            \033[0mdatabase, general:
+              \033[36me2d\033[35m sets -e2d (all -e2* args can be set using ce2* volflags)
+              \033[36md2t\033[35m disables metadata collection, overrides -e2t*
+              \033[36md2d\033[35m disables all database stuff, overrides -e2*
+              \033[36mdhash\033[35m disables file hashing on initial scans, also ehash
+              \033[36mhist=/tmp/cdb\033[35m puts thumbnails and indexes at that location
+              \033[36mscan=60\033[35m scan for new files every 60sec, same as --re-maxage
+            
+            \033[0mdatabase, audio tags:
+            "mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...
+              \033[36mmtp=.bpm=f,audio-bpm.py\033[35m uses the "audio-bpm.py" program to
+                generate ".bpm" tags from uploads (f = overwrite tags)
+              \033[36mmtp=ahash,vhash=media-hash.py\033[35m collects two tags at once
+            \033[0m"""
+            ),
+        ],
+        [
+            "urlform",
+            "",
+            dedent(
+                """
             values for --urlform:
-              "stash" dumps the data to file and returns length + checksum
-              "save,get" dumps to file and returns the page like a GET
-              "print,get" prints the data in the log and returns GET
+              \033[36mstash\033[35m dumps the data to file and returns length + checksum
+              \033[36msave,get\033[35m dumps to file and returns the page like a GET
+              \033[36mprint,get\033[35m prints the data in the log and returns GET
               (leave out the ",get" to return an error instead)
-
-            values for --ls:
-              "USR" is a user to browse as; * is anonymous, ** is all users
-              "VOL" is a single volume to scan, default is * (all vols)
-              "FLAG" is flags;
-                "v" in addition to realpaths, print usernames and vpaths
-                "ln" only prints symlinks leaving the volume mountpoint
-                "p" exits 1 if any such symlinks are found
-                "r" resumes startup after the listing
+            """
+            ),
+        ],
+        [
+            "ls",
+            "volume inspection",
+            dedent(
+                """
+            \033[35m--ls USR,VOL,FLAGS
+              \033[36mUSR\033[0m is a user to browse as; * is anonymous, ** is all users
+              \033[36mVOL\033[0m is a single volume to scan, default is * (all vols)
+              \033[36mFLAG\033[0m is flags;
+                \033[36mv\033[0m in addition to realpaths, print usernames and vpaths
+                \033[36mln\033[0m only prints symlinks leaving the volume mountpoint
+                \033[36mp\033[0m exits 1 if any such symlinks are found
+                \033[36mr\033[0m resumes startup after the listing
             examples:
               --ls '**'          # list all files which are possible to read
               --ls '**,*,ln'     # check for dangerous symlinks
               --ls '**,*,ln,p,r' # check, then start normally if safe
-            \033[0m
             """
-        ),
-    )
+            ),
+        ],
+    ]
+
     # fmt: off
     u = unicode
     ap2 = ap.add_argument_group('general options')
@@ -291,6 +355,7 @@ def run_argparse(argv, formatter):
     ap2.add_argument("-nih", action="store_true", help="no info hostname")
     ap2.add_argument("-nid", action="store_true", help="no info disk-usage")
     ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
+    ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (lifetime volflag)")
 
     ap2 = ap.add_argument_group('safety options')
     ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="scan all volumes; arguments USER,VOL,FLAGS; example [**,*,ln,p,r]")
@@ -326,10 +391,10 @@ def run_argparse(argv, formatter):
     ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
     ap2.add_argument("-e2ds", action="store_true", help="enable up2k db-scanner, sets -e2d")
     ap2.add_argument("-e2dsa", action="store_true", help="scan all folders (for search), sets -e2ds")
-    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume state")
+    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs)")
     ap2.add_argument("--no-hash", action="store_true", help="disable hashing during e2ds folder scans")
     ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
-    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval (0=off)")
+    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval, 0=off, can be set per-volume with the 'scan' volflag")
     ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
     
     ap2 = ap.add_argument_group('metadata db options')
@@ -341,7 +406,9 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--no-mtag-ff", action="store_true", help="never use FFprobe as tag reader")
     ap2.add_argument("-mtm", metavar="M=t,t,t", type=u, action="append", help="add/replace metadata mapping")
     ap2.add_argument("-mte", metavar="M,M,M", type=u, help="tags to index/display (comma-sep.)",
-        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,ac,vc,res,.fps")
+        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,vc,ac,res,.fps,ahash,vhash")
+    ap2.add_argument("-mth", metavar="M,M,M", type=u, help="tags to hide by default (comma-sep.)",
+        default=".vq,.aq,vc,ac,res,.fps")
     ap2.add_argument("-mtp", metavar="M=[f,]bin", type=u, action="append", help="read tag M using bin")
 
     ap2 = ap.add_argument_group('appearance options')
@@ -354,10 +421,22 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead")
     ap2.add_argument("--stackmon", metavar="P,S", type=u, help="write stacktrace to Path every S second")
     ap2.add_argument("--log-thrs", metavar="SEC", type=float, help="list active threads every SEC")
-    
-    return ap.parse_args(args=argv[1:])
     # fmt: on
 
+    ap2 = ap.add_argument_group("help sections")
+    for k, h, _ in sects:
+        ap2.add_argument("--help-" + k, action="store_true", help=h)
+
+    ret = ap.parse_args(args=argv[1:])
+    for k, h, t in sects:
+        k2 = "help_" + k.replace("-", "_")
+        if vars(ret)[k2]:
+            lprint("# {} help page".format(k))
+            lprint(t + "\033[0m")
+            sys.exit(0)
+
+    return ret
+
 
 def main(argv=None):
     time.strptime("19970815", "%Y%m%d")  # python#7980
@@ -396,10 +475,16 @@ def main(argv=None):
     nstrs = []
     anymod = False
     for ostr in al.v or []:
+        m = re_vol.match(ostr)
+        if not m:
+            # not our problem
+            nstrs.append(ostr)
+            continue
+
+        src, dst, perms = m.groups()
+        na = [src, dst]
         mod = False
-        oa = ostr.split(":")
-        na = oa[:2]
-        for opt in oa[2:]:
+        for opt in perms.split(":"):
             if re.match("c[^,]", opt):
                 mod = True
                 na.append("c," + opt[1:])

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 12, 5)
-CODENAME = "fil\033[33med"
-BUILD_DT = (2021, 7, 30)
+VERSION = (0, 13, 1)
+CODENAME = "future-proof"
+BUILD_DT = (2021, 8, 9)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -5,15 +5,29 @@ import re
 import os
 import sys
 import stat
+import time
 import base64
 import hashlib
 import threading
+from datetime import datetime
 
 from .__init__ import WINDOWS
-from .util import IMPLICATIONS, uncyg, undot, absreal, Pebkac, fsdec, fsenc, statdir
+from .util import (
+    IMPLICATIONS,
+    uncyg,
+    undot,
+    unhumanize,
+    absreal,
+    Pebkac,
+    fsenc,
+    statdir,
+)
 from .bos import bos
 
 
+LEELOO_DALLAS = "leeloo_dallas"
+
+
 class AXS(object):
     def __init__(self, uread=None, uwrite=None, umove=None, udel=None):
         self.uread = {} if uread is None else {k: 1 for k in uread}
@@ -30,6 +44,156 @@ class AXS(object):
         )
 
 
+class Lim(object):
+    def __init__(self):
+        self.nups = {}  # num tracker
+        self.bups = {}  # byte tracker list
+        self.bupc = {}  # byte tracker cache
+
+        self.nosub = False  # disallow subdirectories
+
+        self.smin = None  # filesize min
+        self.smax = None  # filesize max
+
+        self.bwin = None  # bytes window
+        self.bmax = None  # bytes max
+        self.nwin = None  # num window
+        self.nmax = None  # num max
+
+        self.rotn = None  # rot num files
+        self.rotl = None  # rot depth
+        self.rotf = None  # rot datefmt
+        self.rot_re = None  # rotf check
+
+    def set_rotf(self, fmt):
+        self.rotf = fmt
+        r = re.escape(fmt).replace("%Y", "[0-9]{4}").replace("%j", "[0-9]{3}")
+        r = re.sub("%[mdHMSWU]", "[0-9]{2}", r)
+        self.rot_re = re.compile("(^|/)" + r + "$")
+
+    def all(self, ip, rem, sz, abspath):
+        self.chk_nup(ip)
+        self.chk_bup(ip)
+        self.chk_rem(rem)
+        if sz != -1:
+            self.chk_sz(sz)
+
+        ap2, vp2 = self.rot(abspath)
+        if abspath == ap2:
+            return ap2, rem
+
+        return ap2, ("{}/{}".format(rem, vp2) if rem else vp2)
+
+    def chk_sz(self, sz):
+        if self.smin is not None and sz < self.smin:
+            raise Pebkac(400, "file too small")
+
+        if self.smax is not None and sz > self.smax:
+            raise Pebkac(400, "file too big")
+
+    def chk_rem(self, rem):
+        if self.nosub and rem:
+            raise Pebkac(500, "no subdirectories allowed")
+
+    def rot(self, path):
+        if not self.rotf and not self.rotn:
+            return path, ""
+
+        if self.rotf:
+            path = path.rstrip("/\\")
+            if self.rot_re.search(path.replace("\\", "/")):
+                return path, ""
+
+            suf = datetime.utcnow().strftime(self.rotf)
+            if path:
+                path += "/"
+
+            return path + suf, suf
+
+        ret = self.dive(path, self.rotl)
+        if not ret:
+            raise Pebkac(500, "no available slots in volume")
+
+        d = ret[len(path) :].strip("/\\").replace("\\", "/")
+        return ret, d
+
+    def dive(self, path, lvs):
+        items = bos.listdir(path)
+
+        if not lvs:
+            # at leaf level
+            return None if len(items) >= self.rotn else ""
+
+        dirs = [int(x) for x in items if x and all(y in "1234567890" for y in x)]
+        dirs.sort()
+
+        if not dirs:
+            # no branches yet; make one
+            sub = os.path.join(path, "0")
+            bos.mkdir(sub)
+        else:
+            # try newest branch only
+            sub = os.path.join(path, str(dirs[-1]))
+
+        ret = self.dive(sub, lvs - 1)
+        if ret is not None:
+            return os.path.join(sub, ret)
+
+        if len(dirs) >= self.rotn:
+            # full branch or root
+            return None
+
+        # make a branch
+        sub = os.path.join(path, str(dirs[-1] + 1))
+        bos.mkdir(sub)
+        ret = self.dive(sub, lvs - 1)
+        if ret is None:
+            raise Pebkac(500, "rotation bug")
+
+        return os.path.join(sub, ret)
+
+    def nup(self, ip):
+        try:
+            self.nups[ip].append(time.time())
+        except:
+            self.nups[ip] = [time.time()]
+
+    def bup(self, ip, nbytes):
+        v = [time.time(), nbytes]
+        try:
+            self.bups[ip].append(v)
+            self.bupc[ip] += nbytes
+        except:
+            self.bups[ip] = [v]
+            self.bupc[ip] = nbytes
+
+    def chk_nup(self, ip):
+        if not self.nmax or ip not in self.nups:
+            return
+
+        nups = self.nups[ip]
+        cutoff = time.time() - self.nwin
+        while nups and nups[0] < cutoff:
+            nups.pop(0)
+
+        if len(nups) >= self.nmax:
+            raise Pebkac(429, "too many uploads")
+
+    def chk_bup(self, ip):
+        if not self.bmax or ip not in self.bups:
+            return
+
+        bups = self.bups[ip]
+        cutoff = time.time() - self.bwin
+        mark = self.bupc[ip]
+        while bups and bups[0][0] < cutoff:
+            mark -= bups.pop(0)[1]
+
+        self.bupc[ip] = mark
+        if mark >= self.bmax:
+            raise Pebkac(429, "ingress saturated")
+
+
 class VFS(object):
     """single level in the virtual fs"""
 
@@ -42,6 +206,7 @@ class VFS(object):
         self.nodes = {}  # child nodes
         self.histtab = None  # all realpath->histpath
         self.dbv = None  # closest full/non-jump parent
+        self.lim = None  # type: Lim  # upload limits; only set for dbv
 
         if realpath:
             self.histpath = os.path.join(realpath, ".hist")  # db / thumbcache
@@ -165,13 +330,14 @@ class VFS(object):
             [will_move, c.umove, "move"],
             [will_del, c.udel, "delete"],
         ]:
-            if req and (uname not in d and "*" not in d):
+            if req and (uname not in d and "*" not in d) and uname != LEELOO_DALLAS:
                 m = "you don't have {}-access for this location"
                 raise Pebkac(403, m.format(msg))
 
         return vn, rem
 
     def get_dbv(self, vrem):
+        # type: (str) -> tuple[VFS, str]
         dbv = self.dbv
         if not dbv:
             return self, vrem
@@ -310,6 +476,12 @@ class VFS(object):
                 yield f
 
 
+if WINDOWS:
+    re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
+else:
+    re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
+
+
 class AuthSrv(object):
     """verifies users against given paths"""
 
@@ -319,11 +491,6 @@ class AuthSrv(object):
         self.warn_anonwrite = warn_anonwrite
         self.line_ctr = 0
 
-        if WINDOWS:
-            self.re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
-        else:
-            self.re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
-
         self.mutex = threading.Lock()
         self.reload()
 
@@ -390,6 +557,9 @@ class AuthSrv(object):
 
     def _read_vol_str(self, lvl, uname, axs, flags):
         # type: (str, str, AXS, any) -> None
+        if lvl.strip("crwmd"):
+            raise Exception("invalid volume flag: {},{}".format(lvl, uname))
+
         if lvl == "c":
             cval = True
             if "=" in uname:
@@ -453,7 +623,7 @@ class AuthSrv(object):
             # list of src:dst:permset:permset:...
             # permset is <rwmd>[,username][,username] or <c>,<flag>[=args]
             for v_str in self.args.v:
-                m = self.re_vol.match(v_str)
+                m = re_vol.match(v_str)
                 if not m:
                     raise Exception("invalid -v argument: [{}]".format(v_str))
 
@@ -545,6 +715,9 @@ class AuthSrv(object):
             )
             raise Exception("invalid config")
 
+        if LEELOO_DALLAS in all_users:
+            raise Exception("sorry, reserved username: " + LEELOO_DALLAS)
+
         promote = []
         demote = []
         for vol in vfs.all_vols.values():
@@ -603,6 +776,51 @@ class AuthSrv(object):
 
         vfs.histtab = {v.realpath: v.histpath for v in vfs.all_vols.values()}
 
+        for vol in vfs.all_vols.values():
+            lim = Lim()
+            use = False
+
+            if vol.flags.get("nosub"):
+                use = True
+                lim.nosub = True
+
+            v = vol.flags.get("sz")
+            if v:
+                use = True
+                lim.smin, lim.smax = [unhumanize(x) for x in v.split("-")]
+
+            v = vol.flags.get("rotn")
+            if v:
+                use = True
+                lim.rotn, lim.rotl = [int(x) for x in v.split(",")]
+
+            v = vol.flags.get("rotf")
+            if v:
+                use = True
+                lim.set_rotf(v)
+
+            v = vol.flags.get("maxn")
+            if v:
+                use = True
+                lim.nmax, lim.nwin = [int(x) for x in v.split(",")]
+
+            v = vol.flags.get("maxb")
+            if v:
+                use = True
+                lim.bmax, lim.bwin = [unhumanize(x) for x in v.split(",")]
+
+            if use:
+                vol.lim = lim
+
+        for vol in vfs.all_vols.values():
+            if "pk" in vol.flags and "gz" not in vol.flags and "xz" not in vol.flags:
+                vol.flags["gz"] = False  # def.pk
+
+            if "scan" in vol.flags:
+                vol.flags["scan"] = int(vol.flags["scan"])
+            elif self.args.re_maxage:
+                vol.flags["scan"] = self.args.re_maxage
+
         all_mte = {}
         errors = False
         for vol in vfs.all_vols.values():
@@ -624,9 +842,11 @@ class AuthSrv(object):
                 if k1 in vol.flags:
                     vol.flags[k2] = True
 
-            # default tag-list if unset
+            # default tag cfgs if unset
             if "mte" not in vol.flags:
                 vol.flags["mte"] = self.args.mte
+            if "mth" not in vol.flags:
+                vol.flags["mth"] = self.args.mth
 
             # append parsers from argv to volume-flags
             self._read_volflag(vol.flags, "mtp", self.args.mtp, True)

copyparty/broker_mpw.py

@@ -1,6 +1,5 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
-from copyparty.authsrv import AuthSrv
 
 import sys
 import signal
@@ -9,6 +8,7 @@ import threading
 from .broker_util import ExceptionalQueue
 from .httpsrv import HttpSrv
 from .util import FAKE_MP
+from copyparty.authsrv import AuthSrv
 
 
 class MpWorker(object):

copyparty/httpcli.py

@@ -13,10 +13,15 @@ import ctypes
 from datetime import datetime
 import calendar
 
+try:
+    import lzma
+except:
+    pass
+
 from .__init__ import E, PY2, WINDOWS, ANYWIN, unicode
 from .util import *  # noqa  # pylint: disable=unused-wildcard-import
 from .bos import bos
-from .authsrv import AuthSrv
+from .authsrv import AuthSrv, Lim
 from .szip import StreamZip
 from .star import StreamTar
 
@@ -107,7 +112,7 @@ class HttpCli(object):
             self.http_ver = "HTTP/1.1"
             # self.log("pebkac at httpcli.run #1: " + repr(ex))
             self.keepalive = False
-            self.loud_reply(unicode(ex), status=ex.code)
+            self.loud_reply(unicode(ex), status=ex.code, volsan=True)
             return self.keepalive
 
         # time.sleep(0.4)
@@ -180,6 +185,9 @@ class HttpCli(object):
                 if kc in cookies and ku not in uparam:
                     uparam[ku] = cookies[kc]
 
+        if len(uparam) > 10 or len(cookies) > 50:
+            raise Pebkac(400, "u wot m8")
+
         self.uparam = uparam
         self.cookies = cookies
         self.vpath = unquotep(vpath)  # not query, so + means +
@@ -216,19 +224,24 @@ class HttpCli(object):
             else:
                 raise Pebkac(400, 'invalid HTTP mode "{0}"'.format(self.mode))
 
-        except Pebkac as ex:
+        except Exception as ex:
+            pex = ex
+            if not hasattr(ex, "code"):
+                pex = Pebkac(500)
+
             try:
-                # self.log("pebkac at httpcli.run #2: " + repr(ex))
                 post = self.mode in ["POST", "PUT"] or "content-length" in self.headers
-                if not self._check_nonfatal(ex, post):
+                if not self._check_nonfatal(pex, post):
                     self.keepalive = False
 
-                self.log("{}\033[0m, {}".format(str(ex), self.vpath), 3)
+                msg = str(ex) if pex == ex else min_ex()
+                self.log("{}\033[0m, {}".format(msg, self.vpath), 3)
+
                 msg = "<pre>{}\r\nURL: {}\r\n".format(str(ex), self.vpath)
                 if self.hint:
                     msg += "hint: {}\r\n".format(self.hint)
 
-                self.reply(msg.encode("utf-8", "replace"), status=ex.code)
+                self.reply(msg.encode("utf-8", "replace"), status=pex.code, volsan=True)
                 return self.keepalive
             except Pebkac:
                 return False
@@ -261,8 +274,12 @@ class HttpCli(object):
         except:
             raise Pebkac(400, "client d/c while replying headers")
 
-    def reply(self, body, status=200, mime=None, headers=None):
+    def reply(self, body, status=200, mime=None, headers=None, volsan=False):
         # TODO something to reply with user-supplied values safely
+
+        if volsan:
+            body = vol_san(self.asrv.vfs.all_vols.values(), body)
+
         self.send_headers(len(body), status, mime, headers)
 
         try:
@@ -491,7 +508,11 @@ class HttpCli(object):
     def dump_to_file(self):
         reader, remains = self.get_body_reader()
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
+        lim = vfs.get_dbv(rem)[0].lim
         fdir = os.path.join(vfs.realpath, rem)
+        if lim:
+            fdir, rem = lim.all(self.ip, rem, remains, fdir)
+            bos.makedirs(fdir)
 
         addr = self.ip.replace(":", ".")
         fn = "put-{:.6f}-{}.bin".format(time.time(), addr)
@@ -499,9 +520,70 @@ class HttpCli(object):
         if self.args.nw:
             path = os.devnull
 
-        with open(fsenc(path), "wb", 512 * 1024) as f:
+        open_f = open
+        open_a = [fsenc(path), "wb", 512 * 1024]
+        open_ka = {}
+
+        # user-request || config-force
+        if ("gz" in vfs.flags or "xz" in vfs.flags) and (
+            "pk" in vfs.flags
+            or "pk" in self.uparam
+            or "gz" in self.uparam
+            or "xz" in self.uparam
+        ):
+            fb = {"gz": 9, "xz": 0}  # default/fallback level
+            lv = {}  # selected level
+            alg = None  # selected algo (gz=preferred)
+
+            # user-prefs first
+            if "gz" in self.uparam or "pk" in self.uparam:  # def.pk
+                alg = "gz"
+            if "xz" in self.uparam:
+                alg = "xz"
+            if alg:
+                v = self.uparam.get(alg)
+                lv[alg] = fb[alg] if v is None else int(v)
+
+            if alg not in vfs.flags:
+                alg = "gz" if "gz" in vfs.flags else "xz"
+
+            # then server overrides
+            pk = vfs.flags.get("pk")
+            if pk is not None:
+                # config-forced on
+                alg = alg or "gz"  # def.pk
+                try:
+                    # config-forced opts
+                    alg, lv = pk.split(",")
+                    lv[alg] = int(lv)
+                except:
+                    pass
+
+            lv[alg] = lv.get(alg) or fb.get(alg)
+
+            self.log("compressing with {} level {}".format(alg, lv.get(alg)))
+            if alg == "gz":
+                open_f = gzip.GzipFile
+                open_a = [fsenc(path), "wb", lv[alg], None, 0x5FEE6600]  # 2021-01-01
+            elif alg == "xz":
+                open_f = lzma.open
+                open_a = [fsenc(path), "wb"]
+                open_ka = {"preset": lv[alg]}
+            else:
+                self.log("fallthrough? thats a bug", 1)
+
+        with open_f(*open_a, **open_ka) as f:
             post_sz, _, sha_b64 = hashcopy(reader, f)
 
+        if lim:
+            lim.nup(self.ip)
+            lim.bup(self.ip, post_sz)
+            try:
+                lim.chk_sz(post_sz)
+            except:
+                bos.unlink(path)
+                raise
+
         if not self.args.nw:
             vfs, vrem = vfs.get_dbv(rem)
             self.conn.hsrv.broker.put(
@@ -583,7 +665,7 @@ class HttpCli(object):
         try:
             remains = int(self.headers["content-length"])
         except:
-            raise Pebkac(400, "you must supply a content-length for JSON POST")
+            raise Pebkac(411)
 
         if remains > 1024 * 1024:
             raise Pebkac(413, "json 2big")
@@ -614,12 +696,9 @@ class HttpCli(object):
             if self.vpath.endswith(k):
                 self.vpath = self.vpath[: -len(k)]
 
-        sub = None
         name = undot(body["name"])
         if "/" in name:
-            sub, name = name.rsplit("/", 1)
-            self.vpath = "/".join([self.vpath, sub]).strip("/")
-            body["name"] = name
+            raise Pebkac(400, "your client is old; press CTRL-SHIFT-R and try again")
 
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         dbv, vrem = vfs.get_dbv(rem)
@@ -630,7 +709,7 @@ class HttpCli(object):
         body["addr"] = self.ip
         body["vcfg"] = dbv.flags
 
-        if sub:
+        if rem:
             try:
                 dst = os.path.join(vfs.realpath, rem)
                 if not bos.path.isdir(dst):
@@ -650,9 +729,6 @@ class HttpCli(object):
 
         x = self.conn.hsrv.broker.put(True, "up2k.handle_json", body)
         ret = x.get()
-        if sub:
-            ret["name"] = "/".join([sub, ret["name"]])
-
         ret = json.dumps(ret)
         self.log(ret)
         self.reply(ret.encode("utf-8"), mime="application/json")
@@ -880,6 +956,11 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
+        lim = vfs.get_dbv(rem)[0].lim
+        fdir_base = os.path.join(vfs.realpath, rem)
+        if lim:
+            fdir_base, rem = lim.all(self.ip, rem, -1, fdir_base)
+
         files = []
         errmsg = ""
         t0 = time.time()
@@ -889,12 +970,9 @@ class HttpCli(object):
                     self.log("discarding incoming file without filename")
                     # fallthrough
 
+                fdir = fdir_base
+                fname = sanitize_fn(p_file, "", [".prologue.html", ".epilogue.html"])
                 if p_file and not nullwrite:
-                    fdir = os.path.join(vfs.realpath, rem)
-                    fname = sanitize_fn(
-                        p_file, "", [".prologue.html", ".epilogue.html"]
-                    )
-
                     if not bos.path.isdir(fdir):
                         raise Pebkac(404, "that folder does not exist")
 
@@ -905,27 +983,43 @@ class HttpCli(object):
                     fname = os.devnull
                     fdir = ""
 
+                if lim:
+                    lim.chk_bup(self.ip)
+                    lim.chk_nup(self.ip)
+                    if not nullwrite:
+                        bos.makedirs(fdir)
+
                 try:
                     with ren_open(fname, "wb", 512 * 1024, **open_args) as f:
                         f, fname = f["orz"]
-                        self.log("writing to {}/{}".format(fdir, fname))
+                        abspath = os.path.join(fdir, fname)
+                        self.log("writing to {}".format(abspath))
                         sz, sha512_hex, _ = hashcopy(p_data, f)
                         if sz == 0:
                             raise Pebkac(400, "empty files in post")
 
-                        files.append([sz, sha512_hex, p_file, fname])
-                        dbv, vrem = vfs.get_dbv(rem)
-                        self.conn.hsrv.broker.put(
-                            False,
-                            "up2k.hash_file",
-                            dbv.realpath,
-                            dbv.flags,
-                            vrem,
-                            fname,
-                            self.ip,
-                            time.time(),
-                        )
-                        self.conn.nbyte += sz
+                    if lim:
+                        lim.nup(self.ip)
+                        lim.bup(self.ip, sz)
+                        try:
+                            lim.chk_sz(sz)
+                        except:
+                            bos.unlink(abspath)
+                            raise
+
+                    files.append([sz, sha512_hex, p_file, fname])
+                    dbv, vrem = vfs.get_dbv(rem)
+                    self.conn.hsrv.broker.put(
+                        False,
+                        "up2k.hash_file",
+                        dbv.realpath,
+                        dbv.flags,
+                        vrem,
+                        fname,
+                        self.ip,
+                        time.time(),
+                    )
+                    self.conn.nbyte += sz
 
                 except Pebkac:
                     if fname != os.devnull:
@@ -944,7 +1038,7 @@ class HttpCli(object):
                     raise
 
         except Pebkac as ex:
-            errmsg = unicode(ex)
+            errmsg = volsan(self.asrv.vfs.all_vols.values(), unicode(ex))
 
         td = max(0.1, time.time() - t0)
         sz_total = sum(x[0] for x in files)
@@ -1023,6 +1117,20 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
+        clen = int(self.headers.get("content-length", -1))
+        if clen == -1:
+            raise Pebkac(411)
+
+        rp, fn = vsplit(rem)
+        fp = os.path.join(vfs.realpath, rp)
+        lim = vfs.get_dbv(rem)[0].lim
+        if lim:
+            fp, rp = lim.all(self.ip, rp, clen, fp)
+            bos.makedirs(fp)
+
+        fp = os.path.join(fp, fn)
+        rem = "{}/{}".format(rp, fn).strip("/")
+
         if not rem.endswith(".md"):
             raise Pebkac(400, "only markdown pls")
 
@@ -1034,7 +1142,6 @@ class HttpCli(object):
             self.reply(response.encode("utf-8"))
             return True
 
-        fp = os.path.join(vfs.realpath, rem)
         srv_lastmod = srv_lastmod3 = -1
         try:
             st = bos.stat(fp)
@@ -1088,6 +1195,15 @@ class HttpCli(object):
         with open(fsenc(fp), "wb", 512 * 1024) as f:
             sz, sha512, _ = hashcopy(p_data, f)
 
+        if lim:
+            lim.nup(self.ip)
+            lim.bup(self.ip, sz)
+            try:
+                lim.chk_sz(sz)
+            except:
+                bos.unlink(fp)
+                raise
+
         new_lastmod = bos.stat(fp).st_mtime
         new_lastmod3 = int(new_lastmod * 1000)
         sha512 = sha512[:56]
@@ -1755,7 +1871,7 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": json.dumps(perms),
             "taglist": [],
-            "tag_order": [],
+            "def_hcols": [],
             "have_up2k_idx": ("e2d" in vn.flags),
             "have_tags_idx": ("e2t" in vn.flags),
             "have_mv": (not self.args.no_mv),
@@ -1952,8 +2068,8 @@ class HttpCli(object):
         j2a["logues"] = logues
         j2a["taglist"] = taglist
 
-        if "mte" in vn.flags:
-            j2a["tag_order"] = json.dumps(vn.flags["mte"].split(","))
+        if "mth" in vn.flags:
+            j2a["def_hcols"] = vn.flags["mth"].split(",")
 
         if self.args.css_browser:
             j2a["css"] = self.args.css_browser

copyparty/httpsrv.py

@@ -174,25 +174,26 @@ class HttpSrv(object):
         now = time.time()
 
         if now - (self.tp_time or now) > 300:
+            m = "httpserver threadpool died: tpt {:.2f}, now {:.2f}, nthr {}, ncli {}"
+            self.log(self.name, m.format(self.tp_time, now, self.tp_nthr, self.ncli), 1)
+            self.tp_time = None
             self.tp_q = None
 
-        if self.tp_q:
-            self.tp_q.put((sck, addr))
-            with self.mutex:
-                self.ncli += 1
+        with self.mutex:
+            self.ncli += 1
+            if self.tp_q:
                 self.tp_time = self.tp_time or now
-                self.tp_ncli = max(self.tp_ncli, self.ncli + 1)
+                self.tp_ncli = max(self.tp_ncli, self.ncli)
                 if self.tp_nthr < self.ncli + 4:
                     self.start_threads(8)
-            return
+
+                self.tp_q.put((sck, addr))
+                return
 
         if not self.args.no_htp:
             m = "looks like the httpserver threadpool died; please make an issue on github and tell me the story of how you pulled that off, thanks and dog bless\n"
             self.log(self.name, m, 1)
 
-        with self.mutex:
-            self.ncli += 1
-
         thr = threading.Thread(
             target=self.thr_client,
             args=(sck, addr),

copyparty/mtag.py

@@ -434,7 +434,15 @@ class MTag(object):
             try:
                 v = getattr(md.info, attr)
             except:
-                continue
+                if k != "ac":
+                    continue
+
+                try:
+                    v = str(md.info).split(".")[1]
+                    if v.startswith("ogg"):
+                        v = v[3:]
+                except:
+                    continue
 
             if not v:
                 continue

copyparty/sutil.py

@@ -18,8 +18,7 @@ def errdesc(errors):
         tf_path = tf.name
         tf.write("\r\n".join(report).encode("utf-8", "replace"))
 
-    dt = datetime.utcfromtimestamp(time.time())
-    dt = dt.strftime("%Y-%m%d-%H%M%S")
+    dt = datetime.utcnow().strftime("%Y-%m%d-%H%M%S")
 
     bos.chmod(tf_path, 0o444)
     return {

copyparty/svchub.py

@@ -14,7 +14,7 @@ from datetime import datetime, timedelta
 import calendar
 
 from .__init__ import E, PY2, WINDOWS, ANYWIN, MACOS, VT100, unicode
-from .util import mp, start_log_thrs, start_stackmon, min_ex
+from .util import mp, start_log_thrs, start_stackmon, min_ex, ansi_re
 from .authsrv import AuthSrv
 from .tcpsrv import TcpSrv
 from .up2k import Up2k
@@ -41,7 +41,6 @@ class SvcHub(object):
         self.stop_cond = threading.Condition()
         self.httpsrv_up = 0
 
-        self.ansi_re = re.compile("\033\\[[^m]*m")
         self.log_mutex = threading.Lock()
         self.next_day = 0
 
@@ -111,7 +110,7 @@ class SvcHub(object):
         thr.start()
 
     def _logname(self):
-        dt = datetime.utcfromtimestamp(time.time())
+        dt = datetime.utcnow()
         fn = self.args.lo
         for fs in "YmdHMS":
             fs = "%" + fs
@@ -244,8 +243,7 @@ class SvcHub(object):
             return
 
         with self.log_mutex:
-            ts = datetime.utcfromtimestamp(time.time())
-            ts = ts.strftime("%Y-%m%d-%H%M%S.%f")[:-3]
+            ts = datetime.utcnow().strftime("%Y-%m%d-%H%M%S.%f")[:-3]
             self.logf.write("@{} [{}] {}\n".format(ts, src, msg))
 
             now = time.time()
@@ -257,7 +255,7 @@ class SvcHub(object):
             self.logf.close()
             self._setup_logfile("")
 
-        dt = datetime.utcfromtimestamp(time.time())
+        dt = datetime.utcnow()
 
         # unix timestamp of next 00:00:00 (leap-seconds safe)
         day_now = dt.day
@@ -280,9 +278,9 @@ class SvcHub(object):
             if not VT100:
                 fmt = "{} {:21} {}\n"
                 if "\033" in msg:
-                    msg = self.ansi_re.sub("", msg)
+                    msg = ansi_re.sub("", msg)
                 if "\033" in src:
-                    src = self.ansi_re.sub("", src)
+                    src = ansi_re.sub("", src)
             elif c:
                 if isinstance(c, int):
                     msg = "\033[3{}m{}".format(c, msg)

copyparty/th_cli.py

@@ -26,6 +26,9 @@ class ThumbCli(object):
         if is_vid and self.args.no_vthumb:
             return None
 
+        if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg"]:
+            return os.path.join(ptop, rem)
+
         if fmt == "j" and self.args.th_no_jpg:
             fmt = "w"
 

copyparty/th_srv.py

@@ -205,8 +205,8 @@ class ThumbSrv(object):
                 try:
                     fun(abspath, tpath)
                 except:
-                    msg = "{} failed on {}\n{}"
-                    self.log(msg.format(fun.__name__, abspath, min_ex()), 3)
+                    msg = "{} could not create thumbnail of {}\n{}"
+                    self.log(msg.format(fun.__name__, abspath, min_ex()), "1;30")
                     with open(tpath, "wb") as _:
                         pass
 
@@ -286,8 +286,9 @@ class ThumbSrv(object):
         cmd += seek
         cmd += [
             b"-i", fsenc(abspath),
+            b"-map", b"0:v:0",
             b"-vf", scale,
-            b"-vframes", b"1",
+            b"-frames:v", b"1",
         ]
         # fmt: on
 
@@ -308,8 +309,9 @@ class ThumbSrv(object):
 
         ret, sout, serr = runcmd(cmd)
         if ret != 0:
-            msg = ["ff: {}".format(x) for x in serr.split("\n")]
-            self.log("FFmpeg failed:\n" + "\n".join(msg), c="1;30")
+            m = "FFmpeg failed (probably a corrupt video file):\n"
+            m += "\n".join(["ff: {}".format(x) for x in serr.split("\n")])
+            self.log(m, c="1;30")
             raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
 
     def poke(self, tdir):

copyparty/u2idx.py

@@ -88,7 +88,7 @@ class U2idx(object):
         is_date = False
         kw_key = ["(", ")", "and ", "or ", "not "]
         kw_val = ["==", "=", "!=", ">", ">=", "<", "<=", "like "]
-        ptn_mt = re.compile(r"^\.?[a-z]+$")
+        ptn_mt = re.compile(r"^\.?[a-z_-]+$")
         mt_ctr = 0
         mt_keycmp = "substr(up.w,1,16)"
         mt_keycmp2 = None

copyparty/up2k.py

@@ -36,7 +36,7 @@ from .util import (
     min_ex,
 )
 from .bos import bos
-from .authsrv import AuthSrv
+from .authsrv import AuthSrv, LEELOO_DALLAS
 from .mtag import MTag, MParser
 
 try:
@@ -176,36 +176,71 @@ class Up2k(object):
         return None
 
     def _sched_rescan(self):
-        maxage = self.args.re_maxage
         volage = {}
         while True:
             time.sleep(self.args.re_int)
             now = time.time()
-            vpaths = list(sorted(self.asrv.vfs.all_vols.keys()))
             with self.mutex:
-                if maxage:
-                    for vp in vpaths:
-                        if vp not in volage:
-                            volage[vp] = now
+                for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
+                    maxage = vol.flags.get("scan")
+                    if not maxage:
+                        continue
 
-                        if now - volage[vp] >= maxage:
-                            self.need_rescan[vp] = 1
+                    if vp not in volage:
+                        volage[vp] = now
 
-                if not self.need_rescan:
-                    continue
+                    if now - volage[vp] >= maxage:
+                        self.need_rescan[vp] = 1
 
                 vols = list(sorted(self.need_rescan.keys()))
                 self.need_rescan = {}
 
-            err = self.rescan(self.asrv.vfs.all_vols, vols)
-            if err:
-                for v in vols:
-                    self.need_rescan[v] = True
+            if vols:
+                err = self.rescan(self.asrv.vfs.all_vols, vols)
+                if err:
+                    for v in vols:
+                        self.need_rescan[v] = True
 
+                    continue
+
+                for v in vols:
+                    volage[v] = now
+
+            if self.args.no_lifetime:
                 continue
 
-            for v in vols:
-                volage[v] = now
+            for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
+                lifetime = vol.flags.get("lifetime")
+                if not lifetime:
+                    continue
+
+                cur = self.cur.get(vol.realpath)
+                if not cur:
+                    continue
+
+                nrm = 0
+                deadline = time.time() - int(lifetime)
+                q = "select rd, fn from up where at > 0 and at < ? limit 100"
+                while True:
+                    with self.mutex:
+                        hits = cur.execute(q, (deadline,)).fetchall()
+
+                    if not hits:
+                        break
+
+                    for rd, fn in hits:
+                        if rd.startswith("//") or fn.startswith("//"):
+                            rd, fn = s3dec(rd, fn)
+
+                        fvp = "{}/{}".format(rd, fn).strip("/")
+                        if vp:
+                            fvp = "{}/{}".format(vp, fvp)
+
+                        self._handle_rm(LEELOO_DALLAS, None, fvp)
+                        nrm += 1
+
+                if nrm:
+                    self.log("{} files graduated in {}".format(nrm, vp))
 
     def _vis_job_progress(self, job):
         perc = 100 - (len(job["need"]) * 100.0 / len(job["hash"]))
@@ -1148,6 +1183,16 @@ class Up2k(object):
                             cur.connection.commit()
 
             if not job:
+                vfs = self.asrv.vfs.all_vols[cj["vtop"]]
+                if vfs.lim:
+                    ap1 = os.path.join(cj["ptop"], cj["prel"])
+                    ap2, cj["prel"] = vfs.lim.all(
+                        cj["addr"], cj["prel"], cj["size"], ap1
+                    )
+                    bos.makedirs(ap2)
+                    vfs.lim.nup(cj["addr"])
+                    vfs.lim.bup(cj["addr"], cj["size"])
+
                 job = {
                     "wark": wark,
                     "t0": now,
@@ -1178,8 +1223,11 @@ class Up2k(object):
 
                 self._new_upload(job)
 
+            purl = "/{}/".format("{}/{}".format(job["vtop"], job["prel"]).strip("/"))
+
             return {
                 "name": job["name"],
+                "purl": purl,
                 "size": job["size"],
                 "lmod": job["lmod"],
                 "hash": job["need"],
@@ -1230,7 +1278,7 @@ class Up2k(object):
                     hops = len(ndst[nc:]) - 1
                     lsrc = "../" * hops + "/".join(lsrc)
             os.symlink(fsenc(lsrc), fsenc(ldst))
-        except (AttributeError, OSError) as ex:
+        except Exception as ex:
             self.log("cannot symlink; creating copy: " + repr(ex))
             shutil.copy2(fsenc(src), fsenc(dst))
 
@@ -1405,7 +1453,7 @@ class Up2k(object):
                     try:
                         ptop = dbv.realpath
                         cur, wark, _, _, _, _ = self._find_from_vpath(ptop, volpath)
-                        self._forget_file(ptop, volpath, cur, wark)
+                        self._forget_file(ptop, volpath, cur, wark, True)
                     finally:
                         cur.connection.commit()
 
@@ -1422,9 +1470,10 @@ class Up2k(object):
         if not srem:
             raise Pebkac(400, "mv: cannot move a mountpoint")
 
-        st = bos.stat(sabs)
-        if stat.S_ISREG(st.st_mode):
-            return self._mv_file(uname, svp, dvp)
+        st = bos.lstat(sabs)
+        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            with self.mutex:
+                return self._mv_file(uname, svp, dvp)
 
         jail = svn.get_dbv(srem)[0]
         permsets = [[True, False, True]]
@@ -1449,7 +1498,8 @@ class Up2k(object):
                     raise Pebkac(500, "mv: bug at {}, top {}".format(svpf, svp))
 
                 dvpf = dvp + svpf[len(svp) :]
-                self._mv_file(uname, svpf, dvpf)
+                with self.mutex:
+                    self._mv_file(uname, svpf, dvpf)
 
         rmdirs(self.log_func, scandir, True, sabs)
         return "k"
@@ -1491,10 +1541,10 @@ class Up2k(object):
             fsize = st.st_size
 
         if w:
-            if c2:
+            if c2 and c2 != c1:
                 self._copy_tags(c1, c2, w)
 
-            self._forget_file(svn.realpath, srem, c1, w)
+            self._forget_file(svn.realpath, srem, c1, w, c1 != c2)
             self._relink(w, svn.realpath, srem, dabs)
             c1.connection.commit()
 
@@ -1535,16 +1585,20 @@ class Up2k(object):
             return cur, wark, ftime, fsize, ip, at
         return cur, None, None, None, None, None
 
-    def _forget_file(self, ptop, vrem, cur, wark):
+    def _forget_file(self, ptop, vrem, cur, wark, drop_tags):
         """forgets file in db, fixes symlinks, does not delete"""
         srd, sfn = vsplit(vrem)
         self.log("forgetting {}".format(vrem))
         if wark:
             self.log("found {} in db".format(wark))
-            self._relink(wark, ptop, vrem, None)
+            if drop_tags:
+                if self._relink(wark, ptop, vrem, None):
+                    drop_tags = False
+
+            if drop_tags:
+                q = "delete from mt where w=?"
+                cur.execute(q, (wark[:16],))
 
-            q = "delete from mt where w=?"
-            cur.execute(q, (wark[:16],))
             self.db_rm(cur, srd, sfn)
 
         reg = self.registry.get(ptop)
@@ -1581,7 +1635,7 @@ class Up2k(object):
                     self.log("found {} dupe: [{}] {}".format(wark, ptop, dvrem))
 
         if not dupes:
-            return
+            return 0
 
         full = {}
         links = {}
@@ -1597,7 +1651,7 @@ class Up2k(object):
             # deleting final remaining full copy; swap it with a symlink
             slabs = list(sorted(links.keys()))[0]
             ptop, rem = links.pop(slabs)
-            self.log("linkswap [{}] and [{}]".format(sabs, dabs))
+            self.log("linkswap [{}] and [{}]".format(sabs, slabs))
             bos.unlink(slabs)
             bos.rename(sabs, slabs)
             self._symlink(slabs, sabs, False)
@@ -1618,6 +1672,8 @@ class Up2k(object):
 
             self._symlink(dabs, alink, False)
 
+        return len(full) + len(links)
+
     def _get_wark(self, cj):
         if len(cj["name"]) > 1024 or len(cj["hash"]) > 512 * 1024:  # 16TiB
             raise Pebkac(400, "name or numchunks not according to spec")

copyparty/util.py

@@ -58,6 +58,9 @@ except:
         return struct.unpack(f.decode("ascii"), *a, **ka)
 
 
+ansi_re = re.compile("\033\\[[^m]*m")
+
+
 surrogateescape.register_surrogateescape()
 FS_ENCODING = sys.getfilesystemencoding()
 if WINDOWS and PY2:
@@ -77,6 +80,7 @@ HTTPCODE = {
     403: "Forbidden",
     404: "Not Found",
     405: "Method Not Allowed",
+    411: "Length Required",
     413: "Payload Too Large",
     416: "Requested Range Not Satisfiable",
     422: "Unprocessable Entity",
@@ -340,6 +344,15 @@ def log_thrs(log, ival, name):
         log(name, "\033[0m \033[33m".join(tv), 3)
 
 
+def vol_san(vols, txt):
+    for vol in vols:
+        txt = txt.replace(
+            vol.realpath.encode("utf-8"), vol.vpath.encode("utf-8")
+        )
+    
+    return txt
+
+
 def min_ex():
     et, ev, tb = sys.exc_info()
     tb = traceback.extract_tb(tb)
@@ -684,6 +697,17 @@ def humansize(sz, terse=False):
     return ret.replace("iB", "").replace(" ", "")
 
 
+def unhumanize(sz):
+    try:
+        return float(sz)
+    except:
+        pass
+
+    mul = sz[-1:].lower()
+    mul = {"k": 1024, "m": 1024 * 1024, "g": 1024 * 1024 * 1024}.get(mul, 1)
+    return float(sz[:-1]) * mul
+
+
 def get_spd(nbyte, t0, t=None):
     if t is None:
         t = time.time()
@@ -1065,7 +1089,7 @@ def statdir(logger, scandir, lstat, top):
 def rmdirs(logger, scandir, lstat, top):
     if not os.path.exists(fsenc(top)) or not os.path.isdir(fsenc(top)):
         top = os.path.dirname(top)
-    
+
     dirs = statdir(logger, scandir, lstat, top)
     dirs = [x[0] for x in dirs if stat.S_ISDIR(x[1].st_mode)]
     dirs = [os.path.join(top, x) for x in dirs]

copyparty/web/browser.css

@@ -22,128 +22,9 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 pre, code, tt {
 	font-family: monospace, monospace;
 }
-
-
-
-#tt, #toast {
-	position: fixed;
-	max-width: 34em;
-	background: #222;
-	border: 0 solid #777;
-	box-shadow: 0 .2em .5em #222;
-	border-radius: .4em;
-	z-index: 9001;
-}
-#tt {
-	overflow: hidden;
-	margin-top: 1em;
-	padding: 0 1.3em;
-	height: 0;
-	opacity: .1;
-	transition: opacity 0.14s, height 0.14s, padding 0.14s;
-}
-#toast {
-	top: 1.4em;
-	right: -1em;
-	line-height: 1.5em;
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	transform: translateX(100%);
-	transition:
-		transform .4s cubic-bezier(.2, 1.2, .5, 1),
-		right .4s cubic-bezier(.2, 1.2, .5, 1);
-	text-shadow: 1px 1px 0 #000;
-	color: #fff;
-}
-#toastc {
-	display: inline-block;
-	position: absolute;
-	overflow: hidden;
-	left: 0;
-	width: 0;
-	opacity: 0;
-	padding: .3em 0;
-	margin: -.3em 0 0 0;
-	line-height: 1.5em;
-	color: #000;
-	border: none;
-	outline: none;
-	text-shadow: none;
-	border-radius: .5em 0 0 .5em;
-	transition: left .3s, width .3s, padding .3s, opacity .3s;
-}
-#toast pre {
-	margin: 0;
-}
-#toast.vis {
-	right: 1.3em;
-	transform: unset;
-}
-#toast.vis #toastc {
-	left: -2em;
-	width: .4em;
-	padding: .3em .8em;
-	opacity: 1;
-}
-#toast.inf {
-	background: #07a;
-	border-color: #0be;
-}
-#toast.inf #toastc {
-	background: #0be;
-}
-#toast.ok {
-	background: #4a0;
-	border-color: #8e4;
-}
-#toast.ok #toastc {
-	background: #8e4;
-}
-#toast.warn {
-	background: #970;
-	border-color: #fc0;
-}
-#toast.warn #toastc {
-	background: #fc0;
-}
-#toast.err {
-	background: #900;
-	border-color: #d06;
-}
-#toast.err #toastc {
-	background: #d06;
-}
-#tt.b {
-	padding: 0 2em;
-	border-radius: .5em;
-	box-shadow: 0 .2em 1em #000;
-}
-#tt.show {
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	height: auto;
-	opacity: 1;
-}
-#tt.show.b {
-	padding: 1.5em 2em;
-	border-width: .5em 0;
-}
-#tt code {
-	background: #3c3c3c;
-	padding: .1em .3em;
-	border-top: 1px solid #777;
-	border-radius: .3em;
-	line-height: 1.7em;
-}
-#tt em {
-	color: #f6a;
-}
 #path,
 #path * {
 	font-size: 1em;
@@ -810,6 +691,7 @@ input.eq_gain {
 #wrap {
 	margin-top: 2em;
 	min-height: 90vh;
+	padding-bottom: 5em;
 }
 #tree {
 	display: none;
@@ -1068,10 +950,69 @@ html.light #ggrid a:hover {
 	margin: 0;
 	padding: 0;
 }
+#rui {
+	position: fixed;
+	top: 0;
+	left: 0;
+	width: calc(100% - 2em);
+	height: auto;
+	overflow: auto;
+	max-height: calc(100% - 2em);
+	border-bottom: .5em solid #999;
+	box-shadow: 0 0 5em rgba(0,0,0,0.8);
+	background: #333;
+	padding: 1em;
+	z-index: 765;
+}
+html.light #rui {
+	color: #fff;
+}
+#rui div+div {
+	margin-top: 1em;
+}
+#rui table {
+	width: 100%;
+	border-collapse: collapse;
+}
+#rui td+td {
+	padding: .2em 0 .2em .5em;
+}
+#rn_vadv input {
+	font-family: monospace, monospace;
+}
+#rui td+td,
+#rui td input[type="text"] {
+	width: 100%;
+}
+#rn_f.m td:first-child {
+	white-space: nowrap;
+}
+#rn_f.m td+td {
+	width: 50%;
+}
+#rn_f .err td {
+	background: #c00;
+}
+#rn_f .err input[readonly] {
+	background: #600;
+	color: #fc0;
+}
+#rui input[readonly] {
+	color: #fff;
+	background: #444;
+	border: 1px solid #777;
+	padding: .2em .25em;
+}
+#rui h1 {
+	margin: 0 0 .3em 0;
+	padding: 0;
+	font-size: 1.5em;
+}
 #pvol,
 #barbuf,
 #barpos,
 #u2conf label,
+#rui label,
 #ops {
 	-webkit-user-select: none;
 	-moz-user-select: none;
@@ -1093,6 +1034,11 @@ html.light #ggrid a:hover {
 
 
 
+
+
+
+
+
 
 
 
@@ -1103,21 +1049,6 @@ html.light {
 	background: #eee;
 	text-shadow: none;
 }
-html.light #tt {
-	background: #fff;
-	border-color: #888 #000 #777 #000;
-}
-html.light #tt,
-html.light #toast {
-	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
-}
-html.light #tt code {
-	background: #060;
-	color: #fff;
-}
-html.light #tt em {
-	color: #d38;
-}
 html.light #ops,
 html.light .opbox,
 html.light #srch_form {
@@ -1334,6 +1265,30 @@ html.light #tree::-webkit-scrollbar {
 
 
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/* bbox */
+
 #bbox-overlay {
 	display: none;
 	opacity: 0;
@@ -1512,3 +1467,331 @@ html.light #bbox-overlay figcaption a {
 	0%, 100% {transform: scale(0)}
 	50% {transform: scale(1)}
 }
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/* upload.css */
+
+#op_up2k {
+	padding: 0 1em 1em 1em;
+}
+#u2form {
+	position: absolute;
+	top: 0;
+	left: 0;
+	width: 2px;
+	height: 2px;
+	overflow: hidden;
+}
+#u2form input {
+	background: #444;
+	border: 0px solid #444;
+	outline: none;
+}
+#u2err.err {
+	color: #f87;
+	padding: .5em;
+}
+#u2err.msg {
+	color: #999;
+	padding: .5em;
+	font-size: .9em;
+}
+#u2btn {
+	color: #eee;
+	background: #555;
+	background: -moz-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	background: -webkit-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	background: linear-gradient(to bottom, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#489', endColorstr='#38788a', GradientType=0);
+	text-decoration: none;
+	line-height: 1.3em;
+	border: 1px solid #222;
+	border-radius: .4em;
+	text-align: center;
+	font-size: 1.5em;
+	margin: .5em auto;
+	padding: .8em 0;
+	width: 16em;
+	cursor: pointer;
+	box-shadow: .4em .4em 0 #111;
+}
+#op_up2k.srch #u2btn {
+	background: linear-gradient(to bottom, #ca3 0%, #fd8 50%, #fc6 51%, #b92 100%);
+	text-shadow: 1px 1px 1px #fc6;
+	color: #333;
+}
+#u2conf #u2btn {
+	margin: -1.5em 0;
+	padding: .8em 0;
+	width: 100%;
+	max-width: 12em;
+	display: inline-block;
+}
+#u2conf #u2btn_cw {
+	text-align: right;
+}
+#u2notbtn {
+	display: none;
+	text-align: center;
+	background: #333;
+	padding-top: 1em;
+}
+#u2notbtn * {
+	line-height: 1.3em;
+}
+#u2tab {
+	margin: 3em auto;
+	width: calc(100% - 2em);
+	max-width: 100em;
+}
+#op_up2k.srch #u2tab {
+	max-width: none;
+}
+#u2tab td {
+	border: 1px solid #ccc;
+	border-width: 0 0px 1px 0;
+	padding: .1em .3em;
+}
+#u2tab td:nth-child(2) {
+	width: 5em;
+	white-space: nowrap;
+}
+#u2tab td:nth-child(3) {
+	width: 40%;
+}
+#op_up2k.srch td.prog {
+	font-family: sans-serif;
+	font-size: 1em;
+	width: auto;
+}
+#u2tab tbody tr:hover td {
+	background: #222;
+}
+#u2cards {
+	padding: 1em 0 .3em 1em;
+	margin: 1.5em auto -2.5em auto;
+	white-space: nowrap;
+	text-align: center;
+	overflow: hidden;
+}
+#u2cards.w {
+	width: 45em;
+	text-align: left;
+}
+#u2cards a {
+	padding: .2em 1em;
+	border: 1px solid #777;
+	border-width: 0 0 1px 0;
+	background: linear-gradient(to bottom, #333, #222);
+}
+#u2cards a:first-child {
+	border-radius: .4em 0 0 0;
+}
+#u2cards a:last-child {
+	border-radius: 0 .4em 0 0;
+}
+#u2cards a.act {
+	padding-bottom: .5em;
+	border-width: 1px 1px .1em 1px;
+	border-radius: .3em .3em 0 0;
+	margin-left: -1px;
+	background: linear-gradient(to bottom, #464, #333 80%);
+	box-shadow: 0 -.17em .67em #280;
+	border-color: #7c5 #583 #333 #583;
+	position: relative;
+	color: #fd7;
+}
+#u2cards span {
+	color: #fff;
+}
+#u2conf {
+	margin: 1em auto;
+	width: 30em;
+}
+#u2conf.has_btn {
+	width: 48em;
+}
+#u2conf * {
+	text-align: center;
+	line-height: 1em;
+	margin: 0;
+	padding: 0;
+	border: none;
+	outline: none;
+}
+#u2conf .txtbox {
+	width: 3em;
+	color: #fff;
+	background: #444;
+	border: 1px solid #777;
+	font-size: 1.2em;
+	padding: .15em 0;
+	height: 1.05em;
+}
+#u2conf .txtbox.err {
+	background: #922;
+}
+#u2conf a {
+	color: #fff;
+	background: #c38;
+	text-decoration: none;
+	border-radius: .1em;
+	font-size: 1.5em;
+	padding: .1em 0;
+	margin: 0 -1px;
+	width: 1.5em;
+	height: 1em;
+	display: inline-block;
+	position: relative;
+	bottom: -0.08em;
+}
+#u2conf input+a {
+	background: #d80;
+}
+#u2conf label {
+	font-size: 1.6em;
+	width: 2em;
+	height: 1em;
+	padding: .4em 0;
+	display: block;
+	border-radius: .25em;
+}
+#u2conf input[type="checkbox"] {
+	position: relative;
+	opacity: .02;
+	top: 2em;
+}
+#u2conf input[type="checkbox"]+label {
+	position: relative;
+	background: #603;
+	border-bottom: .2em solid #a16;
+	box-shadow: 0 .1em .3em #a00 inset;
+}
+#u2conf input[type="checkbox"]:checked+label {
+	background: #6a1;
+	border-bottom: .2em solid #efa;
+	box-shadow: 0 .1em .5em #0c0;
+}
+#u2conf input[type="checkbox"]+label:hover {
+	box-shadow: 0 .1em .3em #fb0;
+	border-color: #fb0;
+}
+#op_up2k.srch #u2conf td:nth-child(1)>*,
+#op_up2k.srch #u2conf td:nth-child(2)>*,
+#op_up2k.srch #u2conf td:nth-child(3)>* {
+	background: #777;
+	border-color: #ccc;
+	box-shadow: none;
+	opacity: .2;
+}
+#u2foot {
+	color: #fff;
+	font-style: italic;
+}
+#u2foot .warn {
+	font-size: 1.3em;
+	padding: .5em .8em;
+	margin: 1em -.6em;
+	color: #f74;
+	background: #322;
+	border: 1px solid #633;
+	border-width: .1em 0;
+	text-align: center;
+}
+#u2foot .warn span {
+	color: #f86;
+}
+html.light #u2foot .warn {
+	color: #b00;
+	background: #fca;
+	border-color: #f70;
+}
+html.light #u2foot .warn span {
+	color: #930;
+}
+#u2foot span {
+	color: #999;
+	font-size: .9em;
+	font-weight: normal;
+}
+#u2footfoot {
+	margin-bottom: -1em;
+}
+.prog {
+	font-family: monospace, monospace;
+}
+#u2tab a>span {
+	font-weight: bold;
+	font-style: italic;
+	color: #fff;
+	padding-left: .2em;
+}
+#u2cleanup {
+	float: right;
+	margin-bottom: -.3em;
+}
+.fsearch_explain {
+	padding-left: .7em;
+	font-size: 1.1em;
+	line-height: 0;
+}
+
+
+
+
+
+html.light #u2btn {
+	box-shadow: .4em .4em 0 #ccc;
+}
+html.light #u2cards span {
+	color: #000;
+}
+html.light #u2cards a {
+	background: linear-gradient(to bottom, #eee, #fff);
+}
+html.light #u2cards a.act {
+	color: #037;
+	background: inherit;
+	box-shadow: 0 -.17em .67em #0ad;
+	border-color: #09c #05a #eee #05a;
+}
+html.light #u2conf .txtbox {
+	background: #fff;
+	color: #444;
+}
+html.light #u2conf .txtbox.err {
+	background: #f96;
+	color: #300;
+}
+html.light #op_up2k.srch #u2btn {
+	border-color: #a80;
+}
+html.light #u2foot {
+	color: #000;
+}
+html.light #u2tab tbody tr:hover td {
+	background: #fff;
+}

copyparty/web/browser.html

@@ -6,10 +6,10 @@
 	<title>⇆🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/browser.css?_={{ ts }}">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/upload.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="/.cpr/browser.css?_={{ ts }}">
 	{%- if css %}
-	<link rel="stylesheet" type="text/css" media="screen" href="{{ css }}?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ css }}?_={{ ts }}">
 	{%- endif %}
 </head>
 
@@ -125,7 +125,7 @@
 	<script>
 		var acct = "{{ acct }}",
 			perms = {{ perms }},
-			tag_order_cfg = {{ tag_order }},
+			def_hcols = {{ def_hcols|tojson }},
 			have_up2k_idx = {{ have_up2k_idx|tojson }},
 			have_tags_idx = {{ have_tags_idx|tojson }},
 			have_mv = {{ have_mv|tojson }},

copyparty/web/browser.js

@@ -133,6 +133,7 @@ ebi('op_cfg').innerHTML = (
 	'	<div>\n' +
 	'		<a id="tooltips" class="tgl btn" href="#" tt="◔ ◡ ◔">ℹ️ tooltips</a>\n' +
 	'		<a id="lightmode" class="tgl btn" href="#">☀️ lightmode</a>\n' +
+	'		<a id="dotfiles" class="tgl btn" href="#" tt="show hidden files (if server permits)">dotfiles</a>\n' +
 	'		<a id="griden" class="tgl btn" href="#" tt="toggle icons or list-view$NHotkey: G">田 the grid</a>\n' +
 	'		<a id="thumbs" class="tgl btn" href="#" tt="in icon view, toggle icons or thumbnails$NHotkey: T">🖼️ thumbs</a>\n' +
 	'	</div>\n' +
@@ -521,15 +522,14 @@ var mp = new MPlayer();
 makeSortable(ebi('files'), mp.read_order.bind(mp));
 
 
-function get_np() {
+function ft2dict(tr) {
 	var th = ebi('files').tHead.rows[0].cells,
-		tr = QS('#files tr.play').cells,
 		rv = [],
 		ra = [],
 		rt = {};
 
 	for (var a = 1, aa = th.length; a < aa; a++) {
-		var tv = tr[a].textContent,
+		var tv = tr.cells[a].textContent,
 			tk = a == 1 ? 'file' : th[a].getAttribute('name').split('/').slice(-1)[0],
 			vis = th[a].className.indexOf('min') === -1;
 
@@ -540,6 +540,12 @@ function get_np() {
 		rt[tk] = tv;
 	}
 	return [rt, rv, ra];
+}
+
+
+function get_np() {
+	var tr = QS('#files tr.play');
+	return ft2dict(tr);
 };
 
 
@@ -1277,11 +1283,12 @@ function play(tid, is_ev, seek, call_depth) {
 		}
 		else {
 			if (call_depth !== undefined)
-				return alert('failed to load ogv.js');
+				return toast.err(0, 'failed to load ogv.js:\ncannot play ogg/opus in this browser\n(try a non-apple device)');
 
-			show_modal('<h1>loading ogv.js</h1><h2>thanks apple</h2>');
+			toast.inf(0, '<h1>loading ogv.js</h1><h2>thanks apple</h2>');
 
 			import_js('/.cpr/deps/ogv.js', function () {
+				toast.hide();
 				play(tid, false, seek, 1);
 			});
 
@@ -1377,7 +1384,7 @@ function evau_error(e) {
 
 	err += '\n\nFile: «' + uricom_dec(eplaya.src.split('/').slice(-1)[0])[0] + '»';
 
-	alert(err);
+	toast.warn(15, err);
 }
 
 
@@ -1454,6 +1461,105 @@ function play_linked() {
 })();
 
 
+
+function fmt_ren(re, md, fmt) {
+	var ptr = 0;
+	function dive(stop_ch) {
+		var ret = '', ng = 0;
+		while (ptr < fmt.length) {
+			var dbg = fmt.slice(ptr),
+				ch = fmt[ptr++];
+
+			if (ch == '\\') {
+				ret += fmt[ptr++];
+				continue;
+			}
+
+			if (ch == ')' || ch == ']' || ch == stop_ch)
+				return [ng, ret];
+
+			if (ch == '[') {
+				var r2 = dive();
+				if (r2[0] == 0)
+					ret += r2[1];
+			}
+			else if (ch == '(') {
+				var end = fmt.indexOf(')', ptr);
+				if (end < 0)
+					throw 'the ( was never closed: ' + fmt.slice(0, ptr);
+
+				var arg = fmt.slice(ptr, end), v = null;
+				ptr = end + 1;
+
+				if (arg != parseInt(arg))
+					v = md[arg];
+				else {
+					arg = parseInt(arg);
+					if (arg >= re.length)
+						throw 'matching group ' + arg + ' exceeds ' + (re.length - 0);
+
+					v = re[arg];
+				}
+
+				if (v !== null && v !== undefined)
+					ret += v;
+				else
+					ng++;
+			}
+			else if (ch == '$') {
+				ch = fmt[ptr++];
+				var end = fmt.indexOf('(', ptr);
+				if (end < 0)
+					throw 'no function name after the $ here: ' + fmt.slice(0, ptr);
+
+				var fun = fmt.slice(ptr - 1, end);
+				ptr = end + 1;
+
+				if (fun == "lpad") {
+					var str = dive(',')[1];
+					var len = dive(',')[1];
+					var chr = dive()[1];
+					if (!len || !chr)
+						throw 'invalid arguments to ' + fun;
+
+					if (!str.length)
+						ng += 1;
+
+					while (str.length < len)
+						str = chr + str;
+
+					ret += str;
+				}
+				else if (fun == "rpad") {
+					var str = dive(',')[1];
+					var len = dive(',')[1];
+					var chr = dive()[1];
+					if (!len || !chr)
+						throw 'invalid arguments to ' + fun;
+
+					if (!str.length)
+						ng += 1;
+
+					while (str.length < len)
+						str += chr;
+
+					ret += str;
+				}
+				else throw 'function not implemented: "' + fun + '"';
+			}
+			else ret += ch;
+		}
+		return [ng, ret];
+	}
+	try {
+		return [true, dive()[1]];
+	}
+	catch (ex) {
+		return [false, ex];
+	}
+}
+
+
 var fileman = (function () {
 	var bren = ebi('fren'),
 		bdel = ebi('fdel'),
@@ -1462,7 +1568,10 @@ var fileman = (function () {
 		r = {};
 
 	r.clip = null;
-	r.bus = new BroadcastChannel("fileman_bus");
+	try {
+		r.bus = new BroadcastChannel("fileman_bus");
+	}
+	catch (ex) { }
 
 	r.render = function () {
 		if (r.clip === null)
@@ -1487,39 +1596,290 @@ var fileman = (function () {
 			return toast.err(3, 'cannot rename:\nyou do not have “move” permission in this folder');
 
 		var sel = msel.getsel();
-		if (sel.length !== 1)
-			return toast.err(3, 'select exactly 1 item to rename');
+		if (!sel.length)
+			return toast.err(3, 'select at least one item to rename');
 
-		var src = sel[0].vp;
-		if (src.endsWith('/'))
-			src = src.slice(0, -1);
+		var f = [],
+			base = vsplit(sel[0].vp)[0],
+			mkeys;
 
-		var vsp = vsplit(src),
-			base = vsp[0],
-			ofn = vsp[1];
+		for (var a = 0; a < sel.length; a++) {
+			var vp = sel[a].vp;
+			if (vp.endsWith('/'))
+				vp = vp.slice(0, -1);
 
-		var fn = prompt('new filename:', ofn);
-		if (!fn || fn == ofn)
-			return toast.warn(1, 'rename aborted');
+			var vsp = vsplit(vp);
+			if (base != vsp[0])
+				return toast.err(0, 'bug:\n' + base + '\n' + vsp[0]);
 
-		var dst = base + fn;
+			var vars = ft2dict(ebi(sel[a].id).closest('tr'));
+			mkeys = vars[1].concat(vars[2]);
 
-		function rename_cb() {
-			if (this.readyState != XMLHttpRequest.DONE)
-				return;
+			var md = vars[0];
+			for (var k in md) {
+				if (!md.hasOwnProperty(k))
+					continue;
 
-			if (this.status !== 200) {
-				var msg = this.responseText;
-				toast.err(9, 'rename failed:\n' + msg);
-				return;
+				md[k.toLowerCase()] = md[k];
+				k = k.toLowerCase();
+
+				if (k.startsWith('.'))
+					md[k.slice(1)] = md[k];
 			}
-			toast.ok(2, 'rename OK');
-			treectl.goto(get_evpath());
+			md.t = md.ext;
+			md.date = md.ts;
+			md.size = md.sz;
+
+			f.push({
+				"src": vp,
+				"ofn": uricom_dec(vsp[1])[0],
+				"md": vars[0],
+				"ok": true
+			});
+		}
+
+		var rui = ebi('rui');
+		if (!rui) {
+			rui = mknod('div');
+			rui.setAttribute('id', 'rui');
+			document.body.appendChild(rui);
+		}
+
+		var html = sel.length > 1 ? ['<div>'] : [
+			'<div>',
+			'<button class="rn_dec" n="0" tt="may fix some cases of broken filenames">url-decode</button>',
+			'//',
+			'<button class="rn_reset" n="0" tt="reset modified filenames back to the original ones">↺ reset</button>'
+		];
+
+		html = html.concat([
+			'<button id="rn_cancel" tt="abort and close this window">❌ cancel</button>',
+			'<button id="rn_apply">✅ apply rename</button>',
+			'<a id="rn_adv" class="tgl btn" href="#" tt="batch / metadata / pattern renaming">advanced</a>',
+			'<a id="rn_case" class="tgl btn" href="#" tt="case-sensitive regex">case</a>',
+			'</div>',
+			'<div id="rn_vadv"><table>',
+			'<tr><td>regex</td><td><input type="text" id="rn_re" tt="regex search pattern to apply to original filenames; capturing groups can be referenced in the format field below like &lt;code&gt;(1)&lt;/code&gt; and &lt;code&gt;(2)&lt;/code&gt; and so on" /></td></tr>',
+			'<tr><td>format</td><td><input type="text" id="rn_fmt" tt="inspired by foobar2000:$N&lt;code&gt;(title)&lt;/code&gt; is replaced by song title,$N&lt;code&gt;[(artist) - ](title)&lt;/code&gt; skips the first part if artist is blank$N&lt;code&gt;$lpad((tn),2,0)&lt;/code&gt; pads tracknumber to 2 digits" /></td></tr>',
+			'<tr><td>preset</td><td><select id="rn_pre"></select>',
+			'<button id="rn_pdel">❌ delete</button>',
+			'<button id="rn_pnew">💾 save as</button>',
+			'</td></tr>',
+			'</table></div>'
+		]);
+
+		if (sel.length == 1)
+			html.push(
+				'<div><table id="rn_f">\n' +
+				'<tr><td>old:</td><td><input type="text" id="rn_old" n="0" readonly /></td></tr>\n' +
+				'<tr><td>new:</td><td><input type="text" id="rn_new" n="0" /></td></tr>');
+		else {
+			html.push(
+				'<div><table id="rn_f" class="m">' +
+				'<tr><td></td><td>new name</td><td>old name</td></tr>');
+			for (var a = 0; a < f.length; a++)
+				html.push(
+					'<tr><td>' +
+					'<button class="rn_dec" n="' + a + '">decode</button>',
+					'<button class="rn_reset" n="' + a + '">↺ reset</button></td>',
+					'<td><input type="text" id="rn_new" n="' + a + '" /></td>' +
+					'<td><input type="text" id="rn_old" n="' + a + '" readonly /></td></tr>');
+		}
+		html.push('</table></div>');
+
+		if (sel.length == 1) {
+			html.push('<div><p style="margin:.6em 0">tags for the selected file (read-only, just for reference):</p><table>');
+			for (var a = 0; a < mkeys.length; a++)
+				html.push('<tr><td>' + esc(mkeys[a]) + '</td><td><input type="text" readonly value="' + esc(f[0].md[mkeys[a]]) + '" /></td></tr>');
+
+			html.push('</table></div>');
+		}
+
+		rui.innerHTML = html.join('\n');
+		for (var a = 0; a < f.length; a++) {
+			var k = '[n="' + a + '"]';
+			f[a].iold = QS('#rn_old' + k);
+			f[a].inew = QS('#rn_new' + k);
+			f[a].inew.value = f[a].iold.value = f[a].ofn;
+
+			(function (a) {
+				f[a].inew.onkeydown = function (e) {
+					rn_ok(a, true);
+
+					if (e.key == 'Escape')
+						return rn_cancel();
+
+					if (e.key == 'Enter')
+						return rn_apply();
+				};
+				QS('.rn_dec' + k).onclick = function () {
+					f[a].inew.value = uricom_dec(f[a].inew.value)[0];
+				};
+				QS('.rn_reset' + k).onclick = function () {
+					rn_reset(a);
+				};
+			})(a);
+		}
+		rn_reset(0);
+		tt.att(rui);
+
+		var adv = bcfg_get('rn_adv', false),
+			cs = bcfg_get('rn_case', false);
+
+		function sadv() {
+			ebi('rn_vadv').style.display = ebi('rn_case').style.display = adv ? '' : 'none';
+		}
+		sadv();
+
+		function rn_ok(n, ok) {
+			f[n].ok = ok;
+			clmod(f[n].inew.closest('tr'), 'err', !ok);
+		}
+
+		function rn_reset(n) {
+			f[n].inew.value = f[n].iold.value = f[n].ofn;
+			f[n].inew.focus();
+			f[n].inew.setSelectionRange(0, f[n].inew.value.lastIndexOf('.'), "forward");
+		}
+		function rn_cancel() {
+			rui.parentNode.removeChild(rui);
+		}
+
+		ebi('rn_cancel').onclick = rn_cancel;
+		ebi('rn_apply').onclick = rn_apply;
+		ebi('rn_adv').onclick = function () {
+			adv = !adv;
+			bcfg_set('rn_adv', adv);
+			sadv();
+		};
+		ebi('rn_case').onclick = function () {
+			cs = !cs;
+			bcfg_set('rn_case', cs);
+		};
+
+		var ire = ebi('rn_re'),
+			ifmt = ebi('rn_fmt'),
+			ipre = ebi('rn_pre'),
+			idel = ebi('rn_pdel'),
+			inew = ebi('rn_pnew'),
+			defp = '$lpad((tn),2,0). [(artist) - ](title).(ext)';
+
+		var presets = {};
+		presets[defp] = ['', defp];
+		presets = jread("rn_pre", presets);
+
+		function spresets() {
+			var keys = Object.keys(presets), o;
+			keys.sort();
+			ipre.innerHTML = '<option value=""></option>';
+			for (var a = 0; a < keys.length; a++) {
+				o = mknod('option');
+				o.setAttribute('value', keys[a]);
+				o.textContent = keys[a];
+				ipre.appendChild(o);
+			}
+		}
+		inew.onclick = function () {
+			modal.prompt('provide a name for your new preset', ifmt.value, function (name) {
+				if (!name)
+					return toast.warn(3, 'aborted');
+
+				presets[name] = [ire.value, ifmt.value];
+				jwrite('rn_pre', presets);
+				spresets();
+				ipre.value = name;
+			});
+		};
+		idel.onclick = function () {
+			delete presets[ipre.value];
+			jwrite('rn_pre', presets);
+			spresets();
+		};
+		ipre.oninput = function () {
+			var cfg = presets[ipre.value];
+			if (cfg) {
+				ire.value = cfg[0];
+				ifmt.value = cfg[1];
+			}
+			ifmt.oninput();
+		};
+		spresets();
+
+		ire.onkeydown = ifmt.onkeydown = function (e) {
+			if (e.key == 'Escape')
+				return rn_cancel();
+
+			if (e.key == 'Enter')
+				return rn_apply();
+		};
+
+		ire.oninput = ifmt.oninput = function (e) {
+			var ptn = ire.value,
+				fmt = ifmt.value,
+				re = null;
+
+			if (!fmt)
+				return;
+
+			try {
+				if (ptn)
+					re = new RegExp(ptn, cs ? 'i' : '');
+			}
+			catch (ex) {
+				return toast.err(5, 'invalid regex:\n' + ex);
+			}
+			toast.hide();
+
+			for (var a = 0; a < f.length; a++) {
+				var m = re ? re.exec(f[a].ofn) : null,
+					ok, txt = '';
+
+				if (re && !m) {
+					txt = 'regex did not match';
+					ok = false;
+				}
+				else {
+					var ret = fmt_ren(m, f[a].md, fmt);
+					ok = ret[0];
+					txt = ret[1];
+				}
+				rn_ok(a, ok);
+				f[a].inew.value = (ok ? '' : 'ERROR: ') + txt;
+			}
+		};
+
+		function rn_apply() {
+			while (f.length && (!f[0].ok || f[0].ofn == f[0].inew.value))
+				f.shift();
+
+			if (!f.length) {
+				toast.ok(2, 'rename OK');
+				treectl.goto(get_evpath());
+				return rn_cancel();
+			}
+
+			toast.inf(0, 'renaming ' + f.length + ' items\n\n' + f[0].ofn);
+			var dst = base + uricom_enc(f[0].inew.value, false);
+
+			function rename_cb() {
+				if (this.readyState != XMLHttpRequest.DONE)
+					return;
+
+				if (this.status !== 200) {
+					var msg = this.responseText;
+					toast.err(9, 'rename failed:\n' + msg);
+					return;
+				}
+
+				f.shift().inew.value = '( OK )';
+				return rn_apply();
+			}
+
+			var xhr = new XMLHttpRequest();
+			xhr.open('GET', f[0].src + '?move=' + dst, true);
+			xhr.onreadystatechange = rename_cb;
+			xhr.send();
 		}
-		var xhr = new XMLHttpRequest();
-		xhr.open('GET', src + '?move=' + dst, true);
-		xhr.onreadystatechange = rename_cb;
-		xhr.send();
 	};
 
 	r.delete = function (e) {
@@ -1536,12 +1896,6 @@ var fileman = (function () {
 		if (!sel.length)
 			return toast.err(3, 'select at least 1 item to delete');
 
-		if (!confirm('===== DANGER =====\nDELETE these ' + vps.length + ' items?\n\n' + vps.join('\n')))
-			return;
-
-		if (!confirm('Last chance! Delete?'))
-			return;
-
 		function deleter() {
 			var xhr = new XMLHttpRequest(),
 				vp = vps.shift();
@@ -1568,7 +1922,10 @@ var fileman = (function () {
 			}
 			deleter();
 		}
-		deleter();
+
+		modal.confirm('===== DANGER =====\nDELETE these ' + vps.length + ' items?\n\n' + uricom_adec(vps).join('\n'), function () {
+			modal.confirm('Last chance! Delete?', deleter, null);
+		}, null);
 	};
 
 	r.cut = function (e) {
@@ -1582,14 +1939,15 @@ var fileman = (function () {
 		if (!sel.length)
 			return toast.err(3, 'select at least 1 item to cut');
 
-		for (var a = 0; a < sel.length; a++) {
-			vps.push(sel[a].vp);
-			var cl = ebi(sel[a].id).closest('tr').classList,
-				inv = cl.contains('c1');
+		if (sel.length < 100)
+			for (var a = 0; a < sel.length; a++) {
+				vps.push(sel[a].vp);
+				var cl = ebi(sel[a].id).closest('tr').classList,
+					inv = cl.contains('c1');
 
-			cl.remove(inv ? 'c1' : 'c2');
-			cl.add(inv ? 'c2' : 'c1');
-		}
+				cl.remove(inv ? 'c1' : 'c2');
+				cl.add(inv ? 'c2' : 'c1');
+			}
 
 		toast.inf(1, 'cut ' + sel.length + ' items');
 		jwrite('fman_clip', vps);
@@ -1611,7 +1969,7 @@ var fileman = (function () {
 			links = QSA('#files tbody td:nth-child(2) a');
 
 		for (var a = 0, aa = links.length; a < aa; a++)
-			indir.push(links[a].getAttribute('name'));
+			indir.push(vsplit(links[a].getAttribute('href'))[1]);
 
 		for (var a = 0; a < r.clip.length; a++) {
 			var found = false;
@@ -1626,14 +1984,11 @@ var fileman = (function () {
 		}
 
 		if (exists.length)
-			alert('these ' + exists.length + ' items cannot be pasted here (names already exist):\n\n' + exists.join('\n'));
+			toast.warn(30, 'these ' + exists.length + ' items cannot be pasted here (names already exist):\n\n' + uricom_adec(exists).join('\n'));
 
 		if (!req.length)
 			return;
 
-		if (!confirm('paste these ' + req.length + ' items here?\n\n' + req.join('\n')))
-			return;
-
 		function paster() {
 			var xhr = new XMLHttpRequest(),
 				vp = req.shift();
@@ -1644,7 +1999,7 @@ var fileman = (function () {
 				r.tx(srcdir);
 				return;
 			}
-			toast.inf(0, 'pasting ' + (req.length + 1) + ' items\n\n' + vp);
+			toast.inf(0, 'pasting ' + (req.length + 1) + ' items\n\n' + uricom_dec(vp)[0]);
 
 			var dst = get_evpath() + vp.split('/').slice(-1)[0];
 
@@ -1663,20 +2018,26 @@ var fileman = (function () {
 			}
 			paster();
 		}
-		paster();
 
-		jwrite('fman_clip', []);
+		modal.confirm('paste these ' + req.length + ' items here?\n\n' + uricom_adec(req).join('\n'), function () {
+			paster();
+			jwrite('fman_clip', []);
+		}, null);
 	};
 
-	r.bus.onmessage = function (e) {
-		r.clip = null;
-		r.render();
-		var me = get_evpath();
-		if (e && e.data == me)
-			treectl.goto(e.data);
-	};
+	if (r.bus)
+		r.bus.onmessage = function (e) {
+			r.clip = null;
+			r.render();
+			var me = get_evpath();
+			if (e && e.data == me)
+				treectl.goto(e.data);
+		};
 
 	r.tx = function (msg) {
+		if (!r.bus)
+			return;
+
 		r.bus.postMessage(msg);
 		r.bus.onmessage();
 	};
@@ -2415,6 +2776,7 @@ var treectl = (function () {
 		prev_atop = null,
 		prev_winh = null,
 		dyn = bcfg_get('dyntree', true),
+		dots = bcfg_get('dotfiles', false),
 		treesz = icfg_get('treesz', 16);
 
 	treesz = Math.min(Math.max(treesz, 4), 50);
@@ -2533,7 +2895,7 @@ var treectl = (function () {
 		xhr.dst = dst;
 		xhr.rst = rst;
 		xhr.ts = Date.now();
-		xhr.open('GET', dst + '?tree=' + top, true);
+		xhr.open('GET', dst + '?tree=' + top + (dots ? '&dots' : ''), true);
 		xhr.onreadystatechange = recvtree;
 		xhr.send();
 		enspin('#tree');
@@ -2637,7 +2999,7 @@ var treectl = (function () {
 		xhr.top = url;
 		xhr.hpush = hpush;
 		xhr.ts = Date.now();
-		xhr.open('GET', xhr.top + '?ls', true);
+		xhr.open('GET', xhr.top + '?ls' + (dots ? '&dots' : ''), true);
 		xhr.onreadystatechange = recvls;
 		xhr.send();
 		if (hpush)
@@ -2774,6 +3136,13 @@ var treectl = (function () {
 		return ret;
 	}
 
+	function tdots(e) {
+		ev(e);
+		dots = !dots;
+		bcfg_set('dotfiles', dots);
+		treectl.goto(get_evpath());
+	}
+
 	function dyntree(e) {
 		ev(e);
 		dyn = !dyn;
@@ -2793,6 +3162,7 @@ var treectl = (function () {
 
 	ebi('entree').onclick = treectl.entree;
 	ebi('detree').onclick = treectl.detree;
+	ebi('dotfiles').onclick = tdots;
 	ebi('dyntree').onclick = dyntree;
 	ebi('twig').onclick = scaletree;
 	ebi('twobytwo').onclick = scaletree;
@@ -2839,7 +3209,7 @@ function apply_perms(newperms) {
 
 	var axs = [],
 		aclass = '>',
-		chk = ['read', 'write', 'rename', 'delete'];
+		chk = ['read', 'write', 'move', 'delete'];
 
 	for (var a = 0; a < chk.length; a++)
 		if (has(perms, chk[a]))
@@ -2955,6 +3325,8 @@ var filecols = (function () {
 			"q": "quality / bitrate",
 			"Ac": "audio codec",
 			"Vc": "video codec",
+			"Ahash": "audio checksum",
+			"Vhash": "video checksum",
 			"Res": "resolution",
 			"T": "filetype",
 			"aq": "audio quality / bitrate",
@@ -2966,6 +3338,21 @@ var filecols = (function () {
 			"hz": "sample rate"
 		};
 
+	if (JSON.stringify(def_hcols) != sread('hfilecols')) {
+		console.log("applying default hidden-cols");
+		jwrite('hfilecols', def_hcols);
+		for (var a = 0; a < def_hcols.length; a++) {
+			var t = def_hcols[a];
+			t = t.slice(0, 1).toUpperCase() + t.slice(1);
+			if (t.startsWith("."))
+				t = t.slice(1);
+
+			if (hidden.indexOf(t) == -1)
+				hidden.push(t);
+		}
+		jwrite("filecols", hidden);
+	}
+
 	var add_btns = function () {
 		var ths = QSA('#files th>span');
 		for (var a = 0, aa = ths.length; a < aa; a++) {
@@ -3308,6 +3695,16 @@ var msel = (function () {
 			return;
 
 		r.sel = [];
+		if (r.all && r.all.length) {
+			for (var a = 0; a < r.all.length; a++) {
+				var ao = r.all[a];
+				ao.sel = ebi(ao.id).closest('tr').classList.contains('sel');
+				if (ao.sel)
+					r.sel.push(ao);
+			}
+			return;
+		}
+
 		r.all = [];
 		var links = QSA('#files tbody td:nth-child(2) a:last-child'),
 			vbase = get_evpath();
@@ -3319,13 +3716,11 @@ var msel = (function () {
 			item.id = links[a].getAttribute('id');
 			item.sel = links[a].closest('tr').classList.contains('sel');
 			item.vp = href.indexOf('/') !== -1 ? href : vbase + href;
-			item.name = href.split('/').slice(-1);
 
 			r.all.push(item);
 			if (item.sel)
 				r.sel.push(item);
 
-			links[a].setAttribute('name', item.name);
 			links[a].closest('tr').setAttribute('tabindex', '0');
 		}
 	};
@@ -3338,8 +3733,11 @@ var msel = (function () {
 		r.load();
 		return r.all;
 	};
-	r.selui = function () {
-		r.sel = r.all = null;
+	r.selui = function (reset) {
+		r.sel = null;
+		if (reset)
+			r.all = null;
+
 		clmod(ebi('wtoggle'), 'sel', r.getsel().length);
 		thegrid.loadsel();
 		fileman.render();
@@ -3365,13 +3763,13 @@ var msel = (function () {
 	};
 	ebi('selzip').onclick = function (e) {
 		ev(e);
-		var names = r.getsel(),
+		var sel = r.getsel(),
 			arg = ebi('selzip').getAttribute('fmt'),
 			frm = mknod('form'),
 			txt = [];
 
-		for (var a = 0; a < names.length; a++)
-			txt.push(names[a].name);
+		for (var a = 0; a < sel.length; a++)
+			txt.push(vsplit(sel[a].vp)[1]);
 
 		txt = txt.join('\n');
 
@@ -3398,7 +3796,7 @@ var msel = (function () {
 		for (var a = 0, aa = tds.length; a < aa; a++) {
 			tds[a].onclick = r.seltgl;
 		}
-		r.selui();
+		r.selui(true);
 		arcfmt.render();
 		fileman.render();
 		ebi('selzip').style.display = ebi('unsearch') ? 'none' : '';

copyparty/web/md.css

@@ -8,140 +8,6 @@ html, body {
 	font-family: sans-serif;
 	line-height: 1.5em;
 }
-
-
-
-#tt, #toast {
-	position: fixed;
-	max-width: 34em;
-	background: #222;
-	border: 0 solid #777;
-	box-shadow: 0 .2em .5em #222;
-	border-radius: .4em;
-	z-index: 9001;
-}
-#tt {
-	overflow: hidden;
-	margin-top: 1em;
-	padding: 0 1.3em;
-	height: 0;
-	opacity: .1;
-	transition: opacity 0.14s, height 0.14s, padding 0.14s;
-}
-#toast {
-	top: 1.4em;
-	right: -1em;
-	line-height: 1.5em;
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	transform: translateX(100%);
-	transition:
-		transform .4s cubic-bezier(.2, 1.2, .5, 1),
-		right .4s cubic-bezier(.2, 1.2, .5, 1);
-	text-shadow: 1px 1px 0 #000;
-	color: #fff;
-}
-#toast pre {
-	margin: 0;
-}
-#toastc {
-	display: inline-block;
-	position: absolute;
-	overflow: hidden;
-	left: 0;
-	width: 0;
-	opacity: 0;
-	padding: .3em 0;
-	margin: -.3em 0 0 0;
-	line-height: 1.5em;
-	color: #000;
-	border: none;
-	outline: none;
-	text-shadow: none;
-	border-radius: .5em 0 0 .5em;
-	transition: left .3s, width .3s, padding .3s, opacity .3s;
-}
-#toast.vis {
-	right: 1.3em;
-	transform: unset;
-}
-#toast.vis #toastc {
-	left: -2em;
-	width: .4em;
-	padding: .3em .8em;
-	opacity: 1;
-}
-#toast.inf {
-	background: #07a;
-	border-color: #0be;
-}
-#toast.inf #toastc {
-	background: #0be;
-}
-#toast.ok {
-	background: #4a0;
-	border-color: #8e4;
-}
-#toast.ok #toastc {
-	background: #8e4;
-}
-#toast.warn {
-	background: #970;
-	border-color: #fc0;
-}
-#toast.warn #toastc {
-	background: #fc0;
-}
-#toast.err {
-	background: #900;
-	border-color: #d06;
-}
-#toast.err #toastc {
-	background: #d06;
-}
-#tt.b {
-	padding: 0 2em;
-	border-radius: .5em;
-	box-shadow: 0 .2em 1em #000;
-}
-#tt.show {
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	height: auto;
-	opacity: 1;
-}
-#tt.show.b {
-	padding: 1.5em 2em;
-	border-width: .5em 0;
-}
-#tt code {
-	background: #3c3c3c;
-	padding: .1em .3em;
-	border-top: 1px solid #777;
-	border-radius: .3em;
-	line-height: 1.7em;
-}
-#tt em {
-	color: #f6a;
-}
-html.light #tt {
-	background: #fff;
-	border-color: #888 #000 #777 #000;
-}
-html.light #tt,
-html.light #toast {
-	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
-}
-html.light #tt code {
-	background: #060;
-	color: #fff;
-}
-html.light #tt em {
-	color: #d38;
-}
-
-
-
 #mtw {
 	display: none;
 }
@@ -149,6 +15,10 @@ html.light #tt em {
 	margin: 0 auto;
 	padding: 0 1.5em;
 }
+#toast {
+	bottom: auto;
+	top: 1.4em;
+}
 pre, code, a {
 	color: #480;
 	background: #f7f7f7;

copyparty/web/md.html

@@ -1,11 +1,12 @@
 <!DOCTYPE html><html><head>
 	<meta charset="utf-8">
-	<title>📝🎉 {{ title }}</title> <!-- 📜 -->
+	<title>📝🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/md.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/md.css?_={{ ts }}">
 	{%- if edit %}
-	<link href="/.cpr/md2.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/md2.css?_={{ ts }}">
 	{%- endif %}
 </head>
 <body>

copyparty/web/md.js

@@ -185,7 +185,7 @@ function md_plug_err(ex, js) {
     errbox.style.cssText = 'position:absolute;top:0;left:0;padding:1em .5em;background:#2b2b2b;color:#fc5'
     errbox.textContent = msg;
     errbox.onclick = function () {
-        alert('' + ex.stack);
+        modal.alert('<pre>' + ex.stack + '</pre>');
     };
     if (o) {
         errbox.appendChild(o);

copyparty/web/md2.js

@@ -326,26 +326,32 @@ function save(e) {
         return toast.inf(2, "no changes");
 
     var force = (save_cls.indexOf('force-save') >= 0);
-    if (force && !confirm('confirm that you wish to lose the changes made on the server since you opened this document'))
-        return toast.inf(3, 'aborted');
+    function save2() {
+        var txt = dom_src.value,
+            fd = new FormData();
 
-    var txt = dom_src.value;
+        fd.append("act", "tput");
+        fd.append("lastmod", (force ? -1 : last_modified));
+        fd.append("body", txt);
 
-    var fd = new FormData();
-    fd.append("act", "tput");
-    fd.append("lastmod", (force ? -1 : last_modified));
-    fd.append("body", txt);
+        var url = (document.location + '').split('?')[0];
+        var xhr = new XMLHttpRequest();
+        xhr.open('POST', url, true);
+        xhr.responseType = 'text';
+        xhr.onreadystatechange = save_cb;
+        xhr.btn = save_btn;
+        xhr.txt = txt;
 
-    var url = (document.location + '').split('?')[0];
-    var xhr = new XMLHttpRequest();
-    xhr.open('POST', url, true);
-    xhr.responseType = 'text';
-    xhr.onreadystatechange = save_cb;
-    xhr.btn = save_btn;
-    xhr.txt = txt;
+        modpoll.skip_one = true;  // skip one iteration while we save
+        xhr.send(fd);
+    }
 
-    modpoll.skip_one = true;  // skip one iteration while we save
-    xhr.send(fd);
+    if (!force)
+        save2();
+    else
+        modal.confirm('confirm that you wish to lose the changes made on the server since you opened this document', save2, function () {
+            toast.inf(3, 'aborted');
+        });
 }
 
 function save_cb() {
@@ -353,14 +359,14 @@ function save_cb() {
         return;
 
     if (this.status !== 200)
-        return alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var r;
     try {
         r = JSON.parse(this.responseText);
     }
     catch (ex) {
-        return alert('Failed to parse reply from server:\n\n' + this.responseText);
+        return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
     }
 
     if (!r.ok) {
@@ -375,12 +381,10 @@ function save_cb() {
                 r.lastmod + ' lastmod on the server now,',
                 r.now + ' server time now,\n',
             ];
-            alert(msg.join('\n'));
+            return toast.err(0, msg.join('\n'));
         }
-        else {
-            alert('Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
-        }
-        return;
+        else
+            return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
     }
 
     this.btn.classList.remove('force-save');
@@ -407,10 +411,8 @@ function savechk_cb() {
     if (this.readyState != XMLHttpRequest.DONE)
         return;
 
-    if (this.status !== 200) {
-        alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
-        return;
-    }
+    if (this.status !== 200)
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var doc1 = this.txt.replace(/\r\n/g, "\n");
     var doc2 = this.responseText.replace(/\r\n/g, "\n");
@@ -423,12 +425,12 @@ function savechk_cb() {
             }, 100);
             return;
         }
-        alert(
+        modal.alert(
             'Error! The document on the server does not appear to have saved correctly (your editor contents and the server copy is not identical). Place the document on your clipboard for now and check the server logs for hints\n\n' +
             'Length: yours=' + doc1.length + ', server=' + doc2.length
         );
-        alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
-        alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
+        modal.alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
+        modal.alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
         return;
     }
 
@@ -865,12 +867,10 @@ function iter_uni(e) {
 function cfg_uni(e) {
     if (e) e.preventDefault();
 
-    var reply = prompt("unicode whitelist", esc_uni_whitelist);
-    if (reply === null)
-        return;
-
-    esc_uni_whitelist = reply;
-    js_uni_whitelist = eval('\'' + esc_uni_whitelist + '\'');
+    modal.prompt("unicode whitelist", esc_uni_whitelist, function (reply) {
+        esc_uni_whitelist = reply;
+        js_uni_whitelist = eval('\'' + esc_uni_whitelist + '\'');
+    }, null);
 }
 
 

copyparty/web/mde.css

@@ -18,6 +18,10 @@ html, body {
 	background: #f7f7f7;
 	color: #333;
 }
+#toast {
+	bottom: auto;
+	top: 1.4em;
+}
 #mn {
 	font-weight: normal;
 	margin: 1.3em 0 .7em 1em;

copyparty/web/mde.html

@@ -3,9 +3,10 @@
 	<title>📝🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/mde.css?_={{ ts }}" rel="stylesheet">
-	<link href="/.cpr/deps/mini-fa.css?_={{ ts }}" rel="stylesheet">
-	<link href="/.cpr/deps/easymde.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/mde.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/deps/mini-fa.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/deps/easymde.css?_={{ ts }}">
 </head>
 <body>
 	<div id="mw">

copyparty/web/mde.js

@@ -110,25 +110,31 @@ function save(mde) {
         return toast.inf(2, 'no changes');
 
     var force = save_btn.classList.contains('force-save');
-    if (force && !confirm('confirm that you wish to lose the changes made on the server since you opened this document'))
-        return toast.inf(3, 'aborted');
+    function save2() {
+        var txt = mde.value();
 
-    var txt = mde.value();
+        var fd = new FormData();
+        fd.append("act", "tput");
+        fd.append("lastmod", (force ? -1 : last_modified));
+        fd.append("body", txt);
 
-    var fd = new FormData();
-    fd.append("act", "tput");
-    fd.append("lastmod", (force ? -1 : last_modified));
-    fd.append("body", txt);
+        var url = (document.location + '').split('?')[0];
+        var xhr = new XMLHttpRequest();
+        xhr.open('POST', url, true);
+        xhr.responseType = 'text';
+        xhr.onreadystatechange = save_cb;
+        xhr.btn = save_btn;
+        xhr.mde = mde;
+        xhr.txt = txt;
+        xhr.send(fd);
+    }
 
-    var url = (document.location + '').split('?')[0];
-    var xhr = new XMLHttpRequest();
-    xhr.open('POST', url, true);
-    xhr.responseType = 'text';
-    xhr.onreadystatechange = save_cb;
-    xhr.btn = save_btn;
-    xhr.mde = mde;
-    xhr.txt = txt;
-    xhr.send(fd);
+    if (!force)
+        save2();
+    else
+        modal.confirm('confirm that you wish to lose the changes made on the server since you opened this document', save2, function () {
+            toast.inf(3, 'aborted');
+        });
 }
 
 function save_cb() {
@@ -136,14 +142,14 @@ function save_cb() {
         return;
 
     if (this.status !== 200)
-        return alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var r;
     try {
         r = JSON.parse(this.responseText);
     }
     catch (ex) {
-        return alert('Failed to parse reply from server:\n\n' + this.responseText);
+        return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
     }
 
     if (!r.ok) {
@@ -158,12 +164,10 @@ function save_cb() {
                 r.lastmod + ' lastmod on the server now,',
                 r.now + ' server time now,\n',
             ];
-            alert(msg.join('\n'));
+            return toast.err(0, msg.join('\n'));
         }
-        else {
-            alert('Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
-        }
-        return;
+        else
+            return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
     }
 
     this.btn.classList.remove('force-save');
@@ -186,35 +190,23 @@ function save_chk() {
     if (this.readyState != XMLHttpRequest.DONE)
         return;
 
-    if (this.status !== 200) {
-        alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
-        return;
-    }
+    if (this.status !== 200)
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var doc1 = this.txt.replace(/\r\n/g, "\n");
     var doc2 = this.responseText.replace(/\r\n/g, "\n");
     if (doc1 != doc2) {
-        alert(
+        modal.alert(
             'Error! The document on the server does not appear to have saved correctly (your editor contents and the server copy is not identical). Place the document on your clipboard for now and check the server logs for hints\n\n' +
             'Length: yours=' + doc1.length + ', server=' + doc2.length
         );
-        alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
-        alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
+        modal.alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
+        modal.alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
         return;
     }
 
     last_modified = this.lastmod;
     md_changed(this.mde, true);
 
-    var ok = mknod('div');
-    ok.setAttribute('style', 'font-size:6em;font-family:serif;font-weight:bold;color:#cf6;background:#444;border-radius:.3em;padding:.6em 0;position:fixed;top:30%;left:calc(50% - 2em);width:4em;text-align:center;z-index:9001;transition:opacity 0.2s ease-in-out;opacity:1');
-    ok.innerHTML = 'OK✔️';
-    var parent = ebi('m');
-    document.documentElement.appendChild(ok);
-    setTimeout(function () {
-        ok.style.opacity = 0;
-    }, 500);
-    setTimeout(function () {
-        ok.parentNode.removeChild(ok);
-    }, 750);
+    toast.ok(2, 'save OK' + (this.ntry ? '\nattempt ' + this.ntry : ''));
 }

copyparty/web/msg.css

@@ -16,9 +16,6 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 #box {
 	padding: .5em 1em;
 	background: #2c2c2c;

copyparty/web/msg.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/msg.css?_={{ ts }}">
+    <link rel="stylesheet" media="screen" href="/.cpr/msg.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/splash.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/splash.css?_={{ ts }}">
+    <link rel="stylesheet" media="screen" href="/.cpr/splash.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/ui.css

@@ -0,0 +1,199 @@
+#tt, #toast {
+	position: fixed;
+	max-width: 34em;
+	background: #222;
+	border: 0 solid #777;
+	box-shadow: 0 .2em .5em #222;
+	border-radius: .4em;
+	z-index: 9001;
+}
+#tt {
+	overflow: hidden;
+	margin-top: 1em;
+	padding: 0 1.3em;
+	height: 0;
+	opacity: .1;
+	transition: opacity 0.14s, height 0.14s, padding 0.14s;
+}
+#toast {
+	bottom: 5em;
+	right: -1em;
+	line-height: 1.5em;
+	padding: 1em 1.3em;
+	border-width: .4em 0;
+	transform: translateX(100%);
+	transition:
+		transform .4s cubic-bezier(.2, 1.2, .5, 1),
+		right .4s cubic-bezier(.2, 1.2, .5, 1);
+	text-shadow: 1px 1px 0 #000;
+	color: #fff;
+}
+#toastc {
+	display: inline-block;
+	position: absolute;
+	overflow: hidden;
+	left: 0;
+	width: 0;
+	opacity: 0;
+	padding: .3em 0;
+	margin: -.3em 0 0 0;
+	line-height: 1.5em;
+	color: #000;
+	border: none;
+	outline: none;
+	text-shadow: none;
+	border-radius: .5em 0 0 .5em;
+	transition: left .3s, width .3s, padding .3s, opacity .3s;
+}
+#toast pre {
+	margin: 0;
+}
+#toast.vis {
+	right: 1.3em;
+	transform: unset;
+}
+#toast.vis #toastc {
+	left: -2em;
+	width: .4em;
+	padding: .3em .8em;
+	opacity: 1;
+}
+#toast.inf {
+	background: #07a;
+	border-color: #0be;
+}
+#toast.inf #toastc {
+	background: #0be;
+}
+#toast.ok {
+	background: #380;
+	border-color: #8e4;
+}
+#toast.ok #toastc {
+	background: #8e4;
+}
+#toast.warn {
+	background: #970;
+	border-color: #fc0;
+}
+#toast.warn #toastc {
+	background: #fc0;
+}
+#toast.err {
+	background: #900;
+	border-color: #d06;
+}
+#toast.err #toastc {
+	background: #d06;
+}
+#tt.b {
+	padding: 0 2em;
+	border-radius: .5em;
+	box-shadow: 0 .2em 1em #000;
+}
+#tt.show {
+	padding: 1em 1.3em;
+	border-width: .4em 0;
+	height: auto;
+	opacity: 1;
+}
+#tt.show.b {
+	padding: 1.5em 2em;
+	border-width: .5em 0;
+}
+#tt code {
+	background: #3c3c3c;
+	padding: .1em .3em;
+	border-top: 1px solid #777;
+	border-radius: .3em;
+	line-height: 1.7em;
+}
+#tt em {
+	color: #f6a;
+}
+html.light #tt {
+	background: #fff;
+	border-color: #888 #000 #777 #000;
+}
+html.light #tt,
+html.light #toast {
+	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
+}
+html.light #tt code {
+	background: #060;
+	color: #fff;
+}
+html.light #tt em {
+	color: #d38;
+}
+#modal {
+	position: fixed;
+    overflow: auto;
+	top: 0;
+	left: 0;
+	right: 0;
+	bottom: 0;
+	width: 100%;
+	height: 100%;
+	z-index: 9001;
+	background: rgba(64,64,64,0.6);
+}
+#modal>table {
+    width: 100%;
+    height: 100%;
+}
+#modal td {
+	text-align: center;
+}
+#modalc {
+	display: inline-block;
+	background: #f7f7f7;
+	color: #333;
+	text-shadow: none;
+	text-align: left;
+	margin: 3em;
+	padding: 1em 1.1em;
+	border-radius: .6em;
+    box-shadow: 0 .3em 3em rgba(0,0,0,0.5);
+	max-width: 50em;
+	max-height: 30em;
+	overflow: auto;
+}
+@media (min-width: 40em) {
+    #modalc {
+        min-width: 30em;
+    }
+}
+#modalb {
+	text-align: right;
+	padding-top: 1em;
+}
+#modalb a {
+	color: #000;
+	background: #ccc;
+	display: inline-block;
+	border-radius: .3em;
+	padding: .5em 1em;
+	outline: none;
+	border: none;
+}
+#modalb a:focus,
+#modalb a:hover {
+	background: #06d;
+	color: #fff;
+}
+#modalb a+a {
+	margin-left: .5em;
+}
+#modali {
+	display: block;
+	width: calc(100% - 1.25em);
+    margin: 1em -.1em 0 -.1em;
+	padding: .5em;
+	outline: none;
+	border: .25em solid #ccc;
+	border-radius: .4em;
+}
+#modali:focus {
+	border-color: #06d;
+}

copyparty/web/up2k.js

@@ -97,13 +97,11 @@ function up2k_flagbus() {
         }
     };
     var do_take = function (now) {
-        //dbg('*', 'do_take');
         tx(now, "have");
         flag.owner = [flag.id, now];
         flag.ours = true;
     };
     var do_want = function (now) {
-        //dbg('*', 'do_want');
         tx(now, "want");
     };
     flag.take = function (now) {
@@ -135,15 +133,16 @@ function up2k_flagbus() {
 
 
 function U2pvis(act, btns) {
-    this.act = act;
-    this.ctr = { "ok": 0, "ng": 0, "bz": 0, "q": 0 };
-    this.tab = [];
-    this.head = 0;
-    this.tail = -1;
-    this.wsz = 3;
+    var r = this;
+    r.act = act;
+    r.ctr = { "ok": 0, "ng": 0, "bz": 0, "q": 0 };
+    r.tab = [];
+    r.head = 0;
+    r.tail = -1;
+    r.wsz = 3;
 
-    this.addfile = function (entry, sz, draw) {
-        this.tab.push({
+    r.addfile = function (entry, sz, draw) {
+        r.tab.push({
             "hn": entry[0],
             "ht": entry[1],
             "hp": entry[2],
@@ -155,34 +154,34 @@ function U2pvis(act, btns) {
             "bd": 0,  // bytes done
             "bd0": 0  // upload start
         });
-        this.ctr["q"]++;
+        r.ctr["q"]++;
         if (!draw)
             return;
 
-        this.drawcard("q");
-        if (this.act == "q") {
-            this.addrow(this.tab.length - 1);
+        r.drawcard("q");
+        if (r.act == "q") {
+            r.addrow(r.tab.length - 1);
         }
-        if (this.act == "bz") {
-            this.bzw();
+        if (r.act == "bz") {
+            r.bzw();
         }
     };
 
-    this.is_act = function (card) {
-        if (this.act == "done")
+    r.is_act = function (card) {
+        if (r.act == "done")
             return card == "ok" || card == "ng";
 
-        return this.act == card;
+        return r.act == card;
     }
 
-    this.seth = function (nfile, field, html) {
-        var fo = this.tab[nfile];
+    r.seth = function (nfile, field, html) {
+        var fo = r.tab[nfile];
         field = ['hn', 'ht', 'hp'][field];
         if (fo[field] === html)
             return;
 
         fo[field] = html;
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}{1}'.format(nfile, field.slice(1)));
@@ -193,26 +192,26 @@ function U2pvis(act, btns) {
         }
     };
 
-    this.setab = function (nfile, nblocks) {
+    r.setab = function (nfile, nblocks) {
         var t = [];
         for (var a = 0; a < nblocks; a++)
             t.push(0);
 
-        this.tab[nfile].cb = t;
+        r.tab[nfile].cb = t;
     };
 
-    this.setat = function (nfile, blocktab) {
-        this.tab[nfile].cb = blocktab;
+    r.setat = function (nfile, blocktab) {
+        r.tab[nfile].cb = blocktab;
 
         var bd = 0;
         for (var a = 0; a < blocktab.length; a++)
             bd += blocktab[a];
 
-        this.tab[nfile].bd = bd;
-        this.tab[nfile].bd0 = bd;
+        r.tab[nfile].bd = bd;
+        r.tab[nfile].bd0 = bd;
     };
 
-    this.perc = function (bd, bd0, sz, t0) {
+    r.perc = function (bd, bd0, sz, t0) {
         var td = Date.now() - t0,
             p = bd * 100.0 / sz,
             nb = bd - bd0,
@@ -222,15 +221,15 @@ function U2pvis(act, btns) {
         return [p, s2ms(eta), spd / (1024 * 1024)];
     };
 
-    this.hashed = function (fobj) {
-        var fo = this.tab[fobj.n],
+    r.hashed = function (fobj) {
+        var fo = r.tab[fobj.n],
             nb = fo.bt * (++fo.nh / fo.cb.length),
-            p = this.perc(nb, 0, fobj.size, fobj.t_hashing);
+            p = r.perc(nb, 0, fobj.size, fobj.t_hashing);
 
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
             p[0].toFixed(2), p[1], p[2].toFixed(2)
         );
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}p'.format(fobj.n)),
@@ -241,31 +240,31 @@ function U2pvis(act, btns) {
         obj.style.background = 'linear-gradient(90deg, #025, #06a ' + o1 + '%, #09d ' + o2 + '%, #333 ' + o3 + '%, #333 99%, #777)';
     };
 
-    this.prog = function (fobj, nchunk, cbd) {
-        var fo = this.tab[fobj.n],
+    r.prog = function (fobj, nchunk, cbd) {
+        var fo = r.tab[fobj.n],
             delta = cbd - fo.cb[nchunk];
 
         fo.cb[nchunk] = cbd;
         fo.bd += delta;
 
-        var p = this.perc(fo.bd, fo.bd0, fo.bt, fobj.t_uploading);
+        var p = r.perc(fo.bd, fo.bd0, fo.bt, fobj.t_uploading);
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
             p[0].toFixed(2), p[1], p[2].toFixed(2)
         );
 
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}p'.format(fobj.n)),
             o1 = p[0] - 2, o2 = p[0] - 0.1, o3 = p[0];
 
-        if (!obj) { //} || true) {
+        if (!obj) {
             var msg = [
-                "act", this.act,
+                "act", r.act,
                 "in", fo.in,
-                "is_act", this.is_act(fo.in),
-                "head", this.head,
-                "tail", this.tail,
+                "is_act", r.is_act(fo.in),
+                "head", r.head,
+                "tail", r.tail,
                 "nfile", fobj.n,
                 "name", fobj.name,
                 "sz", fobj.size,
@@ -283,12 +282,12 @@ function U2pvis(act, btns) {
             for (var a = 0, aa = ds.length; a < aa; a++) {
                 var id = ds[a].parentNode.getAttribute('id').slice(1, -1);
                 console.log("dom %d/%d = [%s] in(%s) is_act(%s) %s",
-                    a, aa, id, this.tab[id].in, this.is_act(fo.in), ds[a].textContent);
+                    a, aa, id, r.tab[id].in, r.is_act(fo.in), ds[a].textContent);
             }
 
-            for (var a = 0, aa = this.tab.length; a < aa; a++)
-                if (this.is_act(this.tab[a].in))
-                    console.log("tab %d/%d = sz %s", a, aa, this.tab[a].bt);
+            for (var a = 0, aa = r.tab.length; a < aa; a++)
+                if (r.is_act(r.tab[a].in))
+                    console.log("tab %d/%d = sz %s", a, aa, r.tab[a].bt);
 
             throw new Error('see console');
         }
@@ -298,27 +297,27 @@ function U2pvis(act, btns) {
         obj.style.background = 'linear-gradient(90deg, #050, #270 ' + o1 + '%, #4b0 ' + o2 + '%, #333 ' + o3 + '%, #333 99%, #777)';
     };
 
-    this.move = function (nfile, newcat) {
-        var fo = this.tab[nfile],
+    r.move = function (nfile, newcat) {
+        var fo = r.tab[nfile],
             oldcat = fo.in,
-            bz_act = this.act == "bz";
+            bz_act = r.act == "bz";
 
         if (oldcat == newcat)
             return;
 
         fo.in = newcat;
-        this.ctr[oldcat]--;
-        this.ctr[newcat]++;
-        this.drawcard(oldcat);
-        this.drawcard(newcat);
-        if (this.is_act(newcat)) {
-            this.tail = Math.max(this.tail, nfile + 1);
+        r.ctr[oldcat]--;
+        r.ctr[newcat]++;
+        r.drawcard(oldcat);
+        r.drawcard(newcat);
+        if (r.is_act(newcat)) {
+            r.tail = Math.max(r.tail, nfile + 1);
             if (!ebi('f' + nfile))
-                this.addrow(nfile);
+                r.addrow(nfile);
         }
-        else if (this.is_act(oldcat)) {
-            while (this.head < Math.min(this.tab.length, this.tail) && this.precard[this.tab[this.head].in])
-                this.head++;
+        else if (r.is_act(oldcat)) {
+            while (r.head < Math.min(r.tab.length, r.tail) && r.precard[r.tab[r.head].in])
+                r.head++;
 
             if (!bz_act) {
                 var tr = ebi("f" + nfile);
@@ -328,10 +327,10 @@ function U2pvis(act, btns) {
         else return;
 
         if (bz_act)
-            this.bzw();
+            r.bzw();
     };
 
-    this.bzw = function () {
+    r.bzw = function () {
         var first = QS('#u2tab>tbody>tr:first-child');
         if (!first)
             return;
@@ -340,93 +339,93 @@ function U2pvis(act, btns) {
         first = parseInt(first.getAttribute('id').slice(1));
         last = parseInt(last.getAttribute('id').slice(1));
 
-        while (this.head - first > this.wsz) {
+        while (r.head - first > r.wsz) {
             var obj = ebi('f' + (first++));
             if (obj)
                 obj.parentNode.removeChild(obj);
         }
-        while (last - this.tail < this.wsz && last < this.tab.length - 2) {
+        while (last - r.tail < r.wsz && last < r.tab.length - 2) {
             var obj = ebi('f' + (++last));
             if (!obj)
-                this.addrow(last);
+                r.addrow(last);
         }
     };
 
-    this.drawcard = function (cat) {
+    r.drawcard = function (cat) {
         var cards = QSA('#u2cards>a>span');
 
         if (cat == "q") {
-            cards[4].innerHTML = this.ctr[cat];
+            cards[4].innerHTML = r.ctr[cat];
             return;
         }
         if (cat == "bz") {
-            cards[3].innerHTML = this.ctr[cat];
+            cards[3].innerHTML = r.ctr[cat];
             return;
         }
 
-        cards[2].innerHTML = this.ctr["ok"] + this.ctr["ng"];
+        cards[2].innerHTML = r.ctr["ok"] + r.ctr["ng"];
 
         if (cat == "ng") {
-            cards[1].innerHTML = this.ctr[cat];
+            cards[1].innerHTML = r.ctr[cat];
         }
         if (cat == "ok") {
-            cards[0].innerHTML = this.ctr[cat];
+            cards[0].innerHTML = r.ctr[cat];
         }
     };
 
-    this.changecard = function (card) {
-        this.act = card;
-        this.precard = has(["ok", "ng", "done"], this.act) ? {} : this.act == "bz" ? { "ok": 1, "ng": 1 } : { "ok": 1, "ng": 1, "bz": 1 };
-        this.postcard = has(["ok", "ng", "done"], this.act) ? { "bz": 1, "q": 1 } : this.act == "bz" ? { "q": 1 } : {};
-        this.head = -1;
-        this.tail = -1;
+    r.changecard = function (card) {
+        r.act = card;
+        r.precard = has(["ok", "ng", "done"], r.act) ? {} : r.act == "bz" ? { "ok": 1, "ng": 1 } : { "ok": 1, "ng": 1, "bz": 1 };
+        r.postcard = has(["ok", "ng", "done"], r.act) ? { "bz": 1, "q": 1 } : r.act == "bz" ? { "q": 1 } : {};
+        r.head = -1;
+        r.tail = -1;
         var html = [];
-        for (var a = 0; a < this.tab.length; a++) {
-            var rt = this.tab[a].in;
-            if (this.is_act(rt)) {
-                html.push(this.genrow(a, true));
+        for (var a = 0; a < r.tab.length; a++) {
+            var rt = r.tab[a].in;
+            if (r.is_act(rt)) {
+                html.push(r.genrow(a, true));
 
-                this.tail = a;
-                if (this.head == -1)
-                    this.head = a;
+                r.tail = a;
+                if (r.head == -1)
+                    r.head = a;
             }
         }
-        if (this.head == -1) {
-            for (var a = 0; a < this.tab.length; a++) {
-                var rt = this.tab[a].in;
-                if (this.precard[rt]) {
-                    this.head = a + 1;
-                    this.tail = a;
+        if (r.head == -1) {
+            for (var a = 0; a < r.tab.length; a++) {
+                var rt = r.tab[a].in;
+                if (r.precard[rt]) {
+                    r.head = a + 1;
+                    r.tail = a;
                 }
-                else if (this.postcard[rt]) {
-                    this.head = a;
-                    this.tail = a - 1;
+                else if (r.postcard[rt]) {
+                    r.head = a;
+                    r.tail = a - 1;
                     break;
                 }
             }
         }
 
-        if (this.head < 0)
-            this.head = 0;
+        if (r.head < 0)
+            r.head = 0;
 
         if (card == "bz") {
-            for (var a = this.head - 1; a >= this.head - this.wsz && a >= 0; a--) {
-                html.unshift(this.genrow(a, true).replace(/><td>/, "><td>a "));
+            for (var a = r.head - 1; a >= r.head - r.wsz && a >= 0; a--) {
+                html.unshift(r.genrow(a, true).replace(/><td>/, "><td>a "));
             }
-            for (var a = this.tail + 1; a <= this.tail + this.wsz && a < this.tab.length; a++) {
-                html.push(this.genrow(a, true).replace(/><td>/, "><td>b "));
+            for (var a = r.tail + 1; a <= r.tail + r.wsz && a < r.tab.length; a++) {
+                html.push(r.genrow(a, true).replace(/><td>/, "><td>b "));
             }
         }
         ebi('u2tab').tBodies[0].innerHTML = html.join('\n');
     };
 
-    this.genrow = function (nfile, as_html) {
-        var r = this.tab[nfile],
+    r.genrow = function (nfile, as_html) {
+        var row = r.tab[nfile],
             td1 = '<td id="f' + nfile,
             td = '</td>' + td1,
-            ret = td1 + 'n">' + r.hn +
-                td + 't">' + r.ht +
-                td + 'p" class="prog">' + r.hp + '</td>';
+            ret = td1 + 'n">' + row.hn +
+                td + 't">' + row.ht +
+                td + 'p" class="prog">' + row.hp + '</td>';
 
         if (as_html)
             return '<tr id="f' + nfile + '">' + ret + '</tr>';
@@ -437,40 +436,50 @@ function U2pvis(act, btns) {
         return obj;
     };
 
-    this.addrow = function (nfile) {
-        var tr = this.genrow(nfile);
+    r.addrow = function (nfile) {
+        var tr = r.genrow(nfile);
         ebi('u2tab').tBodies[0].appendChild(tr);
     };
 
-    var that = this;
     btns = QSA(btns + '>a[act]');
     for (var a = 0; a < btns.length; a++) {
         btns[a].onclick = function (e) {
             ev(e);
             var newtab = this.getAttribute('act');
-            for (var b = 0; b < btns.length; b++) {
-                btns[b].className = (
-                    btns[b].getAttribute('act') == newtab) ? 'act' : '';
+            function go() {
+                for (var b = 0; b < btns.length; b++) {
+                    btns[b].className = (
+                        btns[b].getAttribute('act') == newtab) ? 'act' : '';
+                }
+                r.changecard(newtab);
             }
-            that.changecard(newtab);
+            var nf = r.ctr[newtab];
+            if (nf === undefined)
+                nf = r.ctr["ok"] + r.ctr["ng"];
+
+            if (nf < 9000)
+                return go();
+
+            modal.confirm('about to show ' + nf + ' files\n\nthis may crash your browser, are you sure?', go, null);
         };
     }
 
-    this.changecard(this.act);
+    r.changecard(r.act);
 }
 
 
-function fsearch_explain(e) {
-    ev(e);
-    if (!has(perms, 'write'))
-        return alert('your access to this folder is Read-Only\n\n' + (acct == '*' ? 'you are currently not logged in' : 'you are currently logged in as ' + acct));
+function fsearch_explain(n) {
+    if (n)
+        return toast.inf(60, 'your access to this folder is Read-Only\n\n' + (acct == '*' ? 'you are currently not logged in' : 'you are currently logged in as "' + acct + '"'));
 
-    alert('you are currently in file-search mode\n\nswitch to upload-mode by clicking the green magnifying glass (next to the big yellow search button), and then refresh\n\nsorry');
+    if (bcfg_get('fsearch', false))
+        return toast.inf(60, 'you are currently in file-search mode\n\nswitch to upload-mode by clicking the green magnifying glass (next to the big yellow search button), and then refresh\n\nsorry');
+
+    return toast.inf(60, 'refresh the page and try again, it should work now');
 }
 
 
 function up2k_init(subtle) {
-    // show modal message
     function showmodal(msg) {
         ebi('u2notbtn').innerHTML = msg;
         ebi('u2btn').style.display = 'none';
@@ -478,7 +487,6 @@ function up2k_init(subtle) {
         ebi('u2conf').style.opacity = '0.5';
     }
 
-    // hide modal message
     function unmodal() {
         ebi('u2notbtn').style.display = 'none';
         ebi('u2btn').style.display = 'block';
@@ -495,7 +503,6 @@ function up2k_init(subtle) {
         // chrome<37 firefox<34 edge<12 opera<24 safari<7
         shame = 'your browser is impressively ancient';
 
-    // upload ui hidden by default, clicking the header shows it
     var got_deps = false;
     function init_deps() {
         if (!got_deps && !subtle && !window.asmCrypto) {
@@ -512,11 +519,9 @@ function up2k_init(subtle) {
         got_deps = true;
     }
 
-    // show uploader if the user only has write-access
     if (perms.length && !has(perms, 'read'))
         goto('up2k');
 
-    // shows or clears a message in the basic uploader ui
     function setmsg(msg, type) {
         if (msg !== undefined) {
             ebi('u2err').setAttribute('class', type);
@@ -534,13 +539,11 @@ function up2k_init(subtle) {
         }
     }
 
-    // switches to the basic uploader with msg as error message
     function un2k(msg) {
         setmsg(msg, 'err');
         return false;
     }
 
-    // handle user intent to use the basic uploader instead
     ebi('u2nope').onclick = function (e) {
         ev(e);
         setmsg(suggest_up2k, 'msg');
@@ -642,7 +645,7 @@ function up2k_init(subtle) {
         else files = e.target.files;
 
         if (!files || !files.length)
-            return alert('no files selected??');
+            return toast.err(0, 'no files selected??');
 
         more_one_file();
         var bad_files = [],
@@ -710,9 +713,9 @@ function up2k_init(subtle) {
                 for (var a = 0; a < Math.min(20, missing.length); a++)
                     msg.push(missing[a]);
 
-                alert(msg.join('\n-- '));
-                dirs = [];
-                pf = [];
+                return modal.alert(msg.join('\n-- '), function () {
+                    read_dirs(rd, [], [], good, bad, spins);
+                });
             }
             spins = 0;
         }
@@ -757,7 +760,6 @@ function up2k_init(subtle) {
                 }
                 ngot += 1;
             });
-            // console.log("ngot: " + ngot);
             if (!ngot) {
                 dirs.shift();
                 rd = null;
@@ -777,16 +779,22 @@ function up2k_init(subtle) {
             if (good_files.length - bad_files.length <= 1 && ANDROID)
                 msg += '\nFirefox-Android has a bug which prevents selecting multiple files. Try selecting one file at a time. For more info, see firefox bug 1456557';
 
-            alert(msg);
+            return modal.alert(msg, function () {
+                gotallfiles(good_files, []);
+            });
         }
 
         var msg = ['upload these ' + good_files.length + ' files?'];
         for (var a = 0, aa = Math.min(20, good_files.length); a < aa; a++)
             msg.push(good_files[a][1]);
 
-        if (ask_up && !fsearch && !confirm(msg.join('\n')))
-            return;
+        if (ask_up && !fsearch)
+            return modal.confirm(msg.join('\n'), function () { up_them(good_files); }, null);
 
+        up_them(good_files);
+    }
+
+    function up_them(good_files) {
         var seen = {},
             evpath = get_evpath(),
             draw_each = good_files.length < 50;
@@ -796,17 +804,25 @@ function up2k_init(subtle) {
 
         for (var a = 0; a < good_files.length; a++) {
             var fobj = good_files[a][0],
+                name = good_files[a][1],
+                fdir = '',
                 now = Date.now(),
-                lmod = fobj.lastModified || now;
+                lmod = fobj.lastModified || now,
+                ofs = name.lastIndexOf('/') + 1;
+
+            if (ofs) {
+                fdir = name.slice(0, ofs);
+                name = name.slice(ofs);
+            }
 
             var entry = {
                 "n": st.files.length,
                 "t0": now,
                 "fobj": fobj,
-                "name": good_files[a][1],
+                "name": name,
                 "size": fobj.size,
                 "lmod": lmod / 1000,
-                "purl": evpath,
+                "purl": fdir,
                 "done": false,
                 "hash": []
             },
@@ -907,7 +923,9 @@ function up2k_init(subtle) {
         }
         return handshakes_permitted() && 0 ==
             st.todo.handshake.length +
-            st.busy.handshake.length;
+            st.busy.handshake.length +
+            st.todo.upload.length +
+            st.busy.upload.length;
     }
 
     var tasker = (function () {
@@ -1147,7 +1165,7 @@ function up2k_init(subtle) {
                     return tasker();
                 }
 
-                alert('y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
+                toast.err(0, 'y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
             };
             reader.readAsArrayBuffer(
                 bobslice.call(t.fobj, car, cdr));
@@ -1301,7 +1319,8 @@ function up2k_init(subtle) {
 
                     if (!response || !response.hits || !response.hits.length) {
                         smsg = '404';
-                        msg = 'not found on server <a href="#" onclick="fsearch_explain()" class="fsearch_explain">(explain)</a>';
+                        msg = ('not found on server <a href="#" onclick="fsearch_explain(' +
+                            (has(perms, 'write') ? '0' : '1') + ')" class="fsearch_explain">(explain)</a>');
                     }
                     else {
                         smsg = 'found';
@@ -1325,9 +1344,10 @@ function up2k_init(subtle) {
                     return;
                 }
 
-                if (response.name !== t.name) {
-                    // file exists; server renamed us
-                    console.log("server-rename [" + t.name + "] to [" + response.name + "]");
+                if (response.purl !== t.purl || response.name !== t.name) {
+                    // server renamed us (file exists / path restrictions)
+                    console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + response.purl + "] [" + response.name + "]");
+                    t.purl = response.purl;
                     t.name = response.name;
                     pvis.seth(t.n, 0, linksplit(t.purl + t.name).join(' '));
                 }
@@ -1347,7 +1367,7 @@ function up2k_init(subtle) {
                 for (var a = 0; a < missing.length; a++) {
                     var idx = t.hash.indexOf(missing[a]);
                     if (idx < 0)
-                        return alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
+                        return modal.alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
                             missing[a], JSON.stringify(t)));
 
                     t.postlist.push(idx);
@@ -1436,7 +1456,7 @@ function up2k_init(subtle) {
                     tasker();
                     return;
                 }
-                alert("server broke; hs-err {0} on file [{1}]:\n".format(
+                toast.err(0, "server broke; hs-err {0} on file [{1}]:\n".format(
                     xhr.status, t.name) + (
                         (xhr.response && xhr.response.err) ||
                         (xhr.responseText && xhr.responseText) ||
@@ -1456,7 +1476,7 @@ function up2k_init(subtle) {
         if (fsearch)
             req.srch = 1;
 
-        xhr.open('POST', t.purl + 'handshake.php', true);
+        xhr.open('POST', t.purl, true);
         xhr.responseType = 'text';
         xhr.send(JSON.stringify(req));
     }
@@ -1496,7 +1516,7 @@ function up2k_init(subtle) {
                 console.log("ignoring dupe-segment error", t);
             }
             else {
-                alert("server broke; cu-err {0} on file [{1}]:\n".format(
+                toast.err(0, "server broke; cu-err {0} on file [{1}]:\n".format(
                     xhr.status, t.name) + (txt || "no further information"));
                 return;
             }
@@ -1524,7 +1544,7 @@ function up2k_init(subtle) {
                 console.log('chunkpit onerror, retrying', t);
                 do_send();
             };
-            xhr.open('POST', t.purl + 'chunkpit.php', true);
+            xhr.open('POST', t.purl, true);
             xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
             xhr.setRequestHeader("X-Up2k-Wark", t.wark);
             xhr.setRequestHeader('Content-Type', 'application/octet-stream');
@@ -1544,7 +1564,7 @@ function up2k_init(subtle) {
 
     function onresize(e) {
         var bar = ebi('ops'),
-            wpx = innerWidth,
+            wpx = window.innerWidth,
             fpx = parseInt(getComputedStyle(bar)['font-size']),
             wem = wpx * 1.0 / fpx,
             wide = wem > 54,

copyparty/web/upload.css

@@ -1,302 +0,0 @@
-
-#op_up2k {
-	padding: 0 1em 1em 1em;
-}
-#u2form {
-	position: absolute;
-	top: 0;
-	left: 0;
-	width: 2px;
-	height: 2px;
-	overflow: hidden;
-}
-#u2form input {
-	background: #444;
-	border: 0px solid #444;
-	outline: none;
-}
-#u2err.err {
-	color: #f87;
-	padding: .5em;
-}
-#u2err.msg {
-	color: #999;
-	padding: .5em;
-	font-size: .9em;
-}
-#u2btn {
-	color: #eee;
-	background: #555;
-	background: -moz-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	background: -webkit-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	background: linear-gradient(to bottom, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#489', endColorstr='#38788a', GradientType=0);
-	text-decoration: none;
-	line-height: 1.3em;
-	border: 1px solid #222;
-	border-radius: .4em;
-	text-align: center;
-	font-size: 1.5em;
-	margin: .5em auto;
-	padding: .8em 0;
-	width: 16em;
-	cursor: pointer;
-	box-shadow: .4em .4em 0 #111;
-}
-#op_up2k.srch #u2btn {
-	background: linear-gradient(to bottom, #ca3 0%, #fd8 50%, #fc6 51%, #b92 100%);
-	text-shadow: 1px 1px 1px #fc6;
-	color: #333;
-}
-#u2conf #u2btn {
-	margin: -1.5em 0;
-	padding: .8em 0;
-	width: 100%;
-	max-width: 12em;
-	display: inline-block;
-}
-#u2conf #u2btn_cw {
-	text-align: right;
-}
-#u2notbtn {
-	display: none;
-	text-align: center;
-	background: #333;
-	padding-top: 1em;
-}
-#u2notbtn * {
-	line-height: 1.3em;
-}
-#u2tab {
-	margin: 3em auto;
-	width: calc(100% - 2em);
-	max-width: 100em;
-}
-#op_up2k.srch #u2tab {
-	max-width: none;
-}
-#u2tab td {
-	border: 1px solid #ccc;
-	border-width: 0 0px 1px 0;
-	padding: .1em .3em;
-}
-#u2tab td:nth-child(2) {
-	width: 5em;
-	white-space: nowrap;
-}
-#u2tab td:nth-child(3) {
-	width: 40%;
-}
-#op_up2k.srch td.prog {
-	font-family: sans-serif;
-	font-size: 1em;
-	width: auto;
-}
-#u2tab tbody tr:hover td {
-	background: #222;
-}
-#u2cards {
-	padding: 1em 0 .3em 1em;
-	margin: 1.5em auto -2.5em auto;
-	white-space: nowrap;
-	text-align: center;
-	overflow: hidden;
-}
-#u2cards.w {
-	width: 45em;
-	text-align: left;
-}
-#u2cards a {
-	padding: .2em 1em;
-	border: 1px solid #777;
-	border-width: 0 0 1px 0;
-	background: linear-gradient(to bottom, #333, #222);
-}
-#u2cards a:first-child {
-	border-radius: .4em 0 0 0;
-}
-#u2cards a:last-child {
-	border-radius: 0 .4em 0 0;
-}
-#u2cards a.act {
-	padding-bottom: .5em;
-	border-width: 1px 1px .1em 1px;
-	border-radius: .3em .3em 0 0;
-	margin-left: -1px;
-	background: linear-gradient(to bottom, #464, #333 80%);
-	box-shadow: 0 -.17em .67em #280;
-	border-color: #7c5 #583 #333 #583;
-	position: relative;
-	color: #fd7;
-}
-#u2cards span {
-	color: #fff;
-}
-#u2conf {
-	margin: 1em auto;
-	width: 30em;
-}
-#u2conf.has_btn {
-	width: 48em;
-}
-#u2conf * {
-	text-align: center;
-	line-height: 1em;
-	margin: 0;
-	padding: 0;
-	border: none;
-	outline: none;
-}
-#u2conf .txtbox {
-	width: 3em;
-	color: #fff;
-	background: #444;
-	border: 1px solid #777;
-	font-size: 1.2em;
-	padding: .15em 0;
-	height: 1.05em;
-}
-#u2conf .txtbox.err {
-	background: #922;
-}
-#u2conf a {
-	color: #fff;
-	background: #c38;
-	text-decoration: none;
-	border-radius: .1em;
-	font-size: 1.5em;
-	padding: .1em 0;
-	margin: 0 -1px;
-	width: 1.5em;
-	height: 1em;
-	display: inline-block;
-	position: relative;
-	bottom: -0.08em;
-}
-#u2conf input+a {
-	background: #d80;
-}
-#u2conf label {
-	font-size: 1.6em;
-	width: 2em;
-	height: 1em;
-	padding: .4em 0;
-	display: block;
-	border-radius: .25em;
-}
-#u2conf input[type="checkbox"] {
-	position: relative;
-	opacity: .02;
-	top: 2em;
-}
-#u2conf input[type="checkbox"]+label {
-	position: relative;
-	background: #603;
-	border-bottom: .2em solid #a16;
-	box-shadow: 0 .1em .3em #a00 inset;
-}
-#u2conf input[type="checkbox"]:checked+label {
-	background: #6a1;
-	border-bottom: .2em solid #efa;
-	box-shadow: 0 .1em .5em #0c0;
-}
-#u2conf input[type="checkbox"]+label:hover {
-	box-shadow: 0 .1em .3em #fb0;
-	border-color: #fb0;
-}
-#op_up2k.srch #u2conf td:nth-child(1)>*,
-#op_up2k.srch #u2conf td:nth-child(2)>*,
-#op_up2k.srch #u2conf td:nth-child(3)>* {
-	background: #777;
-	border-color: #ccc;
-	box-shadow: none;
-	opacity: .2;
-}
-#u2foot {
-	color: #fff;
-	font-style: italic;
-}
-#u2foot .warn {
-	font-size: 1.3em;
-	padding: .5em .8em;
-	margin: 1em -.6em;
-	color: #f74;
-	background: #322;
-	border: 1px solid #633;
-	border-width: .1em 0;
-	text-align: center;
-}
-#u2foot .warn span {
-	color: #f86;
-}
-html.light #u2foot .warn {
-	color: #b00;
-	background: #fca;
-	border-color: #f70;
-}
-html.light #u2foot .warn span {
-	color: #930;
-}
-#u2foot span {
-	color: #999;
-	font-size: .9em;
-	font-weight: normal;
-}
-#u2footfoot {
-	margin-bottom: -1em;
-}
-.prog {
-	font-family: monospace, monospace;
-}
-#u2tab a>span {
-	font-weight: bold;
-	font-style: italic;
-	color: #fff;
-	padding-left: .2em;
-}
-#u2cleanup {
-	float: right;
-	margin-bottom: -.3em;
-}
-.fsearch_explain {
-	padding-left: .7em;
-	font-size: 1.1em;
-	line-height: 0;
-}
-
-
-
-
-
-html.light #u2btn {
-	box-shadow: .4em .4em 0 #ccc;
-}
-html.light #u2cards span {
-	color: #000;
-}
-html.light #u2cards a {
-	background: linear-gradient(to bottom, #eee, #fff);
-}
-html.light #u2cards a.act {
-	color: #037;
-	background: inherit;
-	box-shadow: 0 -.17em .67em #0ad;
-	border-color: #09c #05a #eee #05a;
-}
-html.light #u2conf .txtbox {
-	background: #fff;
-	color: #444;
-}
-html.light #u2conf .txtbox.err {
-	background: #f96;
-	color: #300;
-}
-html.light #op_up2k.srch #u2btn {
-	border-color: #a80;
-}
-html.light #u2foot {
-	color: #000;
-}
-html.light #u2tab tbody tr:hover td {
-	background: #fff;
-}

copyparty/web/util.js

@@ -140,10 +140,10 @@ function import_js(url, cb) {
     var script = mknod('script');
     script.type = 'text/javascript';
     script.src = url;
-
-    script.onreadystatechange = cb;
     script.onload = cb;
-
+    script.onerror = function () {
+        toast.err(0, 'Failed to load module:\n' + url);
+    };
     head.appendChild(script);
 }
 
@@ -398,6 +398,15 @@ function uricom_dec(txt) {
 }
 
 
+function uricom_adec(arr) {
+    var ret = [];
+    for (var a = 0; a < arr.length; a++)
+        ret.push(uricom_dec(arr[a])[0]);
+
+    return ret;
+}
+
+
 function get_evpath() {
     var ret = document.location.pathname;
 
@@ -647,14 +656,24 @@ var tt = (function () {
 })();
 
 
+function lf2br(txt) {
+    var html = '', hp = txt.split(/(?=<.?pre>)/i);
+    for (var a = 0; a < hp.length; a++)
+        html += hp[a].startsWith('<pre>') ? hp[a] :
+            hp[a].replace(/<br ?.?>\n/g, '\n').replace(/\n<br ?.?>/g, '\n').replace(/\n/g, '<br />\n');
+
+    return html;
+}
+
+
 var toast = (function () {
     var r = {},
         te = null,
-        visible = false,
         obj = mknod('div');
 
     obj.setAttribute('id', 'toast');
-    document.body.appendChild(obj);;
+    document.body.appendChild(obj);
+    r.visible = false;
 
     r.hide = function (e) {
         ev(e);
@@ -668,12 +687,7 @@ var toast = (function () {
         if (ms)
             te = setTimeout(r.hide, ms * 1000);
 
-        var html = '', hp = txt.split(/(?=<.?pre>)/i);
-        for (var a = 0; a < hp.length; a++)
-            html += hp[a].startsWith('<pre>') ? hp[a] :
-                hp[a].replace(/<br ?.?>\n/g, '\n').replace(/\n<br ?.?>/g, '\n').replace(/\n/g, '<br />\n');
-
-        obj.innerHTML = '<a href="#" id="toastc">x</a>' + html;
+        obj.innerHTML = '<a href="#" id="toastc">x</a>' + lf2br(txt);
         obj.className = cl;
         ms += obj.offsetWidth;
         obj.className += ' vis';
@@ -696,3 +710,113 @@ var toast = (function () {
 
     return r;
 })();
+
+
+var modal = (function () {
+    var r = {},
+        q = [],
+        o = null,
+        cb_ok = null,
+        cb_ng = null;
+
+    r.busy = false;
+
+    r.show = function (html) {
+        o = mknod('div');
+        o.setAttribute('id', 'modal');
+        o.innerHTML = '<table><tr><td><div id="modalc">' + html + '</div></td></tr></table>';
+        document.body.appendChild(o);
+        document.addEventListener('keydown', onkey);
+        r.busy = true;
+
+        var a = ebi('modal-ng');
+        if (a)
+            a.onclick = ng;
+
+        a = ebi('modal-ok');
+        a.onclick = ok;
+
+        (ebi('modali') || a).focus();
+    };
+
+    r.hide = function () {
+        o.parentNode.removeChild(o);
+        document.removeEventListener('keydown', onkey);
+        r.busy = false;
+        setTimeout(next, 50);
+    };
+    function ok(e) {
+        ev(e);
+        var v = ebi('modali');
+        v = v ? v.value : true;
+        r.hide();
+        if (cb_ok)
+            cb_ok(v);
+    }
+    function ng(e) {
+        ev(e);
+        r.hide();
+        if (cb_ng)
+            cb_ng(null);
+    }
+
+    function onkey(e) {
+        if (e.code == 'Enter') {
+            var a = ebi('modal-ng');
+            if (a && document.activeElement == a)
+                return ng();
+
+            return ok();
+        }
+
+        if (e.code == 'Escape')
+            return ng();
+    }
+
+    function next() {
+        if (!r.busy && q.length)
+            q.shift()();
+    }
+
+    r.alert = function (html, cb) {
+        q.push(function () {
+            _alert(lf2br(html), cb);
+        });
+        next();
+    };
+    function _alert(html, cb) {
+        cb_ok = cb_ng = cb;
+        html += '<div id="modalb"><a href="#" id="modal-ok">OK</a></div>';
+        r.show(html);
+    }
+
+    r.confirm = function (html, cok, cng) {
+        q.push(function () {
+            _confirm(lf2br(html), cok, cng);
+        });
+        next();
+    }
+    function _confirm(html, cok, cng) {
+        cb_ok = cok;
+        cb_ng = cng === undefined ? cok : null;
+        html += '<div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        r.show(html);
+    }
+
+    r.prompt = function (html, v, cok, cng) {
+        q.push(function () {
+            _prompt(lf2br(html), v, cok, cng);
+        });
+        next();
+    }
+    function _prompt(html, v, cok, cng) {
+        cb_ok = cok;
+        cb_ng = cng === undefined ? cok : null;
+        html += '<input id="modali" type="text" /><div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        r.show(html);
+
+        ebi('modali').value = v || '';
+    }
+
+    return r;
+})();

docs/notes.sh

@@ -79,6 +79,10 @@ command -v gdate && date() { gdate "$@"; }; while true; do t=$(date +%s.%N); (ti
 # get all up2k search result URLs
 var t=[]; var b=document.location.href.split('#')[0].slice(0, -1); document.querySelectorAll('#u2tab .prog a').forEach((x) => {t.push(b+encodeURI(x.getAttribute("href")))}); console.log(t.join("\n"));
 
+# rename all selected songs to <leading-track-number> + <Title> + <extension>
+var sel=msel.getsel(), ci=find_file_col('Title')[0], re=[]; for (var a=0; a<sel.length; a++) { var url=sel[a].vp, tag=ebi(sel[a].id).closest('tr').querySelectorAll('td')[ci].textContent, name=uricom_dec(vsplit(url)[1])[0], m=/^([0-9]+[\. -]+)?.*(\.[^\.]+$)/.exec(name), name2=(m[1]||'')+tag+m[2], url2=vsplit(url)[0]+uricom_enc(name2,false); if (url!=url2) re.push([url, url2]); }
+console.log(JSON.stringify(re, null, '  '));
+function f() { if (!re.length) return treectl.goto(get_evpath()); var [u1,u2] = re.shift(); fetch(u1+'?move='+u2).then((rsp) => {if (rsp.ok) f(); }); }; f();
 
 ##
 ## bash oneliners

scripts/make-sfx.sh

@@ -2,6 +2,7 @@
 set -e
 echo
 
+help() { exec cat <<'EOF'
 
 # optional args:
 #
@@ -26,6 +27,8 @@ echo
 #
 # `no-dd` saves ~2k by removing the mouse cursor
 
+EOF
+}
 
 # port install gnutar findutils gsed coreutils
 gtar=$(command -v gtar || command -v gnutar) || true
@@ -34,6 +37,7 @@ gtar=$(command -v gtar || command -v gnutar) || true
 	sed()  { gsed  "$@"; }
 	find() { gfind "$@"; }
 	sort() { gsort "$@"; }
+	sha1sum() { shasum "$@"; }
 	unexpand() { gunexpand "$@"; }
 	command -v grealpath >/dev/null &&
 		realpath() { grealpath "$@"; }
@@ -72,6 +76,7 @@ while [ ! -z "$1" ]; do
 		no-cm)  no_cm=1  ; ;;
 		no-sh)  do_sh=   ; ;;
 		no-py)  do_py=   ; ;;
+		*)      help     ; ;;
 	esac
 	shift
 done
@@ -81,16 +86,23 @@ tmv() {
 	mv t "$1"
 }
 
+stamp=$(
+	for d in copyparty scripts; do
+		find $d -type f -printf '%TY-%Tm-%Td %TH:%TM:%TS %p\n'
+	done | sort | tail -n 1 | sha1sum | cut -c-16
+)
+
 rm -rf sfx/*
 mkdir -p sfx build
 cd sfx
 
-[ $repack ] && {
-	old="$(
-		printf '%s\n' "$TMPDIR" /tmp |
-		awk '/./ {print; exit}'
-	)/pe-copyparty"
+tmpdir="$(
+	printf '%s\n' "$TMPDIR" /tmp |
+	awk '/./ {print; exit}'
+)"
 
+[ $repack ] && {
+	old="$tmpdir/pe-copyparty"
 	echo "repack of files in $old"
 	cp -pR "$old/"*{dep-j2,copyparty} .
 }
@@ -172,12 +184,12 @@ mkdir -p ../dist
 sfx_out=../dist/copyparty-sfx
 
 echo cleanup
-find .. -name '*.pyc' -delete
-find .. -name __pycache__ -delete
+find -name '*.pyc' -delete
+find -name __pycache__ -delete
 
 # especially prevent osx from leaking your lan ip (wtf apple)
-find .. -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
-find .. -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
+find -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
+find -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
 
 echo use smol web deps
 rm -f copyparty/web/deps/*.full.* copyparty/web/dbg-* copyparty/web/Makefile
@@ -196,19 +208,24 @@ done
 	rm -rf copyparty/web/mde.* copyparty/web/deps/easymde*
 	echo h > copyparty/web/mde.html
 	f=copyparty/web/md.html
-	sed -r '/edit2">edit \(fancy/d' <$f >t && tmv "$f"
+	sed -r '/edit2">edit \(fancy/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
 	f=copyparty/web/md.css
-	sed -r '/scp\.woff2/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r '/scp\.woff2/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
-	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t
+	tmv "$f"
 }
 
 [ $repack ] ||
@@ -222,7 +239,8 @@ awk '/^LOREM_IPSUM_WORDS/{o=1;print "LOREM_IPSUM_WORDS = u\"a\"";next} !o; /"""/
 tmv "$f"
 
 # up2k goes from 28k to 22k laff
-echo entabbening
+awk 'BEGIN{gensub(//,"",1)}' </dev/null &&
+echo entabbening &&
 find | grep -E '\.css$' | while IFS= read -r f; do
 	awk '{
 		sub(/^[ \t]+/,"");
@@ -236,25 +254,48 @@ find | grep -E '\.css$' | while IFS= read -r f; do
 	' <$f | sed 's/;\}$/}/' >t
 	tmv "$f"
 done
+unexpand -h 2>/dev/null &&
 find | grep -E '\.(js|html)$' | while IFS= read -r f; do
 	unexpand -t 4 --first-only <"$f" >t
 	tmv "$f"
 done
 
-
 gzres() {
-command -v pigz &&
-	pk='pigz -11 -J 34 -I 100' ||
-	pk='gzip'
+	command -v pigz &&
+		pk='pigz -11 -I 256' ||
+		pk='gzip'
 
-echo "$pk"
-find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
-	echo -n .
-	$pk "$f"
-done
-echo
+	echo "$pk"
+	find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
+		echo -n .
+		$pk "$f"
+	done
+	echo
+}
+
+
+zdir="$tmpdir/cpp-mksfx"
+[ -e "$zdir/$stamp" ] || rm -rf "$zdir"
+mkdir -p "$zdir"
+echo a > "$zdir/$stamp"
+nf=$(ls -1 "$zdir"/arc.* | wc -l)
+[ $nf -ge 2 ] && [ ! $repack ] && use_zdir=1 || use_zdir=
+
+[ $use_zdir ] || {
+	echo "$nf alts += 1"
+	gzres
+	[ $repack ] ||
+		tar -cf "$zdir/arc.$(date +%s)" copyparty/web/*.gz
+}
+[ $use_zdir ] && {
+	arcs=("$zdir"/arc.*)
+	arc="${arcs[$RANDOM % ${#arcs[@]} ] }"
+	echo "using $arc"
+	tar -xf "$arc"
+	for f in copyparty/web/*.gz; do
+		rm "${f%.*}"
+	done
 }
-gzres
 
 
 echo gen tarlist

scripts/sfx.py

@@ -364,7 +364,7 @@ def confirm(rv):
     except:
         pass
 
-    sys.exit(rv)
+    sys.exit(rv or 1)
 
 
 def run(tmp, j2):

scripts/test/smoketest.py

@@ -124,7 +124,7 @@ def tc1():
 
         arg = "{}:{}:{}".format(pd, ud, p, hp)
         if hp:
-            arg += ":chist=" + hp
+            arg += ":c,hist=" + hp
 
         args += ["-v", arg]
 

scripts/uncomment.py

@@ -65,9 +65,9 @@ def uncomment(fpath):
 
 
 def main():
-    print("uncommenting", end="")
+    print("uncommenting", end="", flush=True)
     for f in sys.argv[1:]:
-        print(".", end="")
+        print(".", end="", flush=True)
         uncomment(f)
 
     print("k")

setup.py

@@ -99,6 +99,7 @@ args = {
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
+        "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: Implementation :: CPython",
         "Programming Language :: Python :: Implementation :: PyPy",
         "Environment :: Console",

tests/test_httpcli.py

@@ -39,10 +39,12 @@ class Cfg(Namespace):
             no_scandir=False,
             no_sendfile=True,
             no_rescan=True,
+            re_maxage=0,
             ihead=False,
             nih=True,
             mtp=[],
             mte="a",
+            mth="",
             hist=None,
             no_hash=False,
             css_browser=None,

tests/test_vfs.py

@@ -21,10 +21,12 @@ class Cfg(Namespace):
         ex2 = {
             "mtp": [],
             "mte": "a",
+            "mth": "",
             "hist": None,
             "no_hash": False,
             "css_browser": None,
             "no_voldump": True,
+            "re_maxage": 0,
             "rproxy": 0,
         }
         ex.update(ex2)