v0.12.12

README.md

@@ -16,11 +16,6 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
 
 
-## breaking changes \o/
-
-this is the readme for v0.12 which has a different expression for volume permissions (`-v`); see [the v0.11.x readme](https://github.com/9001/copyparty/tree/15b59822112dda56cee576df30f331252fc62628#readme) for stuff regarding the [current stable release](https://github.com/9001/copyparty/releases/tag/v0.11.47)
-
-
 ## readme toc
 
 * top
@@ -41,6 +36,7 @@ this is the readme for v0.12 which has a different expression for volume permiss
     * [uploading](#uploading)
         * [file-search](#file-search)
     * [file manager](#file-manager)
+    * [batch rename](#batch-rename)
     * [markdown viewer](#markdown-viewer)
     * [other tricks](#other-tricks)
 * [searching](#searching)
@@ -125,29 +121,31 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ [accounts](#accounts-and-volumes)
 * upload
   * ☑ basic: plain multipart, ie6 support
-  * ☑ up2k: js, resumable, multithreaded
+  * ☑ [up2k](#uploading): js, resumable, multithreaded
   * ☑ stash: simple PUT filedropper
+  * ☑ unpost: undo/delete accidental uploads
   * ☑ symlink/discard existing files (content-matching)
 * download
   * ☑ single files in browser
-  * ☑ folders as zip / tar files
+  * ☑ [folders as zip / tar files](#zip-downloads)
   * ☑ FUSE client (read-only)
 * browser
   * ☑ navpane (directory tree sidebar)
+  * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
   * ☑ audio player (with OS media controls)
-  * ☑ thumbnails
+  * ☑ image gallery with webm player
+  * ☑ [thumbnails](#thumbnails)
     * ☑ ...of images using Pillow
     * ☑ ...of videos using FFmpeg
     * ☑ cache eviction (max-age; maybe max-size eventually)
-  * ☑ image gallery with webm player
   * ☑ SPA (browse while uploading)
     * if you use the navpane to navigate, not folders in the file list
 * server indexing
-  * ☑ locate files by contents
+  * ☑ [locate files by contents](#file-search)
   * ☑ search by name/path/date/size
-  * ☑ search by ID3-tags etc.
+  * ☑ [search by ID3-tags etc.](#searching)
 * markdown
-  * ☑ viewer
+  * ☑ [viewer](#markdown-viewer)
   * ☑ editor (sure why not)
 
 
@@ -180,7 +178,7 @@ small collection of user feedback
   * this is an msys2 bug, the regular windows edition of python is fine
 
 * VirtualBox: sqlite throws `Disk I/O Error` when running in a VM and the up2k database is in a vboxsf
-  * use `--hist` or the `hist` volflag (`-v [...]:chist=/tmp/foo`) to place the db inside the vm instead
+  * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db inside the vm instead
 
 
 # accounts and volumes
@@ -215,10 +213,11 @@ example:
 ## tabs
 
 * `[🔎]` search by size, date, path/name, mp3-tags ... see [searching](#searching)
+* `[🧯]` unpost: undo/delete accidental uploads
 * `[🚀]` and `[🎈]` are the uploaders, see [uploading](#uploading)
-* `[📂]` mkdir, create directories
-* `[📝]` new-md, create a new markdown document
-* `[📟]` send-msg, either to server-log or into textfiles if `--urlform save`
+* `[📂]` mkdir: create directories
+* `[📝]` new-md: create a new markdown document
+* `[📟]` send-msg: either to server-log or into textfiles if `--urlform save`
 * `[🎺]` audio-player config options
 * `[⚙️]` general client config options
 
@@ -312,8 +311,10 @@ you can also zip a selection of files or folders by clicking them in the browser
 ## uploading
 
 two upload methods are available in the html client:
-* `🎈 bup`, the basic uploader, supports almost every browser since netscape 4.0
-* `🚀 up2k`, the fancy one
+* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
+* `[🚀] up2k`, the fancy one
+
+you can undo/delete uploads using `[🧯] unpost` if the server is running with `-e2d`
 
 up2k has several advantages:
 * you can drop folders into the browser (files are added recursively)
@@ -369,6 +370,53 @@ if you have the required permissions, you can cut/paste, rename, and delete file
 you can move files across browser tabs (cut in one tab, paste in another)
 
 
+## batch rename
+
+![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
+
+select some files and press F2 to bring up the rename UI
+
+quick explanation of the buttons,  
+* `[✅ apply rename]` confirms and begins renaming
+* `[❌ cancel]` aborts and closes the rename window
+* `[↺ reset]` reverts any filename changes back to the original name
+* `[decode]` does a URL-decode on the filename, fixing stuff like `&` and `%20`
+* `[advanced]` toggles advanced mode
+
+advanced mode: rename files based on rules to decide the new names, based on the original name (regex), or based on the tags collected from the file (artist/title/...), or a mix of both
+
+in advanced mode,  
+* `[case]` toggles case-sensitive regex
+* `regex` is the regex pattern to apply to the original filename; any files which don't match will be skipped
+* `format` is the new filename, taking values from regex capturing groups and/or from file tags
+  * very loosely based on foobar2000 syntax
+* `presets` lets you save rename rules for later
+
+available functions:
+* `$lpad(text, length, pad_char)`
+* `$rpad(text, length, pad_char)`
+
+so,
+
+say you have a file named [`meganeko - Eclipse - 07 Sirius A.mp3`](https://www.youtube.com/watch?v=-dtb0vDPruI) (absolutely fantastic album btw) and the tags are: `Album:Eclipse`, `Artist:meganeko`, `Title:Sirius A`, `tn:7`
+
+you could use just regex to rename it:
+* `regex` = `(.*) - (.*) - ([0-9]{2}) (.*)`
+* `format` = `(3). (1) - (4)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+or you could use just tags:
+* `format` = `$lpad((tn),2,0). (artist) - (title).(ext)`
+* `output` = `7. meganeko - Sirius A.mp3`
+
+or a mix of both:
+* `regex` = ` - ([0-9]{2}) `
+* `format` = `(1). (artist) - (title).(ext)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
+
+
 ## markdown viewer
 
 ![copyparty-md-read-fs8](https://user-images.githubusercontent.com/241032/115978057-66419080-a57d-11eb-8539-d2be843991aa.png)
@@ -411,9 +459,9 @@ through arguments:
 * `-e2tsr` deletes all existing tags, does a full reindex
 
 the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
-* `-v ~/music::r:ce2dsa:ce2tsr` does a full reindex of everything on startup
-* `-v ~/music::r:cd2d` disables **all** indexing, even if any `-e2*` are on
-* `-v ~/music::r:cd2t` disables all `-e2t*` (tags), does not affect `-e2d*`
+* `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
+* `-v ~/music::r:c,d2d` disables **all** indexing, even if any `-e2*` are on
+* `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
 
 note:
 * `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
@@ -432,7 +480,7 @@ if you set `--no-hash`, you can enable hashing for specific volumes using flag `
 copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
 
 this can instead be kept in a single place using the `--hist` argument, or the `hist=` volume flag, or a mix of both:
-* `--hist ~/.cache/copyparty -v ~/music::r:chist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
+* `--hist ~/.cache/copyparty -v ~/music::r:c,hist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
 
 note:
 * markdown edits are always stored in a local `.hist` subdirectory
@@ -443,10 +491,12 @@ note:
 ## metadata from audio files
 
 `-mte` decides which tags to index and display in the browser (and also the display order), this can be changed per-volume:
-* `-v ~/music::r:cmte=title,artist` indexes and displays *title* followed by *artist*
+* `-v ~/music::r:c,mte=title,artist` indexes and displays *title* followed by *artist*
 
 if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
 
+but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
+
 `-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
 
 tags that start with a `.` such as `.bpm` and `.dur`(ation) indicate numeric value
@@ -467,7 +517,7 @@ copyparty can invoke external programs to collect additional metadata for files
 
 * `-mtp .bpm=~/bin/audio-bpm.py` will execute `~/bin/audio-bpm.py` with the audio file as argument 1 to provide the `.bpm` tag, if that does not exist in the audio metadata
 * `-mtp key=f,t5,~/bin/audio-key.py` uses `~/bin/audio-key.py` to get the `key` tag, replacing any existing metadata tag (`f,`), aborting if it takes longer than 5sec (`t5,`)
-* `-v ~/music::r:cmtp=.bpm=~/bin/audio-bpm.py:cmtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
+* `-v ~/music::r:c,mtp=.bpm=~/bin/audio-bpm.py:c,mtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
 
 *but wait, there's more!* `-mtp` can be used for non-audio files as well using the `a` flag: `ay` only do audio files, `an` only do non-audio files, or `ad` do all files (d as in dontcare) 
 

bin/mtag/README.md

@@ -4,6 +4,7 @@ some of these rely on libraries which are not MIT-compatible
 
 * [audio-bpm.py](./audio-bpm.py) detects the BPM of music using the BeatRoot Vamp Plugin; imports GPL2
 * [audio-key.py](./audio-key.py) detects the melodic key of music using the Mixxx fork of keyfinder; imports GPL3
+* [media-hash.py](./media-hash.py) generates checksums for audio and video streams; uses FFmpeg (LGPL or GPL)
 
 
 # dependencies
@@ -18,7 +19,10 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 
 # usage from copyparty
 
-`copyparty -e2dsa -e2ts -mtp key=f,audio-key.py -mtp .bpm=f,audio-bpm.py`
+`copyparty -e2dsa -e2ts` followed by any combination of these:
+* `-mtp key=f,audio-key.py`
+* `-mtp .bpm=f,audio-bpm.py`
+* `-mtp ahash,vhash=f,media-hash.py`
 
 * `f,` makes the detected value replace any existing values
 * the `.` in `.bpm` indicates numeric value
@@ -29,6 +33,9 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 ## usage with volume-flags
 
 instead of affecting all volumes, you can set the options for just one volume like so:
-```
-copyparty -v /mnt/nas/music:/music:r:cmtp=key=f,audio-key.py:cmtp=.bpm=f,audio-bpm.py:ce2dsa:ce2ts
-```
+
+`copyparty -v /mnt/nas/music:/music:r:c,e2dsa:c,e2ts` immediately followed by any combination of these:
+
+* `:c,mtp=key=f,audio-key.py`
+* `:c,mtp=.bpm=f,audio-bpm.py`
+* `:c,mtp=ahash,vhash=f,media-hash.py`

bin/mtag/media-hash.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python
+
+import re
+import sys
+import json
+import time
+import base64
+import hashlib
+import subprocess as sp
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p
+
+
+"""
+dep: ffmpeg
+"""
+
+
+def det():
+    # fmt: off
+    cmd = [
+        "ffmpeg",
+        "-nostdin",
+        "-hide_banner",
+        "-v", "fatal",
+        "-i", fsenc(sys.argv[1]),
+        "-f", "framemd5",
+        "-"
+    ]
+    # fmt: on
+
+    p = sp.Popen(cmd, stdout=sp.PIPE)
+    # ps = io.TextIOWrapper(p.stdout, encoding="utf-8")
+    ps = p.stdout
+
+    chans = {}
+    for ln in ps:
+        if ln.startswith(b"#stream#"):
+            break
+
+        m = re.match(r"^#media_type ([0-9]): ([a-zA-Z])", ln.decode("utf-8"))
+        if m:
+            chans[m.group(1)] = m.group(2)
+
+    hashers = [hashlib.sha512(), hashlib.sha512()]
+    for ln in ps:
+        n = int(ln[:1])
+        v = ln.rsplit(b",", 1)[-1].strip()
+        hashers[n].update(v)
+
+    r = {}
+    for k, v in chans.items():
+        dg = hashers[int(k)].digest()[:12]
+        dg = base64.urlsafe_b64encode(dg).decode("ascii")
+        r[v[0].lower() + "hash"] = dg
+
+    print(json.dumps(r, indent=4))
+
+
+def main():
+    try:
+        det()
+    except:
+        pass  # mute
+
+
+if __name__ == "__main__":
+    main()

contrib/systemd/copyparty.service

@@ -13,6 +13,13 @@
 #   But note that journalctl will get the timestamps wrong due to
 #   python disabling line-buffering, so messages are out-of-order:
 #   https://user-images.githubusercontent.com/241032/126040249-cb535cc7-c599-4931-a796-a5d9af691bad.png
+#
+# enable line-buffering for realtime logging (slight performance cost):
+#   modify ExecStart and prefix it with `/usr/bin/stdbuf -oL` like so:
+#   ExecStart=/usr/bin/stdbuf -oL /usr/bin/python3 [...]
+# but some systemd versions require this instead (higher performance cost):
+#   inside the [Service] block, add the following line:
+#   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
 Description=copyparty file server

copyparty/__main__.py

@@ -24,6 +24,7 @@ from .__init__ import E, WINDOWS, VT100, PY2, unicode
 from .__version__ import S_VERSION, S_BUILD_DT, CODENAME
 from .svchub import SvcHub
 from .util import py_desc, align_tab, IMPLICATIONS
+from .authsrv import re_vol
 
 HAVE_SSL = True
 try:
@@ -264,9 +265,12 @@ def run_argparse(argv, formatter):
     ap2.add_argument("-ed", action="store_true", help="enable ?dots")
     ap2.add_argument("-emp", action="store_true", help="enable markdown plugins")
     ap2.add_argument("-mcr", metavar="SEC", type=int, default=60, help="md-editor mod-chk rate")
+    ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-forms; examples: [stash], [save,get]")
+
+    ap2 = ap.add_argument_group('upload options')
     ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="up2k min.size threshold (mswin-only)")
-    ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-forms; examples: [stash], [save,get]")
+    ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled")
 
     ap2 = ap.add_argument_group('network options')
     ap2.add_argument("-i", metavar="IP", type=u, default="0.0.0.0", help="ip to bind (comma-sep.)")
@@ -319,25 +323,29 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age")
     ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat for")
 
-    ap2 = ap.add_argument_group('database options')
+    ap2 = ap.add_argument_group('general db options')
     ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
     ap2.add_argument("-e2ds", action="store_true", help="enable up2k db-scanner, sets -e2d")
     ap2.add_argument("-e2dsa", action="store_true", help="scan all folders (for search), sets -e2ds")
+    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs)")
+    ap2.add_argument("--no-hash", action="store_true", help="disable hashing during e2ds folder scans")
+    ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
+    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval (0=off)")
+    ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
+    
+    ap2 = ap.add_argument_group('metadata db options')
     ap2.add_argument("-e2t", action="store_true", help="enable metadata indexing")
     ap2.add_argument("-e2ts", action="store_true", help="enable metadata scanner, sets -e2t")
     ap2.add_argument("-e2tsr", action="store_true", help="rescan all metadata, sets -e2ts")
-    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume state")
-    ap2.add_argument("--no-hash", action="store_true", help="disable hashing during e2ds folder scans")
     ap2.add_argument("--no-mutagen", action="store_true", help="use FFprobe for tags instead")
     ap2.add_argument("--no-mtag-mt", action="store_true", help="disable tag-read parallelism")
     ap2.add_argument("--no-mtag-ff", action="store_true", help="never use FFprobe as tag reader")
-    ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
-    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval (0=off)")
     ap2.add_argument("-mtm", metavar="M=t,t,t", type=u, action="append", help="add/replace metadata mapping")
     ap2.add_argument("-mte", metavar="M,M,M", type=u, help="tags to index/display (comma-sep.)",
-        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,ac,vc,res,.fps")
+        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,vc,ac,res,.fps,ahash,vhash")
+    ap2.add_argument("-mth", metavar="M,M,M", type=u, help="tags to hide by default (comma-sep.)",
+        default=".vq,.aq,vc,ac,res,.fps")
     ap2.add_argument("-mtp", metavar="M=[f,]bin", type=u, action="append", help="read tag M using bin")
-    ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
 
     ap2 = ap.add_argument_group('appearance options')
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
@@ -391,10 +399,16 @@ def main(argv=None):
     nstrs = []
     anymod = False
     for ostr in al.v or []:
+        m = re_vol.match(ostr)
+        if not m:
+            # not our problem
+            nstrs.append(ostr)
+            continue
+
+        src, dst, perms = m.groups()
+        na = [src, dst]
         mod = False
-        oa = ostr.split(":")
-        na = oa[:2]
-        for opt in oa[2:]:
+        for opt in perms.split(":"):
             if re.match("c[^,]", opt):
                 mod = True
                 na.append("c," + opt[1:])

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 12, 1)
+VERSION = (0, 12, 12)
 CODENAME = "fil\033[33med"
-BUILD_DT = (2021, 7, 28)
+BUILD_DT = (2021, 8, 6)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -310,6 +310,12 @@ class VFS(object):
                 yield f
 
 
+if WINDOWS:
+    re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
+else:
+    re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
+
+
 class AuthSrv(object):
     """verifies users against given paths"""
 
@@ -319,11 +325,6 @@ class AuthSrv(object):
         self.warn_anonwrite = warn_anonwrite
         self.line_ctr = 0
 
-        if WINDOWS:
-            self.re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
-        else:
-            self.re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
-
         self.mutex = threading.Lock()
         self.reload()
 
@@ -401,17 +402,18 @@ class AuthSrv(object):
         if uname == "":
             uname = "*"
 
-        if "r" in lvl:
-            axs.uread[uname] = 1
+        for un in uname.split(","):
+            if "r" in lvl:
+                axs.uread[un] = 1
 
-        if "w" in lvl:
-            axs.uwrite[uname] = 1
+            if "w" in lvl:
+                axs.uwrite[un] = 1
 
-        if "m" in lvl:
-            axs.umove[uname] = 1
+            if "m" in lvl:
+                axs.umove[un] = 1
 
-        if "d" in lvl:
-            axs.udel[uname] = 1
+            if "d" in lvl:
+                axs.udel[un] = 1
 
     def _read_volflag(self, flags, name, value, is_list):
         if name not in ["mtp"]:
@@ -452,7 +454,7 @@ class AuthSrv(object):
             # list of src:dst:permset:permset:...
             # permset is <rwmd>[,username][,username] or <c>,<flag>[=args]
             for v_str in self.args.v:
-                m = self.re_vol.match(v_str)
+                m = re_vol.match(v_str)
                 if not m:
                     raise Exception("invalid -v argument: [{}]".format(v_str))
 
@@ -623,9 +625,11 @@ class AuthSrv(object):
                 if k1 in vol.flags:
                     vol.flags[k2] = True
 
-            # default tag-list if unset
+            # default tag cfgs if unset
             if "mte" not in vol.flags:
                 vol.flags["mte"] = self.args.mte
+            if "mth" not in vol.flags:
+                vol.flags["mth"] = self.args.mth
 
             # append parsers from argv to volume-flags
             self._read_volflag(vol.flags, "mtp", self.args.mtp, True)
@@ -795,7 +799,7 @@ class AuthSrv(object):
 
                 atop = vn.realpath
                 g = vn.walk(
-                    "", "", [], u, True, [[True]], not self.args.no_scandir, False
+                    vn.vpath, "", [], u, [[True]], True, not self.args.no_scandir, False
                 )
                 for _, _, vpath, apath, files, _, _ in g:
                     fnames = [n[0] for n in files]

copyparty/broker_mpw.py

@@ -1,6 +1,5 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
-from copyparty.authsrv import AuthSrv
 
 import sys
 import signal
@@ -9,6 +8,7 @@ import threading
 from .broker_util import ExceptionalQueue
 from .httpsrv import HttpSrv
 from .util import FAKE_MP
+from copyparty.authsrv import AuthSrv
 
 
 class MpWorker(object):

copyparty/httpcli.py

@@ -61,7 +61,10 @@ class HttpCli(object):
         a, b = m.groups()
         return "=\033[7m {} \033[27m{}".format(self.asrv.iacct[a], b)
 
-    def _check_nonfatal(self, ex):
+    def _check_nonfatal(self, ex, post):
+        if post:
+            return ex.code < 300
+
         return ex.code < 400 or ex.code in [404, 429]
 
     def _assert_safe_rem(self, rem):
@@ -103,7 +106,7 @@ class HttpCli(object):
             self.req = "[junk]"
             self.http_ver = "HTTP/1.1"
             # self.log("pebkac at httpcli.run #1: " + repr(ex))
-            self.keepalive = self._check_nonfatal(ex)
+            self.keepalive = False
             self.loud_reply(unicode(ex), status=ex.code)
             return self.keepalive
 
@@ -179,7 +182,7 @@ class HttpCli(object):
 
         self.uparam = uparam
         self.cookies = cookies
-        self.vpath = unquotep(vpath)
+        self.vpath = unquotep(vpath)  # not query, so + means +
 
         pwd = uparam.get("pw")
         self.uname = self.asrv.iacct.get(pwd, "*")
@@ -216,7 +219,8 @@ class HttpCli(object):
         except Pebkac as ex:
             try:
                 # self.log("pebkac at httpcli.run #2: " + repr(ex))
-                if not self._check_nonfatal(ex):
+                post = self.mode in ["POST", "PUT"] or "content-length" in self.headers
+                if not self._check_nonfatal(ex, post):
                     self.keepalive = False
 
                 self.log("{}\033[0m, {}".format(str(ex), self.vpath), 3)
@@ -342,11 +346,36 @@ class HttpCli(object):
             static_path = os.path.join(E.mod, "web/", self.vpath[5:])
             return self.tx_file(static_path)
 
+        x = self.asrv.vfs.can_access(self.vpath, self.uname)
+        self.can_read, self.can_write, self.can_move, self.can_delete = x
+        if not self.can_read and not self.can_write:
+            if self.vpath:
+                self.log("inaccessible: [{}]".format(self.vpath))
+                raise Pebkac(404)
+
+            self.uparam["h"] = False
+
         if "tree" in self.uparam:
             return self.tx_tree()
 
-        if "stack" in self.uparam:
-            return self.tx_stack()
+        if "delete" in self.uparam:
+            return self.handle_rm()
+
+        if "move" in self.uparam:
+            return self.handle_mv()
+
+        if "scan" in self.uparam:
+            return self.scanvol()
+
+        if not self.vpath:
+            if "stack" in self.uparam:
+                return self.tx_stack()
+
+            if "ups" in self.uparam:
+                return self.tx_ups()
+
+            if "h" in self.uparam:
+                return self.tx_mounts()
 
         # conditional redirect to single volumes
         if self.vpath == "" and not self.ouparam:
@@ -362,28 +391,6 @@ class HttpCli(object):
                     self.redirect(vpath, flavor="redirecting to", use302=True)
                     return True
 
-        x = self.asrv.vfs.can_access(self.vpath, self.uname)
-        self.can_read, self.can_write, self.can_move, self.can_delete = x
-        if not self.can_read and not self.can_write:
-            if self.vpath:
-                self.log("inaccessible: [{}]".format(self.vpath))
-                raise Pebkac(404)
-
-            self.uparam = {"h": False}
-
-        if "delete" in self.uparam:
-            return self.handle_rm()
-
-        if "move" in self.uparam:
-            return self.handle_mv()
-
-        if "h" in self.uparam:
-            self.vpath = None
-            return self.tx_mounts()
-
-        if "scan" in self.uparam:
-            return self.scanvol()
-
         return self.tx_browser()
 
     def handle_options(self):
@@ -498,7 +505,14 @@ class HttpCli(object):
         if not self.args.nw:
             vfs, vrem = vfs.get_dbv(rem)
             self.conn.hsrv.broker.put(
-                False, "up2k.hash_file", vfs.realpath, vfs.flags, vrem, fn
+                False,
+                "up2k.hash_file",
+                vfs.realpath,
+                vfs.flags,
+                vrem,
+                fn,
+                self.ip,
+                time.time(),
             )
 
         return post_sz, sha_b64, remains, path
@@ -592,6 +606,9 @@ class HttpCli(object):
         if "srch" in self.uparam or "srch" in body:
             return self.handle_search(body)
 
+        if "delete" in self.uparam:
+            return self.handle_rm(body)
+
         # up2k-php compat
         for k in "chunkpit.php", "handshake.php":
             if self.vpath.endswith(k):
@@ -905,6 +922,8 @@ class HttpCli(object):
                             dbv.flags,
                             vrem,
                             fname,
+                            self.ip,
+                            time.time(),
                         )
                         self.conn.nbyte += sz
 
@@ -1291,11 +1310,9 @@ class HttpCli(object):
         else:
             fn = self.headers.get("host", "hey")
 
-        afn = "".join(
-            [x if x in (string.ascii_letters + string.digits) else "_" for x in fn]
-        )
-
-        bascii = unicode(string.ascii_letters + string.digits).encode("utf-8")
+        safe = (string.ascii_letters + string.digits).replace("%", "")
+        afn = "".join([x if x in safe.replace('"', "") else "_" for x in fn])
+        bascii = unicode(safe).encode("utf-8")
         ufn = fn.encode("utf-8", "xmlcharrefreplace")
         if PY2:
             ufn = [unicode(x) if x in bascii else "%{:02x}".format(ord(x)) for x in ufn]
@@ -1310,6 +1327,7 @@ class HttpCli(object):
 
         cdis = "attachment; filename=\"{}.{}\"; filename*=UTF-8''{}.{}"
         cdis = cdis.format(afn, fmt, ufn, fmt)
+        self.log(cdis)
         self.send_headers(None, mime=mime, headers={"Content-Disposition": cdis})
 
         fgen = vn.zipgen(rem, items, self.uname, dots, not self.args.no_scandir)
@@ -1542,14 +1560,52 @@ class HttpCli(object):
         ret["a"] = dirs
         return ret
 
-    def handle_rm(self):
-        if not self.can_delete:
+    def tx_ups(self):
+        if not self.args.unpost:
+            raise Pebkac(400, "the unpost feature was disabled by server config")
+
+        filt = self.uparam.get("filter")
+        lm = "ups [{}]".format(filt)
+        self.log(lm)
+
+        ret = []
+        t0 = time.time()
+        idx = self.conn.get_u2idx()
+        lim = time.time() - self.args.unpost
+        for vol in self.asrv.vfs.all_vols.values():
+            cur = idx.get_cur(vol.realpath)
+            if not cur:
+                continue
+
+            q = "select sz, rd, fn, at from up where ip=? and at>?"
+            for sz, rd, fn, at in cur.execute(q, (self.ip, lim)):
+                vp = "/" + "/".join([rd, fn]).strip("/")
+                if filt and filt not in vp:
+                    continue
+
+                ret.append({"vp": vp, "sz": sz, "at": at})
+                if len(ret) > 3000:
+                    ret.sort(key=lambda x: x["at"], reverse=True)
+                    ret = ret[:2000]
+
+        ret.sort(key=lambda x: x["at"], reverse=True)
+        ret = ret[:2000]
+
+        jtxt = json.dumps(ret, indent=2, sort_keys=True).encode("utf-8", "replace")
+        self.log("{} #{} {:.2f}sec".format(lm, len(ret), time.time() - t0))
+        self.reply(jtxt, mime="application/json")
+
+    def handle_rm(self, req=None):
+        if not req and not self.can_delete:
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_del:
             raise Pebkac(403, "disabled by argv")
 
-        x = self.conn.hsrv.broker.put(True, "up2k.handle_rm", self.uname, self.vpath)
+        if not req:
+            req = [self.vpath]
+
+        x = self.conn.hsrv.broker.put(True, "up2k.handle_rm", self.uname, self.ip, req)
         self.loud_reply(x.get())
 
     def handle_mv(self):
@@ -1564,6 +1620,9 @@ class HttpCli(object):
         if not dst:
             raise Pebkac(400, "need dst vpath")
 
+        # x-www-form-urlencoded (url query part) uses
+        # either + or %20 for 0x20 so handle both
+        dst = unquotep(dst.replace("+", " "))
         x = self.conn.hsrv.broker.put(
             True, "up2k.handle_mv", self.uname, self.vpath, dst
         )
@@ -1696,12 +1755,13 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": json.dumps(perms),
             "taglist": [],
-            "tag_order": [],
+            "def_hcols": [],
             "have_up2k_idx": ("e2d" in vn.flags),
             "have_tags_idx": ("e2t" in vn.flags),
             "have_mv": (not self.args.no_mv),
             "have_del": (not self.args.no_del),
             "have_zip": (not self.args.no_zip),
+            "have_unpost": (self.args.unpost > 0),
             "have_b_u": (self.can_write and self.uparam.get("b") == "u"),
             "url_suf": url_suf,
             "logues": logues,
@@ -1892,8 +1952,8 @@ class HttpCli(object):
         j2a["logues"] = logues
         j2a["taglist"] = taglist
 
-        if "mte" in vn.flags:
-            j2a["tag_order"] = json.dumps(vn.flags["mte"].split(","))
+        if "mth" in vn.flags:
+            j2a["def_hcols"] = vn.flags["mth"].split(",")
 
         if self.args.css_browser:
             j2a["css"] = self.args.css_browser

copyparty/httpsrv.py

@@ -174,25 +174,26 @@ class HttpSrv(object):
         now = time.time()
 
         if now - (self.tp_time or now) > 300:
+            m = "httpserver threadpool died: tpt {:.2f}, now {:.2f}, nthr {}, ncli {}"
+            self.log(self.name, m.format(self.tp_time, now, self.tp_nthr, self.ncli), 1)
+            self.tp_time = None
             self.tp_q = None
 
-        if self.tp_q:
-            self.tp_q.put((sck, addr))
-            with self.mutex:
-                self.ncli += 1
+        with self.mutex:
+            self.ncli += 1
+            if self.tp_q:
                 self.tp_time = self.tp_time or now
-                self.tp_ncli = max(self.tp_ncli, self.ncli + 1)
+                self.tp_ncli = max(self.tp_ncli, self.ncli)
                 if self.tp_nthr < self.ncli + 4:
                     self.start_threads(8)
-            return
+
+                self.tp_q.put((sck, addr))
+                return
 
         if not self.args.no_htp:
             m = "looks like the httpserver threadpool died; please make an issue on github and tell me the story of how you pulled that off, thanks and dog bless\n"
             self.log(self.name, m, 1)
 
-        with self.mutex:
-            self.ncli += 1
-
         thr = threading.Thread(
             target=self.thr_client,
             args=(sck, addr),

copyparty/mtag.py

@@ -434,7 +434,15 @@ class MTag(object):
             try:
                 v = getattr(md.info, attr)
             except:
-                continue
+                if k != "ac":
+                    continue
+
+                try:
+                    v = str(md.info).split(".")[1]
+                    if v.startswith("ogg"):
+                        v = v[3:]
+                except:
+                    continue
 
             if not v:
                 continue

copyparty/th_cli.py

@@ -26,6 +26,9 @@ class ThumbCli(object):
         if is_vid and self.args.no_vthumb:
             return None
 
+        if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg"]:
+            return os.path.join(ptop, rem)
+
         if fmt == "j" and self.args.th_no_jpg:
             fmt = "w"
 

copyparty/th_srv.py

@@ -205,8 +205,8 @@ class ThumbSrv(object):
                 try:
                     fun(abspath, tpath)
                 except:
-                    msg = "{} failed on {}\n{}"
-                    self.log(msg.format(fun.__name__, abspath, min_ex()), 3)
+                    msg = "{} could not create thumbnail of {}\n{}"
+                    self.log(msg.format(fun.__name__, abspath, min_ex()), "1;30")
                     with open(tpath, "wb") as _:
                         pass
 
@@ -286,8 +286,9 @@ class ThumbSrv(object):
         cmd += seek
         cmd += [
             b"-i", fsenc(abspath),
+            b"-map", b"0:v:0",
             b"-vf", scale,
-            b"-vframes", b"1",
+            b"-frames:v", b"1",
         ]
         # fmt: on
 
@@ -308,8 +309,9 @@ class ThumbSrv(object):
 
         ret, sout, serr = runcmd(cmd)
         if ret != 0:
-            msg = ["ff: {}".format(x) for x in serr.split("\n")]
-            self.log("FFmpeg failed:\n" + "\n".join(msg), c="1;30")
+            m = "FFmpeg failed (probably a corrupt video file):\n"
+            m += "\n".join(["ff: {}".format(x) for x in serr.split("\n")])
+            self.log(m, c="1;30")
             raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
 
     def poke(self, tdir):

copyparty/u2idx.py

@@ -244,7 +244,7 @@ class U2idx(object):
             sret = []
             c = cur.execute(q, v)
             for hit in c:
-                w, ts, sz, rd, fn = hit
+                w, ts, sz, rd, fn, ip, at = hit
                 lim -= 1
                 if lim <= 0:
                     break

copyparty/up2k.py

@@ -45,7 +45,7 @@ try:
 except:
     HAVE_SQLITE3 = False
 
-DB_VER = 4
+DB_VER = 5
 
 
 class Up2k(object):
@@ -522,7 +522,7 @@ class Up2k(object):
 
                     wark = up2k_wark_from_hashlist(self.salt, sz, hashes)
 
-                self.db_add(dbw[0], wark, rd, fn, lmod, sz)
+                self.db_add(dbw[0], wark, rd, fn, lmod, sz, "", 0)
                 dbw[1] += 1
                 ret += 1
                 td = time.time() - dbw[2]
@@ -537,8 +537,8 @@ class Up2k(object):
         rm = []
         nchecked = 0
         nfiles = next(cur.execute("select count(w) from up"))[0]
-        c = cur.execute("select * from up")
-        for dwark, dts, dsz, drd, dfn in c:
+        c = cur.execute("select rd, fn from up")
+        for drd, dfn in c:
             nchecked += 1
             if drd.startswith("//") or dfn.startswith("//"):
                 drd, dfn = s3dec(drd, dfn)
@@ -941,6 +941,15 @@ class Up2k(object):
         if not existed and ver is None:
             return self._create_db(db_path, cur)
 
+        if ver == 4:
+            try:
+                m = "creating backup before upgrade: "
+                cur = self._backup_db(db_path, cur, ver, m)
+                self._upgrade_v4(cur)
+                ver = 5
+            except:
+                self.log("WARN: failed to upgrade from v4", 3)
+
         if ver == DB_VER:
             try:
                 nfiles = next(cur.execute("select count(w) from up"))[0]
@@ -1011,9 +1020,10 @@ class Up2k(object):
             idx = r"create index up_w on up(w)"
 
         for cmd in [
-            r"create table up (w text, mt int, sz int, rd text, fn text)",
+            r"create table up (w text, mt int, sz int, rd text, fn text, ip text, at int)",
             r"create index up_rd on up(rd)",
             r"create index up_fn on up(fn)",
+            r"create index up_ip on up(ip)",
             idx,
             r"create table mt (w text, k text, v int)",
             r"create index mt_w on mt(w)",
@@ -1028,6 +1038,17 @@ class Up2k(object):
         self.log("created DB at {}".format(db_path))
         return cur
 
+    def _upgrade_v4(self, cur):
+        for cmd in [
+            r"alter table up add column ip text",
+            r"alter table up add column at int",
+            r"create index up_ip on up(ip)",
+            r"update kv set v=5 where k='sver'",
+        ]:
+            cur.execute(cmd)
+
+        cur.connection.commit()
+
     def handle_json(self, cj):
         with self.mutex:
             if not self.register_vpath(cj["ptop"], cj["vcfg"]):
@@ -1051,7 +1072,7 @@ class Up2k(object):
                     argv = (wark[:16], wark)
 
                 cur = cur.execute(q, argv)
-                for _, dtime, dsize, dp_dir, dp_fn in cur:
+                for _, dtime, dsize, dp_dir, dp_fn, ip, at in cur:
                     if dp_dir.startswith("//") or dp_fn.startswith("//"):
                         dp_dir, dp_fn = s3dec(dp_dir, dp_fn)
 
@@ -1065,6 +1086,8 @@ class Up2k(object):
                             "ptop": cj["ptop"],
                             "size": dsize,
                             "lmod": dtime,
+                            "addr": ip,
+                            "at": at,
                             "hash": [],
                             "need": [],
                         }
@@ -1119,7 +1142,8 @@ class Up2k(object):
                             self._symlink(src, dst)
 
                         if cur:
-                            a = [cj[x] for x in "prel name lmod size".split()]
+                            a = [cj[x] for x in "prel name lmod size addr".split()]
+                            a += [cj.get("at") or time.time()]
                             self.db_add(cur, wark, *a)
                             cur.connection.commit()
 
@@ -1266,20 +1290,21 @@ class Up2k(object):
                 a = [dst, job["size"], (int(time.time()), int(job["lmod"]))]
                 self.lastmod_q.put(a)
 
-            a = [job[x] for x in "ptop wark prel name lmod size".split()]
+            a = [job[x] for x in "ptop wark prel name lmod size addr".split()]
+            a += [job.get("at") or time.time()]
             if self.idx_wark(*a):
                 del self.registry[ptop][wark]
                 # in-memory registry is reserved for unfinished uploads
 
         return ret, dst
 
-    def idx_wark(self, ptop, wark, rd, fn, lmod, sz):
+    def idx_wark(self, ptop, wark, rd, fn, lmod, sz, ip, at):
         cur = self.cur.get(ptop)
         if not cur:
             return False
 
         self.db_rm(cur, rd, fn)
-        self.db_add(cur, wark, rd, fn, lmod, sz)
+        self.db_add(cur, wark, rd, fn, lmod, sz, ip, at)
         cur.connection.commit()
 
         if "e2t" in self.flags[ptop]:
@@ -1295,53 +1320,99 @@ class Up2k(object):
         except:
             db.execute(sql, s3enc(self.mem_cur, rd, fn))
 
-    def db_add(self, db, wark, rd, fn, ts, sz):
-        sql = "insert into up values (?,?,?,?,?)"
-        v = (wark, int(ts), sz, rd, fn)
+    def db_add(self, db, wark, rd, fn, ts, sz, ip, at):
+        sql = "insert into up values (?,?,?,?,?,?,?)"
+        v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
         try:
             db.execute(sql, v)
         except:
             rd, fn = s3enc(self.mem_cur, rd, fn)
-            v = (wark, int(ts), sz, rd, fn)
+            v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
             db.execute(sql, v)
 
-    def handle_rm(self, uname, vpath):
-        permsets = [[True, False, False, True]]
-        vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+    def handle_rm(self, uname, ip, vpaths):
+        n_files = 0
+        ok = {}
+        ng = {}
+        for vp in vpaths:
+            a, b, c = self._handle_rm(uname, ip, vp)
+            n_files += a
+            for k in b:
+                ok[k] = 1
+            for k in c:
+                ng[k] = 1
+
+        ng = {k: 1 for k in ng if k not in ok}
+        ok = len(ok)
+        ng = len(ng)
+
+        return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
+
+    def _handle_rm(self, uname, ip, vpath):
+        try:
+            permsets = [[True, False, False, True]]
+            vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+            unpost = False
+        except:
+            # unpost with missing permissions? try read+write and verify with db
+            if not self.args.unpost:
+                raise Pebkac(400, "the unpost feature was disabled by server config")
+
+            unpost = True
+            permsets = [[True, True]]
+            vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+            _, _, _, _, dip, dat = self._find_from_vpath(vn.realpath, rem)
+
+            m = "you cannot delete this: "
+            if not dip:
+                m += "file not found"
+            elif dip != ip:
+                m += "not uploaded by (You)"
+            elif dat < time.time() - self.args.unpost:
+                m += "uploaded too long ago"
+            else:
+                m = None
+
+            if m:
+                raise Pebkac(400, m)
+
         ptop = vn.realpath
-        atop = vn.canonical(rem)
+        atop = vn.canonical(rem, False)
         adir, fn = os.path.split(atop)
         st = bos.lstat(atop)
         scandir = not self.args.no_scandir
         if stat.S_ISLNK(st.st_mode) or stat.S_ISREG(st.st_mode):
             dbv, vrem = self.asrv.vfs.get(vpath, uname, *permsets[0])
             dbv, vrem = dbv.get_dbv(vrem)
-            g = [[dbv, vrem, os.path.dirname(vpath), adir, [[fn, 0]], [], []]]
+            voldir = vsplit(vrem)[0]
+            vpath_dir = vsplit(vpath)[0]
+            g = [[dbv, voldir, vpath_dir, adir, [[fn, 0]], [], []]]
         else:
             g = vn.walk("", rem, [], uname, permsets, True, scandir, True)
+            if unpost:
+                raise Pebkac(400, "cannot unpost folders")
 
         n_files = 0
         for dbv, vrem, _, adir, files, rd, vd in g:
             for fn in [x[0] for x in files]:
                 n_files += 1
                 abspath = os.path.join(adir, fn)
-                vpath = "{}/{}".format(vrem, fn).strip("/")
+                volpath = "{}/{}".format(vrem, fn).strip("/")
+                vpath = "{}/{}".format(dbv.vpath, volpath).strip("/")
                 self.log("rm {}\n  {}".format(vpath, abspath))
-                _ = dbv.get(vrem, uname, *permsets[0])
+                _ = dbv.get(volpath, uname, *permsets[0])
                 with self.mutex:
                     try:
                         ptop = dbv.realpath
-                        cur, wark, _, _ = self._find_from_vpath(ptop, vrem)
-                        self._forget_file(ptop, vpath, cur, wark)
+                        cur, wark, _, _, _, _ = self._find_from_vpath(ptop, volpath)
+                        self._forget_file(ptop, volpath, cur, wark, True)
                     finally:
                         cur.connection.commit()
 
                 bos.unlink(abspath)
 
         rm = rmdirs(self.log_func, scandir, True, atop)
-        ok = len(rm[0])
-        ng = len(rm[1])
-        return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
+        return n_files, rm[0], rm[1]
 
     def handle_mv(self, uname, svp, dvp):
         svn, srem = self.asrv.vfs.get(svp, uname, True, False, True)
@@ -1351,9 +1422,10 @@ class Up2k(object):
         if not srem:
             raise Pebkac(400, "mv: cannot move a mountpoint")
 
-        st = bos.stat(sabs)
-        if stat.S_ISREG(st.st_mode):
-            return self._mv_file(uname, svp, dvp)
+        st = bos.lstat(sabs)
+        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
+            with self.mutex:
+                return self._mv_file(uname, svp, dvp)
 
         jail = svn.get_dbv(srem)[0]
         permsets = [[True, False, True]]
@@ -1378,7 +1450,8 @@ class Up2k(object):
                     raise Pebkac(500, "mv: bug at {}, top {}".format(svpf, svp))
 
                 dvpf = dvp + svpf[len(svp) :]
-                self._mv_file(uname, svpf, dvpf)
+                with self.mutex:
+                    self._mv_file(uname, svpf, dvpf)
 
         rmdirs(self.log_func, scandir, True, sabs)
         return "k"
@@ -1411,7 +1484,7 @@ class Up2k(object):
             self.need_rescan[dvn.vpath] = 1
             return "k"
 
-        c1, w, ftime, fsize = self._find_from_vpath(svn.realpath, srem)
+        c1, w, ftime, fsize, ip, at = self._find_from_vpath(svn.realpath, srem)
         c2 = self.cur.get(dvn.realpath)
 
         if ftime is None:
@@ -1420,15 +1493,15 @@ class Up2k(object):
             fsize = st.st_size
 
         if w:
-            if c2:
+            if c2 and c2 != c1:
                 self._copy_tags(c1, c2, w)
 
-            self._forget_file(svn.realpath, srem, c1, w)
+            self._forget_file(svn.realpath, srem, c1, w, c1 != c2)
             self._relink(w, svn.realpath, srem, dabs)
             c1.connection.commit()
 
             if c2:
-                self.db_add(c2, w, drd, dfn, ftime, fsize)
+                self.db_add(c2, w, drd, dfn, ftime, fsize, ip, at)
                 c2.connection.commit()
         else:
             self.log("not found in src db: [{}]".format(svp))
@@ -1452,7 +1525,7 @@ class Up2k(object):
             return None, None
 
         rd, fn = vsplit(vrem)
-        q = "select w, mt, sz from up where rd=? and fn=? limit 1"
+        q = "select w, mt, sz, ip, at from up where rd=? and fn=? limit 1"
         try:
             c = cur.execute(q, (rd, fn))
         except:
@@ -1460,20 +1533,24 @@ class Up2k(object):
 
         hit = c.fetchone()
         if hit:
-            wark, ftime, fsize = hit
-            return cur, wark, ftime, fsize
-        return cur, None, None, None
+            wark, ftime, fsize, ip, at = hit
+            return cur, wark, ftime, fsize, ip, at
+        return cur, None, None, None, None, None
 
-    def _forget_file(self, ptop, vrem, cur, wark):
+    def _forget_file(self, ptop, vrem, cur, wark, drop_tags):
         """forgets file in db, fixes symlinks, does not delete"""
         srd, sfn = vsplit(vrem)
         self.log("forgetting {}".format(vrem))
         if wark:
             self.log("found {} in db".format(wark))
-            self._relink(wark, ptop, vrem, None)
+            if drop_tags:
+                if self._relink(wark, ptop, vrem, None):
+                    drop_tags = False
+
+            if drop_tags:
+                q = "delete from mt where w=?"
+                cur.execute(q, (wark[:16],))
 
-            q = "delete from mt where w=?"
-            cur.execute(q, (wark[:16],))
             self.db_rm(cur, srd, sfn)
 
         reg = self.registry.get(ptop)
@@ -1510,7 +1587,7 @@ class Up2k(object):
                     self.log("found {} dupe: [{}] {}".format(wark, ptop, dvrem))
 
         if not dupes:
-            return
+            return 0
 
         full = {}
         links = {}
@@ -1526,7 +1603,7 @@ class Up2k(object):
             # deleting final remaining full copy; swap it with a symlink
             slabs = list(sorted(links.keys()))[0]
             ptop, rem = links.pop(slabs)
-            self.log("linkswap [{}] and [{}]".format(sabs, dabs))
+            self.log("linkswap [{}] and [{}]".format(sabs, slabs))
             bos.unlink(slabs)
             bos.rename(sabs, slabs)
             self._symlink(slabs, sabs, False)
@@ -1547,6 +1624,8 @@ class Up2k(object):
 
             self._symlink(dabs, alink, False)
 
+        return len(full) + len(links)
+
     def _get_wark(self, cj):
         if len(cj["name"]) > 1024 or len(cj["hash"]) > 512 * 1024:  # 16TiB
             raise Pebkac(400, "name or numchunks not according to spec")
@@ -1753,7 +1832,7 @@ class Up2k(object):
                 self.n_hashq -= 1
             # self.log("hashq {}".format(self.n_hashq))
 
-            ptop, rd, fn = self.hashq.get()
+            ptop, rd, fn, ip, at = self.hashq.get()
             # self.log("hashq {} pop {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
             if "e2d" not in self.flags[ptop]:
                 continue
@@ -1764,12 +1843,12 @@ class Up2k(object):
             hashes = self._hashlist_from_file(abspath)
             wark = up2k_wark_from_hashlist(self.salt, inf.st_size, hashes)
             with self.mutex:
-                self.idx_wark(ptop, wark, rd, fn, inf.st_mtime, inf.st_size)
+                self.idx_wark(ptop, wark, rd, fn, inf.st_mtime, inf.st_size, ip, at)
 
-    def hash_file(self, ptop, flags, rd, fn):
+    def hash_file(self, ptop, flags, rd, fn, ip, at):
         with self.mutex:
             self.register_vpath(ptop, flags)
-            self.hashq.put([ptop, rd, fn])
+            self.hashq.put([ptop, rd, fn, ip, at])
             self.n_hashq += 1
         # self.log("hashq {} push {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
 

copyparty/util.py

@@ -1063,6 +1063,9 @@ def statdir(logger, scandir, lstat, top):
 
 
 def rmdirs(logger, scandir, lstat, top):
+    if not os.path.exists(fsenc(top)) or not os.path.isdir(fsenc(top)):
+        top = os.path.dirname(top)
+    
     dirs = statdir(logger, scandir, lstat, top)
     dirs = [x[0] for x in dirs if stat.S_ISDIR(x[1].st_mode)]
     dirs = [os.path.join(top, x) for x in dirs]

copyparty/web/browser.css

@@ -22,9 +22,6 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 pre, code, tt {
 	font-family: monospace, monospace;
 }
@@ -49,7 +46,7 @@ pre, code, tt {
 	transition: opacity 0.14s, height 0.14s, padding 0.14s;
 }
 #toast {
-	top: 1.4em;
+	bottom: 5em;
 	right: -1em;
 	line-height: 1.5em;
 	padding: 1em 1.3em;
@@ -78,6 +75,9 @@ pre, code, tt {
 	border-radius: .5em 0 0 .5em;
 	transition: left .3s, width .3s, padding .3s, opacity .3s;
 }
+#toast pre {
+	margin: 0;
+}
 #toast.vis {
 	right: 1.3em;
 	transform: unset;
@@ -807,6 +807,7 @@ input.eq_gain {
 #wrap {
 	margin-top: 2em;
 	min-height: 90vh;
+	padding-bottom: 5em;
 }
 #tree {
 	display: none;
@@ -952,7 +953,8 @@ input.eq_gain {
 	color: #300;
 	background: #fea;
 }
-.opwide {
+.opwide,
+#op_unpost {
 	max-width: none;
 	margin-right: 1.5em;
 }
@@ -1054,10 +1056,78 @@ html.light #ggrid a:hover {
 	color: #015;
 	box-shadow: 0 .1em .5em #aaa;
 }
+#op_unpost {
+	padding: 1em;
+}
+#op_unpost td {
+	padding: .2em .4em;
+}
+#op_unpost a {
+	margin: 0;
+	padding: 0;
+}
+#rui {
+	position: fixed;
+	top: 0;
+	left: 0;
+	width: calc(100% - 2em);
+	height: auto;
+	overflow: auto;
+	max-height: calc(100% - 2em);
+	border-bottom: .5em solid #999;
+	background: #333;
+	padding: 1em;
+	z-index: 765;
+}
+html.light #rui {
+	color: #fff;
+}
+#rui div+div {
+	margin-top: 1em;
+}
+#rui table {
+	width: 100%;
+	border-collapse: collapse;
+}
+#rui td+td {
+	padding: .2em 0 .2em .5em;
+}
+#rn_vadv input {
+	font-family: monospace, monospace;
+}
+#rui td+td,
+#rui td input[type="text"] {
+	width: 100%;
+}
+#rn_f.m td:first-child {
+	white-space: nowrap;
+}
+#rn_f.m td+td {
+	width: 50%;
+}
+#rn_f .err td {
+	background: #c00;
+}
+#rn_f .err input[readonly] {
+	background: #600;
+	color: #fc0;
+}
+#rui input[readonly] {
+	color: #fff;
+	background: #444;
+	border: 1px solid #777;
+	padding: .2em .25em;
+}
+#rui h1 {
+	margin: 0 0 .3em 0;
+	padding: 0;
+	font-size: 1.5em;
+}
 #pvol,
 #barbuf,
 #barpos,
 #u2conf label,
+#rui label,
 #ops {
 	-webkit-user-select: none;
 	-moz-user-select: none;

copyparty/web/browser.html

@@ -59,6 +59,8 @@
 		</form>
 	</div>
 
+	<div id="op_unpost" class="opview opbox"></div>
+
 	<div id="op_up2k" class="opview"></div>
 
 	<div id="op_cfg" class="opview opbox opwide"></div>
@@ -123,11 +125,12 @@
 	<script>
 		var acct = "{{ acct }}",
 			perms = {{ perms }},
-			tag_order_cfg = {{ tag_order }},
+			def_hcols = {{ def_hcols|tojson }},
 			have_up2k_idx = {{ have_up2k_idx|tojson }},
 			have_tags_idx = {{ have_tags_idx|tojson }},
 			have_mv = {{ have_mv|tojson }},
 			have_del = {{ have_del|tojson }},
+			have_unpost = {{ have_unpost|tojson }},
 			have_zip = {{ have_zip|tojson }};
 	</script>
 	<script src="/.cpr/util.js?_={{ ts }}"></script>

copyparty/web/browser.js

@@ -12,6 +12,7 @@ ebi('ops').innerHTML = (
 	'<a href="#" data-dest="" tt="close submenu">---</a>\n' +
 	(have_up2k_idx ? (
 		'<a href="#" data-perm="read" data-dest="search" tt="search for files by attributes, path/name, music tags, or any combination of those.$N$N&lt;code&gt;foo bar&lt;/code&gt; = must contain both foo and bar,$N&lt;code&gt;foo -bar&lt;/code&gt; = must contain foo but not bar,$N&lt;code&gt;^yana .opus$&lt;/code&gt; = must start with yana and have the opus extension">🔎</a>\n' +
+		(have_del && have_unpost ? '<a href="#" data-dest="unpost" tt="unpost: delete your recent uploads">🧯</a>\n' : '') +
 		'<a href="#" data-dest="up2k" tt="up2k: upload files (if you have write-access) or toggle into the search-mode and drag files onto the search button to see if they exist somewhere on the server">🚀</a>\n'
 	) : (
 		'<a href="#" data-perm="write" data-dest="up2k" tt="up2k: upload files with resume support (close your browser and drop the same files in later)">🚀</a>\n'
@@ -132,6 +133,7 @@ ebi('op_cfg').innerHTML = (
 	'	<div>\n' +
 	'		<a id="tooltips" class="tgl btn" href="#" tt="◔ ◡ ◔">ℹ️ tooltips</a>\n' +
 	'		<a id="lightmode" class="tgl btn" href="#">☀️ lightmode</a>\n' +
+	'		<a id="dotfiles" class="tgl btn" href="#" tt="show hidden files (if server permits)">dotfiles</a>\n' +
 	'		<a id="griden" class="tgl btn" href="#" tt="toggle icons or list-view$NHotkey: G">田 the grid</a>\n' +
 	'		<a id="thumbs" class="tgl btn" href="#" tt="in icon view, toggle icons or thumbnails$NHotkey: T">🖼️ thumbs</a>\n' +
 	'	</div>\n' +
@@ -213,17 +215,6 @@ function goto(dest) {
 }
 
 
-(function () {
-	goto();
-	var op = sread('opmode');
-	if (op !== null && op !== '.')
-		try {
-			goto(op);
-		}
-		catch (ex) { }
-})();
-
-
 var have_webp = null;
 (function () {
 	var img = new Image();
@@ -531,15 +522,14 @@ var mp = new MPlayer();
 makeSortable(ebi('files'), mp.read_order.bind(mp));
 
 
-function get_np() {
+function ft2dict(tr) {
 	var th = ebi('files').tHead.rows[0].cells,
-		tr = QS('#files tr.play').cells,
 		rv = [],
 		ra = [],
 		rt = {};
 
 	for (var a = 1, aa = th.length; a < aa; a++) {
-		var tv = tr[a].textContent,
+		var tv = tr.cells[a].textContent,
 			tk = a == 1 ? 'file' : th[a].getAttribute('name').split('/').slice(-1)[0],
 			vis = th[a].className.indexOf('min') === -1;
 
@@ -550,6 +540,12 @@ function get_np() {
 		rt[tk] = tv;
 	}
 	return [rt, rv, ra];
+}
+
+
+function get_np() {
+	var tr = QS('#files tr.play');
+	return ft2dict(tr);
 };
 
 
@@ -1464,6 +1460,105 @@ function play_linked() {
 })();
 
 
+
+function fmt_ren(re, md, fmt) {
+	var ptr = 0;
+	function dive(stop_ch) {
+		var ret = '', ng = 0;
+		while (ptr < fmt.length) {
+			var dbg = fmt.slice(ptr),
+				ch = fmt[ptr++];
+
+			if (ch == '\\') {
+				ret += fmt[ptr++];
+				continue;
+			}
+
+			if (ch == ')' || ch == ']' || ch == stop_ch)
+				return [ng, ret];
+
+			if (ch == '[') {
+				var r2 = dive();
+				if (r2[0] == 0)
+					ret += r2[1];
+			}
+			else if (ch == '(') {
+				var end = fmt.indexOf(')', ptr);
+				if (end < 0)
+					throw 'the ( was never closed: ' + fmt.slice(0, ptr);
+
+				var arg = fmt.slice(ptr, end), v = null;
+				ptr = end + 1;
+
+				if (arg != parseInt(arg))
+					v = md[arg];
+				else {
+					arg = parseInt(arg);
+					if (arg >= re.length)
+						throw 'matching group ' + arg + ' exceeds ' + (re.length - 0);
+
+					v = re[arg];
+				}
+
+				if (v !== null && v !== undefined)
+					ret += v;
+				else
+					ng++;
+			}
+			else if (ch == '$') {
+				ch = fmt[ptr++];
+				var end = fmt.indexOf('(', ptr);
+				if (end < 0)
+					throw 'no function name after the $ here: ' + fmt.slice(0, ptr);
+
+				var fun = fmt.slice(ptr - 1, end);
+				ptr = end + 1;
+
+				if (fun == "lpad") {
+					var str = dive(',')[1];
+					var len = dive(',')[1];
+					var chr = dive()[1];
+					if (!len || !chr)
+						throw 'invalid arguments to ' + fun;
+
+					if (!str.length)
+						ng += 1;
+
+					while (str.length < len)
+						str = chr + str;
+
+					ret += str;
+				}
+				else if (fun == "rpad") {
+					var str = dive(',')[1];
+					var len = dive(',')[1];
+					var chr = dive()[1];
+					if (!len || !chr)
+						throw 'invalid arguments to ' + fun;
+
+					if (!str.length)
+						ng += 1;
+
+					while (str.length < len)
+						str += chr;
+
+					ret += str;
+				}
+				else throw 'function not implemented: "' + fun + '"';
+			}
+			else ret += ch;
+		}
+		return [ng, ret];
+	}
+	try {
+		return [true, dive()[1]];
+	}
+	catch (ex) {
+		return [false, ex];
+	}
+}
+
+
 var fileman = (function () {
 	var bren = ebi('fren'),
 		bdel = ebi('fdel'),
@@ -1478,10 +1573,10 @@ var fileman = (function () {
 		if (r.clip === null)
 			r.clip = jread('fman_clip', []);
 
-		var sel = msel.getsel();
-		clmod(bren, 'en', sel.length == 1);
-		clmod(bdel, 'en', sel.length);
-		clmod(bcut, 'en', sel.length);
+		var nsel = msel.getsel().length;
+		clmod(bren, 'en', nsel == 1);
+		clmod(bdel, 'en', nsel);
+		clmod(bcut, 'en', nsel);
 		clmod(bpst, 'en', r.clip && r.clip.length);
 		bren.style.display = have_mv && has(perms, 'write') && has(perms, 'move') ? '' : 'none';
 		bdel.style.display = have_del && has(perms, 'delete') ? '' : 'none';
@@ -1497,39 +1592,289 @@ var fileman = (function () {
 			return toast.err(3, 'cannot rename:\nyou do not have “move” permission in this folder');
 
 		var sel = msel.getsel();
-		if (sel.length !== 1)
-			return toast.err(3, 'select exactly 1 item to rename');
+		if (!sel.length)
+			return toast.err(3, 'select at least one item to rename');
 
-		var src = sel[0].vp;
-		if (src.endsWith('/'))
-			src = src.slice(0, -1);
+		var f = [],
+			base = vsplit(sel[0].vp)[0],
+			mkeys;
 
-		var vsp = vsplit(src),
-			base = vsp[0],
-			ofn = vsp[1];
+		for (var a = 0; a < sel.length; a++) {
+			var vp = sel[a].vp;
+			if (vp.endsWith('/'))
+				vp = vp.slice(0, -1);
 
-		var fn = prompt('new filename:', ofn);
-		if (!fn || fn == ofn)
-			return toast.warn(1, 'rename aborted');
+			var vsp = vsplit(vp);
+			if (base != vsp[0])
+				return toast.err(0, 'bug:\n' + base + '\n' + vsp[0]);
 
-		var dst = base + fn;
+			var vars = ft2dict(ebi(sel[a].id).closest('tr'));
+			mkeys = vars[1].concat(vars[2]);
 
-		function rename_cb() {
-			if (this.readyState != XMLHttpRequest.DONE)
-				return;
+			var md = vars[0];
+			for (var k in md) {
+				if (!md.hasOwnProperty(k))
+					continue;
 
-			if (this.status !== 200) {
-				var msg = this.responseText;
-				toast.err(9, 'rename failed:\n' + msg);
-				return;
+				md[k.toLowerCase()] = md[k];
+				k = k.toLowerCase();
+
+				if (k.startsWith('.'))
+					md[k.slice(1)] = md[k];
 			}
-			toast.ok(2, 'rename OK');
-			treectl.goto(get_evpath());
+			md.t = md.ext;
+			md.date = md.ts;
+			md.size = md.sz;
+
+			f.push({
+				"src": vp,
+				"ofn": uricom_dec(vsp[1])[0],
+				"md": vars[0],
+				"ok": true
+			});
+		}
+
+		var rui = ebi('rui');
+		if (!rui) {
+			rui = mknod('div');
+			rui.setAttribute('id', 'rui');
+			document.body.appendChild(rui);
+		}
+
+		var html = sel.length > 1 ? ['<div>'] : [
+			'<div>',
+			'<button class="rn_dec" n="0" tt="may fix some cases of broken filenames">url-decode</button>',
+			'//',
+			'<button class="rn_reset" n="0" tt="reset modified filenames back to the original ones">↺ reset</button>'
+		];
+
+		html = html.concat([
+			'<button id="rn_cancel" tt="abort and close this window">❌ cancel</button>',
+			'<button id="rn_apply">✅ apply rename</button>',
+			'<a id="rn_adv" class="tgl btn" href="#" tt="batch / metadata / pattern renaming">advanced</a>',
+			'<a id="rn_case" class="tgl btn" href="#" tt="case-sensitive regex">case</a>',
+			'</div>',
+			'<div id="rn_vadv"><table>',
+			'<tr><td>regex</td><td><input type="text" id="rn_re" tt="regex search pattern to apply to original filenames; capturing groups can be referenced in the format field below like &lt;code&gt;(1)&lt;/code&gt; and &lt;code&gt;(2)&lt;/code&gt; and so on" /></td></tr>',
+			'<tr><td>format</td><td><input type="text" id="rn_fmt" tt="inspired by foobar2000:$N&lt;code&gt;(title)&lt;/code&gt; is replaced by song title,$N&lt;code&gt;[(artist) - ](title)&lt;/code&gt; skips the first part if artist is blank$N&lt;code&gt;$lpad((tn),2,0)&lt;/code&gt; pads tracknumber to 2 digits" /></td></tr>',
+			'<tr><td>preset</td><td><select id="rn_pre"></select>',
+			'<button id="rn_pdel">❌ delete</button>',
+			'<button id="rn_pnew">💾 save as</button>',
+			'</td></tr>',
+			'</table></div>'
+		]);
+
+		if (sel.length == 1)
+			html.push(
+				'<div><table id="rn_f">\n' +
+				'<tr><td>old:</td><td><input type="text" id="rn_old" n="0" readonly /></td></tr>\n' +
+				'<tr><td>new:</td><td><input type="text" id="rn_new" n="0" /></td></tr>');
+		else {
+			html.push(
+				'<div><table id="rn_f" class="m">' +
+				'<tr><td></td><td>new name</td><td>old name</td></tr>');
+			for (var a = 0; a < f.length; a++)
+				html.push(
+					'<tr><td>' +
+					'<button class="rn_dec" n="' + a + '">decode</button>',
+					'<button class="rn_reset" n="' + a + '">↺ reset</button></td>',
+					'<td><input type="text" id="rn_new" n="' + a + '" /></td>' +
+					'<td><input type="text" id="rn_old" n="' + a + '" readonly /></td></tr>');
+		}
+		html.push('</table></div>');
+
+		if (sel.length == 1) {
+			html.push('<div><p style="margin:.6em 0">tags for the selected file (read-only, just for reference):</p><table>');
+			for (var a = 0; a < mkeys.length; a++)
+				html.push('<tr><td>' + esc(mkeys[a]) + '</td><td><input type="text" readonly value="' + esc(f[0].md[mkeys[a]]) + '" /></td></tr>');
+
+			html.push('</table></div>');
+		}
+
+		rui.innerHTML = html.join('\n');
+		for (var a = 0; a < f.length; a++) {
+			var k = '[n="' + a + '"]';
+			f[a].iold = QS('#rn_old' + k);
+			f[a].inew = QS('#rn_new' + k);
+			f[a].inew.value = f[a].iold.value = f[a].ofn;
+
+			(function (a) {
+				f[a].inew.onkeydown = function (e) {
+					rn_ok(a, true);
+
+					if (e.key == 'Escape')
+						return rn_cancel();
+
+					if (e.key == 'Enter')
+						return rn_apply();
+				};
+				QS('.rn_dec' + k).onclick = function () {
+					f[a].inew.value = uricom_dec(f[a].inew.value)[0];
+				};
+				QS('.rn_reset' + k).onclick = function () {
+					rn_reset(a);
+				};
+			})(a);
+		}
+		rn_reset(0);
+		tt.att(rui);
+
+		var adv = bcfg_get('rn_adv', false),
+			cs = bcfg_get('rn_case', false);
+
+		function sadv() {
+			ebi('rn_vadv').style.display = ebi('rn_case').style.display = adv ? '' : 'none';
+		}
+		sadv();
+
+		function rn_ok(n, ok) {
+			f[n].ok = ok;
+			clmod(f[n].inew.closest('tr'), 'err', !ok);
+		}
+
+		function rn_reset(n) {
+			f[n].inew.value = f[n].iold.value = f[n].ofn;
+			f[n].inew.focus();
+			f[n].inew.setSelectionRange(0, f[n].inew.value.lastIndexOf('.'), "forward");
+		}
+		function rn_cancel() {
+			rui.parentNode.removeChild(rui);
+		}
+
+		ebi('rn_cancel').onclick = rn_cancel;
+		ebi('rn_apply').onclick = rn_apply;
+		ebi('rn_adv').onclick = function () {
+			adv = !adv;
+			bcfg_set('rn_adv', adv);
+			sadv();
+		};
+		ebi('rn_case').onclick = function () {
+			cs = !cs;
+			bcfg_set('rn_case', cs);
+		};
+
+		var ire = ebi('rn_re'),
+			ifmt = ebi('rn_fmt'),
+			ipre = ebi('rn_pre'),
+			idel = ebi('rn_pdel'),
+			inew = ebi('rn_pnew'),
+			defp = '$lpad((tn),2,0). [(artist) - ](title).(ext)';
+
+		var presets = {};
+		presets[defp] = ['', defp];
+		presets = jread("rn_pre", presets);
+
+		function spresets() {
+			var keys = Object.keys(presets), o;
+			keys.sort();
+			ipre.innerHTML = '<option value=""></option>';
+			for (var a = 0; a < keys.length; a++) {
+				o = mknod('option');
+				o.setAttribute('value', keys[a]);
+				o.textContent = keys[a];
+				ipre.appendChild(o);
+			}
+		}
+		inew.onclick = function () {
+			var name = prompt('provide a name for your new preset', ifmt.value);
+			if (!name)
+				return toast.warn(3, 'aborted');
+
+			presets[name] = [ire.value, ifmt.value];
+			jwrite('rn_pre', presets);
+			spresets();
+			ipre.value = name;
+		};
+		idel.onclick = function () {
+			delete presets[ipre.value];
+			jwrite('rn_pre', presets);
+			spresets();
+		};
+		ipre.oninput = function () {
+			var cfg = presets[ipre.value];
+			if (cfg) {
+				ire.value = cfg[0];
+				ifmt.value = cfg[1];
+			}
+			ifmt.oninput();
+		};
+		spresets();
+
+		ire.onkeydown = ifmt.onkeydown = function (e) {
+			if (e.key == 'Escape')
+				return rn_cancel();
+
+			if (e.key == 'Enter')
+				return rn_apply();
+		};
+
+		ire.oninput = ifmt.oninput = function (e) {
+			var ptn = ire.value,
+				fmt = ifmt.value,
+				re = null;
+
+			if (!fmt)
+				return;
+
+			try {
+				if (ptn)
+					re = new RegExp(ptn, cs ? 'i' : '');
+			}
+			catch (ex) {
+				return toast.err(5, 'invalid regex:\n' + ex);
+			}
+			toast.hide();
+
+			for (var a = 0; a < f.length; a++) {
+				var m = re ? re.exec(f[a].ofn) : null,
+					ok, txt = '';
+
+				if (re && !m) {
+					txt = 'regex did not match';
+					ok = false;
+				}
+				else {
+					var ret = fmt_ren(m, f[a].md, fmt);
+					ok = ret[0];
+					txt = ret[1];
+				}
+				rn_ok(a, ok);
+				f[a].inew.value = (ok ? '' : 'ERROR: ') + txt;
+			}
+		};
+
+		function rn_apply() {
+			while (f.length && (!f[0].ok || f[0].ofn == f[0].inew.value))
+				f.shift();
+
+			if (!f.length) {
+				toast.ok(2, 'rename OK');
+				treectl.goto(get_evpath());
+				return rn_cancel();
+			}
+
+			toast.inf(0, 'renaming ' + f.length + ' items\n\n' + f[0].ofn);
+			var dst = base + uricom_enc(f[0].inew.value, false);
+
+			function rename_cb() {
+				if (this.readyState != XMLHttpRequest.DONE)
+					return;
+
+				if (this.status !== 200) {
+					var msg = this.responseText;
+					toast.err(9, 'rename failed:\n' + msg);
+					return;
+				}
+
+				f.shift().inew.value = '( OK )';
+				return rn_apply();
+			}
+
+			var xhr = new XMLHttpRequest();
+			xhr.open('GET', f[0].src + '?move=' + dst, true);
+			xhr.onreadystatechange = rename_cb;
+			xhr.send();
 		}
-		var xhr = new XMLHttpRequest();
-		xhr.open('GET', src + '?move=' + dst, true);
-		xhr.onreadystatechange = rename_cb;
-		xhr.send();
 	};
 
 	r.delete = function (e) {
@@ -1621,7 +1966,7 @@ var fileman = (function () {
 			links = QSA('#files tbody td:nth-child(2) a');
 
 		for (var a = 0, aa = links.length; a < aa; a++)
-			indir.push(links[a].getAttribute('name'));
+			indir.push(vsplit(links[a].getAttribute('href'))[1]);
 
 		for (var a = 0; a < r.clip.length; a++) {
 			var found = false;
@@ -1636,12 +1981,12 @@ var fileman = (function () {
 		}
 
 		if (exists.length)
-			alert('these ' + exists.length + ' items cannot be pasted here (names already exist):\n\n' + exists.join('\n'));
+			alert('these ' + exists.length + ' items cannot be pasted here (names already exist):\n\n' + uricom_adec(exists).join('\n'));
 
 		if (!req.length)
 			return;
 
-		if (!confirm('paste these ' + req.length + ' items here?\n\n' + req.join('\n')))
+		if (!confirm('paste these ' + req.length + ' items here?\n\n' + uricom_adec(req).join('\n')))
 			return;
 
 		function paster() {
@@ -1654,7 +1999,7 @@ var fileman = (function () {
 				r.tx(srcdir);
 				return;
 			}
-			toast.inf(0, 'pasting ' + (req.length + 1) + ' items\n\n' + vp);
+			toast.inf(0, 'pasting ' + (req.length + 1) + ' items\n\n' + uricom_dec(vp)[0]);
 
 			var dst = get_evpath() + vp.split('/').slice(-1)[0];
 
@@ -2425,6 +2770,7 @@ var treectl = (function () {
 		prev_atop = null,
 		prev_winh = null,
 		dyn = bcfg_get('dyntree', true),
+		dots = bcfg_get('dotfiles', false),
 		treesz = icfg_get('treesz', 16);
 
 	treesz = Math.min(Math.max(treesz, 4), 50);
@@ -2543,7 +2889,7 @@ var treectl = (function () {
 		xhr.dst = dst;
 		xhr.rst = rst;
 		xhr.ts = Date.now();
-		xhr.open('GET', dst + '?tree=' + top, true);
+		xhr.open('GET', dst + '?tree=' + top + (dots ? '&dots' : ''), true);
 		xhr.onreadystatechange = recvtree;
 		xhr.send();
 		enspin('#tree');
@@ -2647,7 +2993,7 @@ var treectl = (function () {
 		xhr.top = url;
 		xhr.hpush = hpush;
 		xhr.ts = Date.now();
-		xhr.open('GET', xhr.top + '?ls', true);
+		xhr.open('GET', xhr.top + '?ls' + (dots ? '&dots' : ''), true);
 		xhr.onreadystatechange = recvls;
 		xhr.send();
 		if (hpush)
@@ -2784,6 +3130,13 @@ var treectl = (function () {
 		return ret;
 	}
 
+	function tdots(e) {
+		ev(e);
+		dots = !dots;
+		bcfg_set('dotfiles', dots);
+		treectl.goto(get_evpath());
+	}
+
 	function dyntree(e) {
 		ev(e);
 		dyn = !dyn;
@@ -2803,6 +3156,7 @@ var treectl = (function () {
 
 	ebi('entree').onclick = treectl.entree;
 	ebi('detree').onclick = treectl.detree;
+	ebi('dotfiles').onclick = tdots;
 	ebi('dyntree').onclick = dyntree;
 	ebi('twig').onclick = scaletree;
 	ebi('twobytwo').onclick = scaletree;
@@ -2849,7 +3203,7 @@ function apply_perms(newperms) {
 
 	var axs = [],
 		aclass = '>',
-		chk = ['read', 'write', 'rename', 'delete'];
+		chk = ['read', 'write', 'move', 'delete'];
 
 	for (var a = 0; a < chk.length; a++)
 		if (has(perms, chk[a]))
@@ -2965,6 +3319,8 @@ var filecols = (function () {
 			"q": "quality / bitrate",
 			"Ac": "audio codec",
 			"Vc": "video codec",
+			"Ahash": "audio checksum",
+			"Vhash": "video checksum",
 			"Res": "resolution",
 			"T": "filetype",
 			"aq": "audio quality / bitrate",
@@ -2976,6 +3332,21 @@ var filecols = (function () {
 			"hz": "sample rate"
 		};
 
+	if (JSON.stringify(def_hcols) != sread('hfilecols')) {
+		console.log("applying default hidden-cols");
+		jwrite('hfilecols', def_hcols);
+		for (var a = 0; a < def_hcols.length; a++) {
+			var t = def_hcols[a];
+			t = t.slice(0, 1).toUpperCase() + t.slice(1);
+			if (t.startsWith("."))
+				t = t.slice(1);
+
+			if (hidden.indexOf(t) == -1)
+				hidden.push(t);
+		}
+		jwrite("filecols", hidden);
+	}
+
 	var add_btns = function () {
 		var ths = QSA('#files th>span');
 		for (var a = 0, aa = ths.length; a < aa; a++) {
@@ -3329,13 +3700,11 @@ var msel = (function () {
 			item.id = links[a].getAttribute('id');
 			item.sel = links[a].closest('tr').classList.contains('sel');
 			item.vp = href.indexOf('/') !== -1 ? href : vbase + href;
-			item.name = href.split('/').slice(-1);
 
 			r.all.push(item);
 			if (item.sel)
 				r.sel.push(item);
 
-			links[a].setAttribute('name', item.name);
 			links[a].closest('tr').setAttribute('tabindex', '0');
 		}
 	};
@@ -3375,10 +3744,15 @@ var msel = (function () {
 	};
 	ebi('selzip').onclick = function (e) {
 		ev(e);
-		var names = r.getsel(),
+		var sel = r.getsel(),
 			arg = ebi('selzip').getAttribute('fmt'),
-			txt = names.join('\n'),
-			frm = mknod('form');
+			frm = mknod('form'),
+			txt = [];
+
+		for (var a = 0; a < sel.length; a++)
+			txt.push(vsplit(sel[a].vp)[1]);
+
+		txt = txt.join('\n');
 
 		frm.setAttribute('action', '?' + arg);
 		frm.setAttribute('method', 'post');
@@ -3435,6 +3809,160 @@ function ev_row_tgl(e) {
 }
 
 
+var unpost = (function () {
+	ebi('op_unpost').innerHTML = (
+		"you can delete your recent uploads below &ndash; click the fire-extinguisher icon to refresh" +
+		'<p>optional filter:&nbsp; URL must contain <input type="text" id="unpost_filt" size="20" /><a id="unpost_nofilt" href="#">clear filter</a></p>' +
+		'<div id="unpost"></div>'
+	);
+
+	var r = {},
+		ct = ebi('unpost'),
+		filt = ebi('unpost_filt');
+
+	r.files = [];
+	r.me = null;
+
+	r.load = function () {
+		var me = Date.now(),
+			html = [];
+
+		function unpost_load_cb() {
+			if (this.readyState != XMLHttpRequest.DONE)
+				return;
+
+			if (this.status !== 200) {
+				var msg = this.responseText;
+				toast.err(9, 'unpost-load failed:\n' + msg);
+				ebi('op_unpost').innerHTML = html.join('\n');
+				return;
+			}
+
+			var res = JSON.parse(this.responseText);
+			if (res.length) {
+				if (res.length == 2000)
+					html.push("<p>showing first 2000 files (use the filter)");
+				else
+					html.push("<p>" + res.length + " uploads can be deleted");
+
+				html.push(" &ndash; sorted by upload time &ndash; most recent first:</p>");
+				html.push("<table><thead><tr><td></td><td>time</td><td>size</td><td>file</td></tr></thead><tbody>");
+			}
+			else
+				html.push("<p>sike! no uploads " + (filt.value ? 'matching that filter' : '') + " are sufficiently recent</p>");
+
+			var mods = [1000, 100, 10];
+			for (var a = 0; a < res.length; a++) {
+				for (var b = 0; b < mods.length; b++)
+					if (a % mods[b] == 0 && res.length > a + mods[b] / 10)
+						html.push(
+							'<tr><td></td><td colspan="3" style="padding:.5em">' +
+							'<a me="' + me + '" class="n' + a + '" n2="' + (a + mods[b]) +
+							'" href="#">delete the next ' + Math.min(mods[b], res.length - a) + ' files below</a></td></tr>');
+				html.push(
+					'<tr><td><a me="' + me + '" class="n' + a + '" href="#">delete</a></td>' +
+					'<td>' + unix2iso(res[a].at) + '</td>' +
+					'<td>' + res[a].sz + '</td>' +
+					'<td>' + linksplit(res[a].vp).join(' ') + '</td></tr>');
+			}
+
+			html.push("</tbody></table>");
+			ct.innerHTML = html.join('\n');
+			r.files = res;
+			r.me = me;
+		}
+
+		var q = '/?ups';
+		if (filt.value)
+			q += '&filter=' + uricom_enc(filt.value, true);
+
+		var xhr = new XMLHttpRequest();
+		xhr.open('GET', q, true);
+		xhr.onreadystatechange = unpost_load_cb;
+		xhr.send();
+
+		ct.innerHTML = "<p><em>loading your recent uploads...</em></p>";
+	};
+
+	function unpost_delete_cb() {
+		if (this.readyState != XMLHttpRequest.DONE)
+			return;
+
+		if (this.status !== 200) {
+			var msg = this.responseText;
+			toast.err(9, 'unpost-delete failed:\n' + msg);
+			return;
+		}
+
+		for (var a = this.n; a < this.n2; a++) {
+			var o = QSA('#op_unpost a.n' + a);
+			for (var b = 0; b < o.length; b++) {
+				var o2 = o[b].closest('tr');
+				o2.parentNode.removeChild(o2);
+			}
+		}
+		toast.ok(5, this.responseText);
+
+		if (!QS('#op_unpost a[me]'))
+			ebi(goto_unpost());
+	}
+
+	ct.onclick = function (e) {
+		var tgt = e.target.closest('a[me]');
+		if (!tgt)
+			return;
+
+		if (!tgt.getAttribute('href'))
+			return;
+
+		var ame = tgt.getAttribute('me');
+		if (ame != r.me)
+			return toast.err(0, 'something broke, please try a refresh');
+
+		var n = parseInt(tgt.className.slice(1)),
+			n2 = parseInt(tgt.getAttribute('n2') || n + 1),
+			req = [];
+
+		for (var a = n; a < n2; a++)
+			if (QS('#op_unpost a.n' + a))
+				req.push(r.files[a].vp);
+
+		var links = QSA('#op_unpost a.n' + n);
+		for (var a = 0, aa = links.length; a < aa; a++) {
+			links[a].removeAttribute('href');
+			links[a].innerHTML = '[busy]';
+		}
+
+		toast.inf(0, "deleting " + req.length + " files...");
+
+		var xhr = new XMLHttpRequest();
+		xhr.n = n;
+		xhr.n2 = n2;
+		xhr.open('POST', '/?delete', true);
+		xhr.onreadystatechange = unpost_delete_cb;
+		xhr.send(JSON.stringify(req));
+	};
+
+	var tfilt = null;
+	filt.oninput = function () {
+		clearTimeout(tfilt);
+		tfilt = setTimeout(r.load, 250);
+	};
+
+	ebi('unpost_nofilt').onclick = function () {
+		filt.value = '';
+		r.load();
+	};
+
+	return r;
+})();
+
+
+function goto_unpost(e) {
+	unpost.load();
+}
+
+
 function reload_mp() {
 	if (mp && mp.au) {
 		mp.au.pause();

copyparty/web/md.css

@@ -41,6 +41,9 @@ html, body {
 	text-shadow: 1px 1px 0 #000;
 	color: #fff;
 }
+#toast pre {
+	margin: 0;
+}
 #toastc {
 	display: inline-block;
 	position: absolute;

copyparty/web/md2.js

@@ -285,15 +285,15 @@ function Modpoll() {
             console.log("modpoll diff |" + server_ref.length + "|, |" + server_now.length + "|");
             this.modpoll.disabled = true;
             var msg = [
-                "The document has changed on the server.<br />" +
+                "The document has changed on the server.",
                 "The changes will NOT be loaded into your editor automatically.",
-
-                "Press F5 or CTRL-R to refresh the page,<br />" +
+                "",
+                "Press F5 or CTRL-R to refresh the page,",
                 "replacing your document with the server copy.",
-
+                "",
                 "You can close this message to ignore and contnue."
             ];
-            return toast.warn(0, "<p>" + msg.join('</p>\n<p>') + '</p>');
+            return toast.warn(0, msg.join('\n'));
         }
 
         console.log('modpoll eq');

copyparty/web/mde.js

@@ -75,7 +75,7 @@ function set_jumpto() {
 }
 
 function jumpto(ev) {
-    var tgt = ev.target || ev.srcElement;
+    var tgt = ev.target;
     var ln = null;
     while (tgt && !ln) {
         ln = tgt.getAttribute('data-ln');

copyparty/web/msg.css

@@ -16,9 +16,6 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 #box {
 	padding: .5em 1em;
 	background: #2c2c2c;

copyparty/web/up2k.js

@@ -1773,3 +1773,14 @@ if (QS('#op_up2k.act'))
     goto_up2k();
 
 apply_perms(perms);
+
+
+(function () {
+    goto();
+    var op = sread('opmode');
+    if (op !== null && op !== '.')
+        try {
+            goto(op);
+        }
+        catch (ex) { }
+})();

copyparty/web/util.js

@@ -398,6 +398,15 @@ function uricom_dec(txt) {
 }
 
 
+function uricom_adec(arr) {
+    var ret = [];
+    for (var a = 0; a < arr.length; a++)
+        ret.push(uricom_dec(arr[a])[0]);
+
+    return ret;
+}
+
+
 function get_evpath() {
     var ret = document.location.pathname;
 

docs/notes.sh

@@ -44,7 +44,7 @@ avg() { awk 'function pr(ncsz) {if (nsmp>0) {printf "%3s %s\n", csz, sum/nsmp} c
 dirs=("$HOME/vfs/ほげ" "$HOME/vfs/ほげ/ぴよ" "$HOME/vfs/$(printf \\xed\\x91)" "$HOME/vfs/$(printf \\xed\\x91/\\xed\\x92)")
 mkdir -p "${dirs[@]}"
 for dir in "${dirs[@]}"; do for fn in ふが "$(printf \\xed\\x93)" 'qwe,rty;asd fgh+jkl%zxc&vbn <qwe>"rty'"'"'uio&asd&nbsp;fgh'; do echo "$dir" > "$dir/$fn.html"; done; done
-
+# qw er+ty%20ui%%20op<as>df&gh&amp;jk#zx'cv"bn`m=qw*er^ty?ui@op,as.df-gh_jk
 
 ##
 ## upload mojibake
@@ -79,6 +79,10 @@ command -v gdate && date() { gdate "$@"; }; while true; do t=$(date +%s.%N); (ti
 # get all up2k search result URLs
 var t=[]; var b=document.location.href.split('#')[0].slice(0, -1); document.querySelectorAll('#u2tab .prog a').forEach((x) => {t.push(b+encodeURI(x.getAttribute("href")))}); console.log(t.join("\n"));
 
+# rename all selected songs to <leading-track-number> + <Title> + <extension>
+var sel=msel.getsel(), ci=find_file_col('Title')[0], re=[]; for (var a=0; a<sel.length; a++) { var url=sel[a].vp, tag=ebi(sel[a].id).closest('tr').querySelectorAll('td')[ci].textContent, name=uricom_dec(vsplit(url)[1])[0], m=/^([0-9]+[\. -]+)?.*(\.[^\.]+$)/.exec(name), name2=(m[1]||'')+tag+m[2], url2=vsplit(url)[0]+uricom_enc(name2,false); if (url!=url2) re.push([url, url2]); }
+console.log(JSON.stringify(re, null, '  '));
+function f() { if (!re.length) return treectl.goto(get_evpath()); var [u1,u2] = re.shift(); fetch(u1+'?move='+u2).then((rsp) => {if (rsp.ok) f(); }); }; f();
 
 ##
 ## bash oneliners

scripts/make-sfx.sh

@@ -2,6 +2,7 @@
 set -e
 echo
 
+help() { exec cat <<'EOF'
 
 # optional args:
 #
@@ -26,6 +27,8 @@ echo
 #
 # `no-dd` saves ~2k by removing the mouse cursor
 
+EOF
+}
 
 # port install gnutar findutils gsed coreutils
 gtar=$(command -v gtar || command -v gnutar) || true
@@ -34,6 +37,7 @@ gtar=$(command -v gtar || command -v gnutar) || true
 	sed()  { gsed  "$@"; }
 	find() { gfind "$@"; }
 	sort() { gsort "$@"; }
+	sha1sum() { shasum "$@"; }
 	unexpand() { gunexpand "$@"; }
 	command -v grealpath >/dev/null &&
 		realpath() { grealpath "$@"; }
@@ -72,6 +76,7 @@ while [ ! -z "$1" ]; do
 		no-cm)  no_cm=1  ; ;;
 		no-sh)  do_sh=   ; ;;
 		no-py)  do_py=   ; ;;
+		*)      help     ; ;;
 	esac
 	shift
 done
@@ -81,16 +86,23 @@ tmv() {
 	mv t "$1"
 }
 
+stamp=$(
+	for d in copyparty scripts; do
+		find $d -type f -printf '%TY-%Tm-%Td %TH:%TM:%TS %p\n'
+	done | sort | tail -n 1 | sha1sum | cut -c-16
+)
+
 rm -rf sfx/*
 mkdir -p sfx build
 cd sfx
 
-[ $repack ] && {
-	old="$(
-		printf '%s\n' "$TMPDIR" /tmp |
-		awk '/./ {print; exit}'
-	)/pe-copyparty"
+tmpdir="$(
+	printf '%s\n' "$TMPDIR" /tmp |
+	awk '/./ {print; exit}'
+)"
 
+[ $repack ] && {
+	old="$tmpdir/pe-copyparty"
 	echo "repack of files in $old"
 	cp -pR "$old/"*{dep-j2,copyparty} .
 }
@@ -172,12 +184,12 @@ mkdir -p ../dist
 sfx_out=../dist/copyparty-sfx
 
 echo cleanup
-find .. -name '*.pyc' -delete
-find .. -name __pycache__ -delete
+find -name '*.pyc' -delete
+find -name __pycache__ -delete
 
 # especially prevent osx from leaking your lan ip (wtf apple)
-find .. -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
-find .. -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
+find -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
+find -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
 
 echo use smol web deps
 rm -f copyparty/web/deps/*.full.* copyparty/web/dbg-* copyparty/web/Makefile
@@ -196,19 +208,24 @@ done
 	rm -rf copyparty/web/mde.* copyparty/web/deps/easymde*
 	echo h > copyparty/web/mde.html
 	f=copyparty/web/md.html
-	sed -r '/edit2">edit \(fancy/d' <$f >t && tmv "$f"
+	sed -r '/edit2">edit \(fancy/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
 	f=copyparty/web/md.css
-	sed -r '/scp\.woff2/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r '/scp\.woff2/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
-	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t
+	tmv "$f"
 }
 
 [ $repack ] ||
@@ -241,20 +258,42 @@ find | grep -E '\.(js|html)$' | while IFS= read -r f; do
 	tmv "$f"
 done
 
-
 gzres() {
-command -v pigz &&
-	pk='pigz -11 -J 34 -I 100' ||
-	pk='gzip'
+	command -v pigz &&
+		pk='pigz -11 -I 256' ||
+		pk='gzip'
 
-echo "$pk"
-find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
-	echo -n .
-	$pk "$f"
-done
-echo
+	echo "$pk"
+	find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
+		echo -n .
+		$pk "$f"
+	done
+	echo
+}
+
+
+zdir="$tmpdir/cpp-mksfx"
+[ -e "$zdir/$stamp" ] || rm -rf "$zdir"
+mkdir -p "$zdir"
+echo a > "$zdir/$stamp"
+nf=$(ls -1 "$zdir"/arc.* | wc -l)
+[ $nf -ge 2 ] && [ ! $repack ] && use_zdir=1 || use_zdir=
+
+[ $use_zdir ] || {
+	echo "$nf alts += 1"
+	gzres
+	[ $repack ] ||
+		tar -cf "$zdir/arc.$(date +%s)" copyparty/web/*.gz
+}
+[ $use_zdir ] && {
+	arcs=("$zdir"/arc.*)
+	arc="${arcs[$RANDOM % ${#arcs[@]} ] }"
+	echo "using $arc"
+	tar -xf "$arc"
+	for f in copyparty/web/*.gz; do
+		rm "${f%.*}"
+	done
 }
-gzres
 
 
 echo gen tarlist

scripts/test/smoketest.py

@@ -124,7 +124,7 @@ def tc1():
 
         arg = "{}:{}:{}".format(pd, ud, p, hp)
         if hp:
-            arg += ":chist=" + hp
+            arg += ":c,hist=" + hp
 
         args += ["-v", arg]
 

scripts/uncomment.py

@@ -65,9 +65,9 @@ def uncomment(fpath):
 
 
 def main():
-    print("uncommenting", end="")
+    print("uncommenting", end="", flush=True)
     for f in sys.argv[1:]:
-        print(".", end="")
+        print(".", end="", flush=True)
         uncomment(f)
 
     print("k")

tests/test_httpcli.py

@@ -31,6 +31,7 @@ class Cfg(Namespace):
             rproxy=0,
             ed=False,
             nw=False,
+            unpost=600,
             no_mv=False,
             no_del=False,
             no_zip=False,
@@ -42,6 +43,7 @@ class Cfg(Namespace):
             nih=True,
             mtp=[],
             mte="a",
+            mth="",
             hist=None,
             no_hash=False,
             css_browser=None,

tests/test_vfs.py

@@ -21,6 +21,7 @@ class Cfg(Namespace):
         ex2 = {
             "mtp": [],
             "mte": "a",
+            "mth": "",
             "hist": None,
             "no_hash": False,
             "css_browser": None,