v1.0.8

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,40 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: '9001'
+
+---
+
+NOTE:
+all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md
+
+
+**Describe the bug**
+a description of what the bug is
+
+**To Reproduce**
+List of steps to reproduce the issue, or, if it's hard to reproduce, then at least a detailed explanation of what you did to run into it
+
+**Expected behavior**
+a description of what you expected to happen
+
+**Screenshots**
+if applicable, add screenshots to help explain your problem, such as the kickass crashpage :^)
+
+**Server details**
+if the issue is possibly on the server-side, then mention some of the following:
+* server OS / version: 
+* python version: 
+* copyparty arguments: 
+* filesystem (`lsblk -f` on linux): 
+
+**Client details**
+if the issue is possibly on the client-side, then mention some of the following:
+* the device type and model: 
+* OS version: 
+* browser version: 
+
+**Additional context**
+any other context about the problem here

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,22 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: '9001'
+
+---
+
+all of the below are optional, consider them as inspiration, delete and rewrite at will
+
+**is your feature request related to a problem? Please describe.**
+a description of what the problem is, for example, `I'm always frustrated when [...]` or `Why is it not possible to [...]`
+
+**Describe the idea / solution you'd like**
+a description of what you want to happen
+
+**Describe any alternatives you've considered**
+a description of any alternative solutions or features you've considered
+
+**Additional context**
+add any other context or screenshots about the feature request here

.github/ISSUE_TEMPLATE/something-else.md

@@ -0,0 +1,10 @@
+---
+name: Something else
+about: "┐(ﾟ∀ﾟ)┌"
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+

.github/branch-rename.md

@@ -0,0 +1,7 @@
+modernize your local checkout of the repo like so,
+```sh
+git branch -m master hovudstraum
+git fetch origin
+git branch -u origin/hovudstraum hovudstraum
+git remote set-head origin -a
+```

.gitignore

@@ -20,3 +20,7 @@ sfx/
 # derived
 copyparty/web/deps/
 srv/
+
+# state/logs
+up.*.txt
+.hist/

.vscode/launch.json

@@ -17,7 +17,7 @@
                 "-mtp",
                 ".bpm=f,bin/mtag/audio-bpm.py",
                 "-aed:wark",
-                "-vsrv::r:aed:cnodupe",
+                "-vsrv::r:rw,ed:c,dupe",
                 "-vdist:dist:r"
             ]
         },

.vscode/settings.json

@@ -55,4 +55,5 @@
         "py27"
     ],
     "python.linting.enabled": true,
+    "python.pythonPath": "/usr/bin/python3"
 }

CODE_OF_CONDUCT.md

@@ -0,0 +1,24 @@
+in the words of Abraham Lincoln:
+
+> Be excellent to each other... and... PARTY ON, DUDES!
+
+more specifically I'll paraphrase some examples from a german automotive corporation as they cover all the bases without being too wordy
+
+## Examples of unacceptable behavior
+* intimidation, harassment, trolling
+* insulting, derogatory, harmful or prejudicial comments
+* posting private information without permission
+* political or personal attacks
+
+## Examples of expected behavior
+* being nice, friendly, welcoming, inclusive, mindful and empathetic
+* acting considerate, modest, respectful
+* using polite and inclusive language
+* criticize constructively and accept constructive criticism
+* respect different points of view
+
+## finally and even more specifically,
+* parse opinions and feedback objectively without prejudice
+  * it's the message that matters, not who said it
+
+aaand that's how you say `be nice` in a way that fills half a floppy w

CONTRIBUTING.md

@@ -0,0 +1,3 @@
+* do something cool
+
+really tho, send a PR or an issue or whatever, all appreciated, anything goes, just behave aight

README.md

@@ -6,87 +6,106 @@
 
 ## summary
 
-turn your phone or raspi into a portable file server with resumable uploads/downloads using IE6 or any other browser
+turn your phone or raspi into a portable file server with resumable uploads/downloads using *any* web browser
 
-* server runs on anything with `py2.7` or `py3.3+`
+* server only needs `py2.7` or `py3.3+`, all dependencies optional
 * browse/upload with IE4 / netscape4.0 on win3.11 (heh)
 * *resumable* uploads need `firefox 34+` / `chrome 41+` / `safari 7+` for full speed
 * code standard: `black`
 
-📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
+📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer) // [ie4](#browser-support)
 
 
 ## readme toc
 
 * top
-    * [quickstart](#quickstart)
+    * [quickstart](#quickstart) - download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
-        * [on debian](#on-debian)
+        * [on servers](#on-servers) - you may also want these, especially on servers
-    * [notes](#notes)
+        * [on debian](#on-debian) - recommended additional steps on debian
-    * [status](#status)
+    * [notes](#notes) - general notes
-    * [testimonials](#testimonials)
+    * [status](#status) - feature summary
+    * [testimonials](#testimonials) - small collection of user feedback
+* [motivations](#motivations) - project goals / philosophy
+    * [future plans](#future-plans) - some improvement ideas
 * [bugs](#bugs)
     * [general bugs](#general-bugs)
     * [not my bugs](#not-my-bugs)
-* [the browser](#the-browser)
+* [FAQ](#FAQ) - "frequently" asked questions
-    * [tabs](#tabs)
+* [accounts and volumes](#accounts-and-volumes) - per-folder, per-user permissions
-    * [hotkeys](#hotkeys)
+* [the browser](#the-browser) - accessing a copyparty server using a web-browser
-    * [navpane](#navpane)
+    * [tabs](#tabs) - the main tabs in the ui
-    * [thumbnails](#thumbnails)
+    * [hotkeys](#hotkeys) - the browser has the following hotkeys
-    * [zip downloads](#zip-downloads)
+    * [navpane](#navpane) - switching between breadcrumbs or navpane
-    * [uploading](#uploading)
+    * [thumbnails](#thumbnails) - press `g` to toggle grid-view instead of the file listing
-        * [file-search](#file-search)
+    * [zip downloads](#zip-downloads) - download folders (or file selections) as `zip` or `tar` files
-    * [file manager](#file-manager)
+    * [uploading](#uploading) - drag files/folders into the web-browser to upload
-    * [batch rename](#batch-rename)
+        * [file-search](#file-search) - dropping files into the browser also lets you see if they exist on the server
-    * [markdown viewer](#markdown-viewer)
+        * [unpost](#unpost) - undo/delete accidental uploads
+    * [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
+    * [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
+    * [markdown viewer](#markdown-viewer) - and there are *two* editors
     * [other tricks](#other-tricks)
-    * [searching](#searching)
+    * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config)
-    * [upload rules](#upload-rules)
+    * [file indexing](#file-indexing)
-    * [database location](#database-location)upload rules
+    * [upload rules](#upload-rules) - set upload rules using volume flags
-    * [metadata from audio files](#metadata-from-audio-files)
+    * [compress uploads](#compress-uploads) - files can be autocompressed on upload
-    * [file parser plugins](#file-parser-plugins)
+    * [database location](#database-location) - in-volume (`.hist/up2k.db`, default) or somewhere else
+    * [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
+    * [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
     * [complete examples](#complete-examples)
-* [browser support](#browser-support)
+* [browser support](#browser-support) - TLDR: yes
-* [client examples](#client-examples)
+* [client examples](#client-examples) - interact with copyparty using non-browser clients
-* [up2k](#up2k)
+* [up2k](#up2k) - quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
-* [performance](#performance)
+    * [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
-* [dependencies](#dependencies)
+* [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
-    * [optional dependencies](#optional-dependencies)
+* [security](#security) - some notes on hardening
+    * [gotchas](#gotchas) - behavior that might be unexpected
+* [recovering from crashes](#recovering-from-crashes)
+    * [client crashes](#client-crashes)
+        * [frefox wsod](#frefox-wsod) - firefox 87 can crash during uploads
+* [dependencies](#dependencies) - mandatory deps
+    * [optional dependencies](#optional-dependencies) - install these to enable bonus features
     * [install recommended deps](#install-recommended-deps)
     * [optional gpl stuff](#optional-gpl-stuff)
-* [sfx](#sfx)
+* [sfx](#sfx) - there are two self-contained "binaries"
-    * [sfx repack](#sfx-repack)
+    * [sfx repack](#sfx-repack) - reduce the size of an sfx by removing features
 * [install on android](#install-on-android)
 * [building](#building)
     * [dev env setup](#dev-env-setup)
     * [just the sfx](#just-the-sfx)
     * [complete release](#complete-release)
-* [todo](#todo)
+* [todo](#todo) - roughly sorted by priority
     * [discarded ideas](#discarded-ideas)
 
 
 ## quickstart
 
-download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
+download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
 
-running the sfx without arguments (for example doubleclicking it on Windows) will give everyone full access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
+running the sfx without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
 
 some recommended options:
-* `-e2dsa` enables general file indexing, see [search configuration](#search-configuration)
+* `-e2dsa` enables general [file indexing](#file-indexing)
 * `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen), see [optional dependencies](#optional-dependencies)
 * `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
   * replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
-  * see [accounts and volumes](#accounts-and-volumes) for the syntax and other access levels (`r`ead, `w`rite, `m`ove, `d`elete)
+  * see [accounts and volumes](#accounts-and-volumes) for the syntax and other permissions (`r`ead, `w`rite, `m`ove, `d`elete, `g`et)
 * `--ls '**,*,ln,p,r'` to crash on startup if any of the volumes contain a symlink which point outside the volume, as that could give users unintended access
 
+
+### on servers
+
 you may also want these, especially on servers:
+
 * [contrib/systemd/copyparty.service](contrib/systemd/copyparty.service) to run copyparty as a systemd service
+* [contrib/systemd/prisonparty.service](contrib/systemd/prisonparty.service) to run it in a chroot (for extra security)
 * [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to reverse-proxy behind nginx (for better https)
 
 
 ### on debian
 
-recommended steps to enable audio metadata and thumbnails (from images and videos):
+recommended additional steps on debian  which enable audio metadata and thumbnails (from images and videos):
 
 * as root, run the following:  
   `apt install python3 python3-pip python3-dev ffmpeg`
@@ -99,7 +118,7 @@ recommended steps to enable audio metadata and thumbnails (from images and video
 
 ## notes
 
-general:
+general notes:
 * paper-printing is affected by dark/light-mode! use lightmode for color, darkmode for grayscale
   * because no browsers currently implement the media-query to do this properly orz
 
@@ -108,12 +127,12 @@ browser-specific:
 * Android-Chrome: increase "parallel uploads" for higher speed (android bug)
 * Android-Firefox: takes a while to select files (their fix for ☝️)
 * Desktop-Firefox: ~~may use gigabytes of RAM if your files are massive~~ *seems to be OK now*
-* Desktop-Firefox: may stop you from deleting folders you've uploaded until you visit `about:memory` and click `Minimize memory usage`
+* Desktop-Firefox: may stop you from deleting files you've uploaded until you visit `about:memory` and click `Minimize memory usage`
 
 
 ## status
 
-summary: all planned features work! now please enjoy the bloatening
+feature summary
 
 * backend stuff
   * ☑ sanic multipart parser
@@ -124,14 +143,14 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ basic: plain multipart, ie6 support
   * ☑ [up2k](#uploading): js, resumable, multithreaded
   * ☑ stash: simple PUT filedropper
-  * ☑ unpost: undo/delete accidental uploads
+  * ☑ [unpost](#unpost): undo/delete accidental uploads
   * ☑ symlink/discard existing files (content-matching)
 * download
   * ☑ single files in browser
   * ☑ [folders as zip / tar files](#zip-downloads)
   * ☑ FUSE client (read-only)
 * browser
-  * ☑ navpane (directory tree sidebar)
+  * ☑ [navpane](#navpane) (directory tree sidebar)
   * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
   * ☑ audio player (with OS media controls)
   * ☑ image gallery with webm player
@@ -157,17 +176,51 @@ small collection of user feedback
 `good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`
 
 
+# motivations
+
+project goals / philosophy
+
+* inverse linux philosophy -- do all the things, and do an *okay* job
+  * quick drop-in service to get a lot of features in a pinch
+  * there are probably [better alternatives](https://github.com/awesome-selfhosted/awesome-selfhosted) if you have specific/long-term needs
+* run anywhere, support everything
+  * as many web-browsers and python versions as possible
+    * every browser should at least be able to browse, download, upload files
+    * be a good emergency solution for transferring stuff between ancient boxes
+  * minimal dependencies
+    * but optional dependencies adding bonus-features are ok
+    * everything being plaintext makes it possible to proofread for malicious code
+  * no preparations / setup necessary, just run the sfx (which is also plaintext)
+* adaptable, malleable, hackable
+  * no build steps; modify the js/python without needing node.js or anything like that
+
+
+## future plans
+
+some improvement ideas
+
+* the JS is a mess -- a preact rewrite would be nice
+  * preferably without build dependencies like webpack/babel/node.js, maybe a python thing to assemble js files into main.js
+  * good excuse to look at using virtual lists (browsers start to struggle when folders contain over 5000 files)
+* the UX is a mess -- a proper design would be nice
+  * very organic (much like the python/js), everything was an afterthought
+  * true for both the layout and the visual flair
+  * something like the tron board-room ui (or most other hollywood ones, like ironman) would be :100:
+* some of the python files are way too big
+  * `up2k.py` ended up doing all the file indexing / db management
+  * `httpcli.py` should be separated into modules in general
+
+
 # bugs
 
 * Windows: python 3.7 and older cannot read tags with FFprobe, so use Mutagen or upgrade
 * Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
 * Windows: python 2.7 cannot handle filenames with mojibake
-* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions
+* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
 
 ## general bugs
 
 * all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
-* cannot mount something at `/d1/d2/d3` unless `d2` exists inside `d1`
 * probably more, pls let me know
 
 ## not my bugs
@@ -182,20 +235,34 @@ small collection of user feedback
   * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db inside the vm instead
 
 
+# FAQ
+
+"frequently" asked questions
+
+* is it possible to block read-access to folders unless you know the exact URL for a particular file inside?
+  * yes, using the [`g` permission](#accounts-and-volumes), see the examples there
+
+* can I make copyparty download a file to my server if I give it a URL?
+  * not officially, but there is a [terrible hack](https://github.com/9001/copyparty/blob/hovudstraum/bin/mtag/wget.py) which makes it possible
+
+
 # accounts and volumes
 
+per-folder, per-user permissions
 * `-a usr:pwd` adds account `usr` with password `pwd`
 * `-v .::r` adds current-folder `.` as the webroot, `r`eadable by anyone
   * the syntax is `-v src:dst:perm:perm:...` so local-path, url-path, and one or more permissions to set
-  * when granting permissions to an account, the names are comma-separated: `-v .::r,usr1,usr2:rw,usr3,usr4`
+  * granting the same permissions to multiple accounts:  
+    `-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
 
 permissions:
 * `r` (read): browse folder contents, download files, download as zip/tar
-* `w` (write): upload files, move files *into* folder
+* `w` (write): upload files, move files *into* this folder
-* `m` (move): move files/folders *from* folder
+* `m` (move): move files/folders *from* this folder
 * `d` (delete): delete files/folders
+* `g` (get): only download files, cannot see folder contents or zip/tar
 
-example:
+examples:
 * add accounts named u1, u2, u3 with passwords p1, p2, p3: `-a u1:p1 -a u2:p2 -a u3:p3`
 * make folder `/srv` the root of the filesystem, read-only by anyone: `-v /srv::r`
 * make folder `/mnt/music` available at `/music`, read-only for u1 and u2, read-write for u3: `-v /mnt/music:music:r,u1,u2:rw,u3`
@@ -204,18 +271,25 @@ example:
   * unauthorized users accessing the webroot can see that the `inc` folder exists, but cannot open it
   * `u1` can open the `inc` folder, but cannot see the contents, only upload new files to it
   * `u2` can browse it and move files *from* `/inc` into any folder where `u2` has write-access
+* make folder `/mnt/ss` available at `/i`, read-write for u1, get-only for everyone else, and enable accesskeys: `-v /mnt/ss:i:rw,u1:g:c,fk=4`
+  * `c,fk=4` sets the `fk` volume-flag to 4, meaning each file gets a 4-character accesskey
+  * `u1` can upload files, browse the folder, and see the generated accesskeys
+  * other users cannot browse the folder, but can access the files if they have the full file URL with the accesskey
 
 
 # the browser
 
-![copyparty-browser-fs8](https://user-images.githubusercontent.com/241032/115978054-65106380-a57d-11eb-98f8-59e3dee73557.png)
+accessing a copyparty server using a web-browser
+
+![copyparty-browser-fs8](https://user-images.githubusercontent.com/241032/129635359-d6dd9b07-8079-4020-ad77-2bfdb9ebd8d5.png)
 
 
 ## tabs
 
-* `[🔎]` search by size, date, path/name, mp3-tags ... see [searching](#searching)
+the main tabs in the ui
-* `[🧯]` unpost: undo/delete accidental uploads
+* `[🔎]` [search](#searching) by size, date, path/name, mp3-tags ...
-* `[🚀]` and `[🎈]` are the uploaders, see [uploading](#uploading)
+* `[🧯]` [unpost](#unpost): undo/delete accidental uploads
+* `[🚀]` and `[🎈]` are the [uploaders](#uploading)
 * `[📂]` mkdir: create directories
 * `[📝]` new-md: create a new markdown document
 * `[📟]` send-msg: either to server-log or into textfiles if `--urlform save`
@@ -225,15 +299,15 @@ example:
 
 ## hotkeys
 
-the browser has the following hotkeys (assumes qwerty, ignores actual layout)
+the browser has the following hotkeys  (always qwerty)
-* `B` toggle breadcrumbs / navpane
+* `B` toggle breadcrumbs / [navpane](#navpane)
 * `I/K` prev/next folder
 * `M` parent folder (or unexpand current)
-* `G` toggle list / grid view
+* `G` toggle list / [grid view](#thumbnails)
 * `T` toggle thumbnails / icons
 * `ctrl-X` cut selected files/folders
 * `ctrl-V` paste
-* `F2` rename selected file/folder
+* `F2` [rename](#batch-rename) selected file/folder
 * when a file/folder is selected (in not-grid-view):
   * `Up/Down` move cursor
   * shift+`Up/Down` select and move cursor
@@ -249,17 +323,18 @@ the browser has the following hotkeys (assumes qwerty, ignores actual layout)
   * `J/L, Left/Right` prev/next file
   * `Home/End` first/last file
   * `S` toggle selection
+  * `R` rotate clockwise (shift=ccw)
   * `Esc` close viewer
   * videos:
     * `U/O` skip 10sec back/forward
     * `P/K/Space` play/pause
     * `F` fullscreen
     * `C` continue playing next video
-    * `R` loop
+    * `V` loop
     * `M` mute
 * when the navpane is open:
   * `A/D` adjust tree width
-* in the grid view:
+* in the [grid view](#thumbnails):
   * `S` toggle multiselect
   * shift+`A/D` zoom
 * in the markdown editor:
@@ -273,14 +348,20 @@ the browser has the following hotkeys (assumes qwerty, ignores actual layout)
 
 ## navpane
 
-by default there's a breadcrumbs path; you can replace this with a navpane (tree-browser sidebar thing) by clicking the `🌲` or pressing the `B` hotkey
+switching between breadcrumbs or navpane
 
-click `[-]` and `[+]` (or hotkeys `A`/`D`) to adjust the size, and the `[a]` toggles if the tree should widen dynamically as you go deeper or stay fixed-size
+click the `🌲` or pressing the `B` hotkey to toggle between breadcrumbs path (default), or a navpane (tree-browser sidebar thing)
+
+* `[-]` and `[+]` (or hotkeys `A`/`D`) adjust the size
+* `[v]` jumps to the currently open folder
+* `[a]` toggles automatic widening as you go deeper
 
 
 ## thumbnails
 
-![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/120070302-10836b00-c08a-11eb-8eb4-82004a34c342.png)
+press `g` to toggle grid-view instead of the file listing,  and `t` toggles icons / thumbnails
+
+![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
 
 it does static images with Pillow and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
 
@@ -291,7 +372,9 @@ in the grid/thumbnail view, if the audio player panel is open, songs will start
 
 ## zip downloads
 
-the `zip` link next to folders can produce various types of zip/tar files using these alternatives in the browser settings tab:
+download folders (or file selections) as `zip` or `tar` files
+
+select which type of archive you want in the `[⚙️] config` tab:
 
 | name | url-suffix | description |
 |--|--|--|
@@ -308,15 +391,18 @@ the `zip` link next to folders can produce various types of zip/tar files using
 
 you can also zip a selection of files or folders by clicking them in the browser, that brings up a selection editor and zip button in the bottom right
 
-![copyparty-zipsel-fs8](https://user-images.githubusercontent.com/241032/116008321-372a2e00-a614-11eb-9a4a-4a1fd9074224.png)
+![copyparty-zipsel-fs8](https://user-images.githubusercontent.com/241032/129635374-e5136e01-470a-49b1-a762-848e8a4c9cdc.png)
+
 
 ## uploading
 
-two upload methods are available in the html client:
+drag files/folders into the web-browser to upload
+
+this initiates an upload using `up2k`; there are two uploaders available:
 * `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
 * `[🚀] up2k`, the fancy one
 
-you can undo/delete uploads using `[🧯] unpost` if the server is running with `-e2d`
+you can also undo/delete uploads by using `[🧯]` [unpost](#unpost)
 
 up2k has several advantages:
 * you can drop folders into the browser (files are added recursively)
@@ -330,53 +416,80 @@ up2k has several advantages:
 
 see [up2k](#up2k) for details on how it works
 
-![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/115978061-680b5400-a57d-11eb-9ef6-cbb5f60aeccc.png)
+![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
 
 **protip:** you can avoid scaring away users with [docs/minimal-up2k.html](docs/minimal-up2k.html) which makes it look [much simpler](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
 
 the up2k UI is the epitome of polished inutitive experiences:
 * "parallel uploads" specifies how many chunks to upload at the same time
 * `[🏃]` analysis of other files should continue while one is uploading
-* `[💭]` ask for confirmation before files are added to the list
+* `[💭]` ask for confirmation before files are added to the queue
-* `[💤]` sync uploading between other copyparty tabs so only one is active
+* `[💤]` sync uploading between other copyparty browser-tabs so only one is active
-* `[🔎]` switch between upload and file-search mode
+* `[🔎]` switch between upload and [file-search](#file-search) mode
+  * ignore `[🔎]` if you add files by dragging them into the browser
 
 and then theres the tabs below it,
-* `[ok]` is uploads which completed successfully
+* `[ok]` is the files which completed successfully
-* `[ng]` is the uploads which failed / got rejected (already exists, ...)
+* `[ng]` is the ones that failed / got rejected (already exists, ...)
 * `[done]` shows a combined list of `[ok]` and `[ng]`, chronological order
 * `[busy]` files which are currently hashing, pending-upload, or uploading
   * plus up to 3 entries each from `[done]` and `[que]` for context
 * `[que]` is all the files that are still queued
 
+note that since up2k has to read each file twice, `[🎈 bup]` can *theoretically* be up to 2x faster in some extreme cases (files bigger than your ram, combined with an internet connection faster than the read-speed of your HDD, or if you're uploading from a cuo2duo)
+
+if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
+
+
 ### file-search
 
-![copyparty-fsearch-fs8](https://user-images.githubusercontent.com/241032/116008320-36919780-a614-11eb-803f-04162326a700.png)
+dropping files into the browser also lets you see if they exist on the server
 
-in the `[🚀 up2k]` tab, after toggling the `[🔎]` switch green, any files/folders you drop onto the dropzone will be hashed on the client-side. Each hash is sent to the server which checks if that file exists somewhere already
+![copyparty-fsearch-fs8](https://user-images.githubusercontent.com/241032/129635361-c79286f0-b8f1-440e-aaf4-6e929428fac9.png)
+
+when you drag/drop files into the browser, you will see two dropzones: `Upload` and `Search`
+
+> on a phone? toggle the `[🔎]` switch green before tapping the big yellow Search button to select your files
+
+the files will be hashed on the client-side, and each hash is sent to the server, which checks if that file exists somewhere
 
 files go into `[ok]` if they exist (and you get a link to where it is), otherwise they land in `[ng]`
 * the main reason filesearch is combined with the uploader is cause the code was too spaghetti to separate it out somewhere else, this is no longer the case but now i've warmed up to the idea too much
 
 adding the same file multiple times is blocked, so if you first search for a file and then decide to upload it, you have to click the `[cleanup]` button to discard `[done]` files (or just refresh the page)
 
-note that since up2k has to read the file twice, `[🎈 bup]` can be up to 2x faster in extreme cases (if your internet connection is faster than the read-speed of your HDD)
 
-up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
+### unpost
+
+undo/delete accidental uploads
+
+![copyparty-unpost-fs8](https://user-images.githubusercontent.com/241032/129635368-3afa6634-c20f-418c-90dc-ec411f3b3897.png)
+
+you can unpost even if you don't have regular move/delete access, however only for files uploaded within the past `--unpost` seconds (default 12 hours) and the server must be running with `-e2d`
 
 
 ## file manager
 
-if you have the required permissions, you can cut/paste, rename, and delete files/folders
+cut/paste, rename, and delete files/folders (if you have permission)
+
+file selection: click somewhere on the line (not the link itsef), then:
+* `space` to toggle
+* `up/down` to move
+* `shift-up/down` to move-and-select
+* `ctrl-shift-up/down` to also scroll
+
+* cut: select some files and `ctrl-x`
+* paste: `ctrl-v` in another folder
+* rename: `F2`
 
 you can move files across browser tabs (cut in one tab, paste in another)
 
 
 ## batch rename
 
-![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
+select some files and press `F2` to bring up the rename UI
 
-select some files and press F2 to bring up the rename UI
+![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
 
 quick explanation of the buttons,  
 * `[✅ apply rename]` confirms and begins renaming
@@ -421,6 +534,8 @@ the metadata keys you can use in the format field are the ones in the file-brows
 
 ## markdown viewer
 
+and there are *two* editors
+
 ![copyparty-md-read-fs8](https://user-images.githubusercontent.com/241032/115978057-66419080-a57d-11eb-8539-d2be843991aa.png)
 
 * the document preview has a max-width which is the same as an A4 paper when printed
@@ -432,10 +547,18 @@ the metadata keys you can use in the format field are the ones in the file-brows
 
 * if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
 
+* click the bottom-left `π` to open a javascript prompt for debugging
+
+* files named `.prologue.html` / `.epilogue.html` will be rendered before/after directory listings unless `--no-logues`
+
+* files named `README.md` / `readme.md` will be rendered after directory listings unless `--no-readme` (but `.epilogue.html` takes precedence)
+
 
 ## searching
 
-![copyparty-search-fs8](https://user-images.githubusercontent.com/241032/115978060-6772bd80-a57d-11eb-81d3-174e869b72c3.png)
+search by size, date, path/name, mp3-tags, ...
+
+![copyparty-search-fs8](https://user-images.githubusercontent.com/241032/129635365-c0ff2a9f-0ee5-4fc3-8bb6-006033cf67b8.png)
 
 when started with `-e2dsa` copyparty will scan/index all your files. This avoids duplicates on upload, and also makes the volumes searchable through the web-ui:
 * make search queries by `size`/`date`/`directory-path`/`filename`, or...
@@ -450,26 +573,29 @@ add the argument `-e2ts` to also scan/index tags from music files, which brings
 
 # server config
 
-file indexing relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
+## file indexing
+
+file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volume flags, or a mix of both.
 
 through arguments:
 * `-e2d` enables file indexing on upload
-* `-e2ds` scans writable folders for new files on startup
+* `-e2ds` also scans writable folders for new files on startup
-* `-e2dsa` scans all mounted volumes (including readonly ones)
+* `-e2dsa` also scans all mounted volumes (including readonly ones)
 * `-e2t` enables metadata indexing on upload
-* `-e2ts` scans for tags in all files that don't have tags yet
+* `-e2ts` also scans for tags in all files that don't have tags yet
-* `-e2tsr` deletes all existing tags, does a full reindex
+* `-e2tsr` also deletes all existing tags, doing a full reindex
 
 the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
-* `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
+* `-v ~/music::r:c,e2dsa,e2tsr` does a full reindex of everything on startup
 * `-v ~/music::r:c,d2d` disables **all** indexing, even if any `-e2*` are on
 * `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
 
 note:
+* the parser can finally handle `c,e2dsa,e2tsr` so you no longer have to `c,e2dsa:c,e2tsr`
 * `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
 * the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
 
-you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
+to save some time, you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
 * initial indexing is way faster, especially when the volume is on a network disk
 * makes it impossible to [file-search](#file-search)
 * if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
@@ -479,7 +605,7 @@ if you set `--no-hash`, you can enable hashing for specific volumes using flag `
 
 ## upload rules
 
-you can set upload rules using volume flags, some examples:
+set upload rules using volume flags,  some examples:
 
 * `:c,sz=1k-3m` sets allowed filesize between 1 KiB and 3 MiB inclusive (suffixes: b, k, m, g)
 * `:c,nosub` disallow uploading into subdirectories; goes well with `rotn` and `rotf`:
@@ -519,6 +645,8 @@ some examples,
 
 ## database location
 
+in-volume (`.hist/up2k.db`, default) or somewhere else
+
 copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
 
 this can instead be kept in a single place using the `--hist` argument, or the `hist=` volume flag, or a mix of both:
@@ -532,18 +660,20 @@ note:
 
 ## metadata from audio files
 
+set `-e2t` to index tags on upload
+
 `-mte` decides which tags to index and display in the browser (and also the display order), this can be changed per-volume:
 * `-v ~/music::r:c,mte=title,artist` indexes and displays *title* followed by *artist*
 
 if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
 
-but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
+but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the `[⚙️] config` pane
 
 `-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
 
 tags that start with a `.` such as `.bpm` and `.dur`(ation) indicate numeric value
 
-see the beautiful mess of a dictionary in [mtag.py](https://github.com/9001/copyparty/blob/master/copyparty/mtag.py) for the default mappings (should cover mp3,opus,flac,m4a,wav,aif,)
+see the beautiful mess of a dictionary in [mtag.py](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/mtag.py) for the default mappings (should cover mp3,opus,flac,m4a,wav,aif,)
 
 `--no-mutagen` disables Mutagen and uses FFprobe instead, which...
 * is about 20x slower than Mutagen
@@ -555,6 +685,8 @@ see the beautiful mess of a dictionary in [mtag.py](https://github.com/9001/copy
 
 ## file parser plugins
 
+provide custom parsers to index additional tags
+
 copyparty can invoke external programs to collect additional metadata for files using `mtp` (either as argument or volume flag), there is a default timeout of 30sec
 
 * `-mtp .bpm=~/bin/audio-bpm.py` will execute `~/bin/audio-bpm.py` with the audio file as argument 1 to provide the `.bpm` tag, if that does not exist in the audio metadata
@@ -575,49 +707,56 @@ copyparty can invoke external programs to collect additional metadata for files
 
 # browser support
 
+TLDR: yes
+
 ![copyparty-ie4-fs8](https://user-images.githubusercontent.com/241032/118192791-fb31fe00-b446-11eb-9647-898ea8efc1f7.png)
 
 `ie` = internet-explorer, `ff` = firefox, `c` = chrome, `iOS` = iPhone/iPad, `Andr` = Android
 
 | feature         | ie6 | ie9  | ie10 | ie11 | ff 52 | c 49 | iOS | Andr |
-| --------------- | --- | --- | ---- | ---- | ----- | ---- | --- | ---- |
+| --------------- | --- | ---- | ---- | ---- | ----- | ---- | --- | ---- |
 | browse files    | yep | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
+| thumbnail view  |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | basic uploader  | yep | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
+| up2k            |  -  |  -   | `*1` | `*1` |  yep  | yep  | yep | yep  |
 | make directory  | yep | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | send message    | yep | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | set sort order  |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | zip selection   |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
-| navpane         |  -  |  -  | `*1` | yep  | yep   | yep  | yep | yep  |
+| file rename     |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
-| up2k            |  -  |  -  | yep  | yep  | yep   | yep  | yep | yep  |
+| file cut/paste  |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
+| navpane         |  -  | `*2` | yep  | yep  |  yep  | yep  | yep | yep  |
+| image viewer    |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
+| video player    |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | markdown editor |  -  |  -   | yep  | yep  |  yep  | yep  | yep | yep  |
-| markdown viewer |  -  |  -  | yep  | yep  | yep   | yep  | yep | yep  |
+| markdown viewer |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
 | play mp3/m4a    |  -  | yep  | yep  | yep  |  yep  | yep  | yep | yep  |
-| play ogg/opus   |  -  |  -  |  -   |  -   | yep   | yep  | `*2` | yep |
+| play ogg/opus   |  -  |  -   |  -   |  -   |  yep  | yep  | `*3` | yep |
-| thumbnail view  |  -  |  -  | -    | -    | yep   | yep  | yep | yep  |
-| image viewer    |  -  |  -  | -    | -    | yep   | yep  | yep | yep  |
 | **= feature =** | ie6 | ie9  | ie10 | ie11 | ff 52 | c 49 | iOS | Andr |
 
 * internet explorer 6 to 8 behave the same
-* firefox 52 and chrome 49 are the last winxp versions
+* firefox 52 and chrome 49 are the final winxp versions
-* `*1` only public folders (login session is dropped) and no history / back-button
+* `*1` yes, but extremely slow (ie10: `1 MiB/s`, ie11: `270 KiB/s`)
-* `*2` using a wasm decoder which can sometimes get stuck and consumes a bit more power
+* `*2` causes a full-page refresh on each navigation
+* `*3` using a wasm decoder which consumes a bit more power
 
 quick summary of more eccentric web-browsers trying to view a directory index:
 
 | browser | will it blend |
 | ------- | ------------- |
-| **safari** (14.0.3/macos) | is chrome with janky wasm, so playing opus can deadlock the javascript engine |
-| **safari** (14.0.1/iOS)   | same as macos, except it recovers from the deadlocks if you poke it a bit |
 | **links** (2.21/macports) | can browse, login, upload/mkdir/msg |
 | **lynx** (2.8.9/macports) | can browse, login, upload/mkdir/msg |
 | **w3m** (0.5.3/macports)  | can browse, login, upload at 100kB/s, mkdir/msg |
 | **netsurf** (3.10/arch)   | is basically ie6 with much better css (javascript has almost no effect) | 
-| **ie4** and **netscape** 4.0  | can browse (text is yellow on white), upload with `?b=u` |
+| **opera** (11.60/winxp)   | OK: thumbnails, image-viewer, zip-selection, rename/cut/paste. NG: up2k, navpane, markdown, audio |
+| **ie4** and **netscape** 4.0  | can browse, upload with `?b=u` |
 | **SerenityOS** (7e98457)  | hits a page fault, works with `?b=u`, file upload not-impl |
 
 
 # client examples
 
+interact with copyparty using non-browser clients
+
 * javascript: dump some state into a file (two separate examples)
   * `await fetch('https://127.0.0.1:3923/', {method:"PUT", body: JSON.stringify(foo)});`
   * `var xhr = new XMLHttpRequest(); xhr.open('POST', 'https://127.0.0.1:3923/msgs?raw'); xhr.send('foo');`
@@ -630,6 +769,14 @@ quick summary of more eccentric web-browsers trying to view a directory index:
   * `chunk(){ curl -b cppwd=wark -T- http://127.0.0.1:3923/;}`  
     `chunk <movie.mkv`
 
+* bash: when curl and wget is not available or too boring
+  * `(printf 'PUT /junk?pw=wark HTTP/1.1\r\n\r\n'; cat movie.mkv) | nc 127.0.0.1 3923`
+  * `(printf 'PUT / HTTP/1.1\r\n\r\n'; cat movie.mkv) >/dev/tcp/127.0.0.1/3923`
+
+* python: [up2k.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) is a command-line up2k client [(webm)](https://ocv.me/stuff/u2cli.webm)
+  * file uploads, file-search, autoresume of aborted/broken uploads
+  * see [./bin/README.md#up2kpy](bin/README.md#up2kpy)
+
 * FUSE: mount a copyparty server as a local filesystem
   * cross-platform python client available in [./bin/](bin/)
   * [rclone](https://rclone.org/) as client can give ~5x performance, see [./docs/rclone.md](docs/rclone.md)
@@ -641,6 +788,8 @@ copyparty returns a truncated sha512sum of your PUT/POST as base64; you can gene
     b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
     b512 <movie.mkv
 
+you can provide passwords using cookie 'cppwd=hunter2', as a url query `?pw=hunter2`, or with basic-authentication (either as the username or password)
+
 
 # up2k
 
@@ -657,10 +806,25 @@ quick outline of the up2k protocol, see [uploading](#uploading) for the web-clie
   * server writes chunks into place based on the hash
 * client does another handshake with the hashlist; server replies with OK or a list of chunks to reupload
 
+up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
+
+
+## why chunk-hashes
+
+a single sha512 would be better, right?
+
+this is due to `crypto.subtle` not providing a streaming api (or the option to seed the sha512 hasher with a starting hash)
+
+as a result, the hashes are much less useful than they could have been (search the server by sha512, provide the sha512 in the response http headers, ...)
+
+hashwasm would solve the streaming issue but reduces hashing speed for sha512 (xxh128 does 6 GiB/s), and it would make old browsers and [iphones](https://bugs.webkit.org/show_bug.cgi?id=228552) unsupported
+
 
 # performance
 
-defaults are good for most cases, don't mind the `cannot efficiently use multiple CPU cores` message, it's very unlikely to be a problem
+defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
+
+you can ignore the `cannot efficiently use multiple CPU cores` message, very unlikely to be a problem
 
 below are some tweaks roughly ordered by usefulness:
 
@@ -675,28 +839,68 @@ below are some tweaks roughly ordered by usefulness:
   ...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
 
 
+# security
+
+some notes on hardening
+
+on public copyparty instances with anonymous upload enabled:
+
+* users can upload html/css/js which will evaluate for other visitors in a few ways,
+  * unless `--no-readme` is set: by uploading/modifying a file named `readme.md`
+  * if `move` access is granted AND none of `--no-logues`, `--no-dot-mv`, `--no-dot-ren` is set: by uploading some .html file and renaming it to `.epilogue.html` (uploading it directly is blocked)
+
+other misc:
+
+* you can disable directory listings by giving permission `g` instead of `r`, only accepting direct URLs to files
+  * combine this with volume-flag `c,fk` to generate per-file accesskeys; users which have full read-access will then see URLs with `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
+
+
+## gotchas
+
+behavior that might be unexpected
+
+* users without read-access to a folder can still see the `.prologue.html` / `.epilogue.html` / `README.md` contents, for the purpose of showing a description on how to use the uploader for example
+
+
+# recovering from crashes
+
+## client crashes
+
+### frefox wsod
+
+firefox 87 can crash during uploads  -- the entire browser goes, including all other browser tabs, everything turns white
+
+however you can hit `F12` in the up2k tab and use the devtools to see how far you got in the uploads:
+
+* get a complete list of all uploads, organized by statuts (ok / no-good / busy / queued):  
+  `var tabs = { ok:[], ng:[], bz:[], q:[] }; for (var a of up2k.ui.tab) tabs[a.in].push(a); tabs`
+
+* list of filenames which failed:  
+  `​var ng = []; for (var a of up2k.ui.tab) if (a.in != 'ok') ng.push(a.hn.split('<a href=\"').slice(-1)[0].split('\">')[0]); ng`
+
+* send the list of filenames to copyparty for safekeeping:  
+  `await fetch('/inc', {method:'PUT', body:JSON.stringify(ng,null,1)})`
+
+
 # dependencies
 
+mandatory deps:
 * `jinja2` (is built into the SFX)
 
 
 ## optional dependencies
 
+install these to enable bonus features
+
 enable music tags:
 * either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
 * or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
 
-enable thumbnails of images:
+enable [thumbnails](#thumbnails) of...
-* `Pillow` (requires py2.7 or py3.5+)
+* **images:** `Pillow` (requires py2.7 or py3.5+)
-
+* **videos:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
-enable thumbnails of videos:
+* **HEIF pictures:** `pyheif-pillow-opener` (requires Linux or a C compiler)
-* `ffmpeg` and `ffprobe` somewhere in `$PATH`
+* **AVIF pictures:** `pillow-avif-plugin`
-
-enable thumbnails of HEIF pictures:
-* `pyheif-pillow-opener` (requires Linux or a C compiler)
-
-enable thumbnails of AVIF pictures:
-* `pillow-avif-plugin`
 
 
 ## install recommended deps
@@ -714,7 +918,7 @@ these are standalone programs and will never be imported / evaluated by copypart
 
 # sfx
 
-currently there are two self-contained "binaries":
+there are two self-contained "binaries":
 * [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) -- pure python, works everywhere, **recommended**
 * [copyparty-sfx.sh](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.sh) -- smaller, but only for linux and macos, kinda deprecated
 
@@ -725,14 +929,18 @@ pls note that `copyparty-sfx.sh` will fail if you rename `copyparty-sfx.py` to `
 
 ## sfx repack
 
+reduce the size of an sfx by removing features
+
 if you don't need all the features, you can repack the sfx and save a bunch of space; all you need is an sfx and a copy of this repo (nothing else to download or build, except if you're on windows then you need msys2 or WSL)
 * `525k` size of original sfx.py as of v0.11.30
 * `315k` after `./scripts/make-sfx.sh re no-ogv`
 * `223k` after `./scripts/make-sfx.sh re no-ogv no-cm`
 
 the features you can opt to drop are
-* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files
+* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files, saves ~192k
-* `cm`/easymde, the "fancy" markdown editor
+* `cm`/easymde, the "fancy" markdown editor, saves ~92k
+* `fnt`, source-code-pro, the monospace font, saves ~9k
+* `dd`, the custom mouse cursor for the media player tray tab, saves ~2k
 
 for the `re`pack to work, first run one of the sfx'es once to unpack it
 
@@ -768,7 +976,7 @@ pip install black bandit pylint flake8  # vscode tooling
 
 ## just the sfx
 
-unless you need to modify something in the web-dependencies, it's faster to grab those from a previous release:
+first grab the web-dependencies from a previous sfx (assuming you don't need to modify something in those):
 
 ```sh
 rm -rf copyparty/web/deps
@@ -788,14 +996,14 @@ then build the sfx using any of the following examples:
 
 ## complete release
 
-also builds the sfx so disregard the sfx section above
+also builds the sfx so skip the sfx section above
 
 in the `scripts` folder:
 
 * run `make -C deps-docker` to build all dependencies
 * `git tag v1.2.3 && git push origin --tags`
-* create github release with `make-tgz-release.sh`
 * upload to pypi with `make-pypi-release.(sh|bat)`
+* create github release with `make-tgz-release.sh`
 * create sfx with `make-sfx.sh`
 
 
@@ -803,8 +1011,7 @@ in the `scripts` folder:
 
 roughly sorted by priority
 
-* hls framework for Someone Else to drop code into :^)
+* nothing! currently
-* readme.md as epilogue
 
 
 ## discarded ideas
@@ -832,3 +1039,5 @@ roughly sorted by priority
 * indexedDB for hashes, cfg enable/clear/sz, 2gb avail, ~9k for 1g, ~4k for 100m, 500k items before autoeviction
   * blank hashlist when up-ok to skip handshake
     * too many confusing side-effects
+* hls framework for Someone Else to drop code into :^)
+  * probably not, too much stuff to consider -- seeking, start at offset, task stitching (probably np-hard), conditional passthru, rate-control (especially multi-consumer), session keepalive, cache mgmt...

bin/README.md

@@ -1,3 +1,11 @@
+# [`up2k.py`](up2k.py)
+* command-line up2k client [(webm)](https://ocv.me/stuff/u2cli.webm)
+* file uploads, file-search, autoresume of aborted/broken uploads
+* faster than browsers
+* early beta, if something breaks just restart it
+
+
+
 # [`copyparty-fuse.py`](copyparty-fuse.py)
 * mount a copyparty server as a local filesystem (read-only)
 * **supports Windows!** -- expect `194 MiB/s` sequential read
@@ -47,6 +55,7 @@ you could replace winfsp with [dokan](https://github.com/dokan-dev/dokany/releas
 * copyparty can Popen programs like these during file indexing to collect additional metadata
 
 
+
 # [`dbtool.py`](dbtool.py)
 upgrade utility which can show db info and help transfer data between databases, for example when a new version of copyparty is incompatible with the old DB and automatically rebuilds the DB from scratch, but you have some really expensive `-mtp` parsers and want to copy over the tags from the old db
 
@@ -63,6 +72,7 @@ cd /mnt/nas/music/.hist
 ```
 
 
+
 # [`prisonparty.sh`](prisonparty.sh)
 * run copyparty in a chroot, preventing any accidental file access
 * creates bindmounts for /bin, /lib, and so on, see `sysdirs=`

bin/copyparty-fuse.py

@@ -22,7 +22,7 @@ dependencies:
 
 note:
   you probably want to run this on windows clients:
-  https://github.com/9001/copyparty/blob/master/contrib/explorer-nothumbs-nofoldertypes.reg
+  https://github.com/9001/copyparty/blob/hovudstraum/contrib/explorer-nothumbs-nofoldertypes.reg
 
 get server cert:
   awk '/-BEGIN CERTIFICATE-/ {a=1} a; /-END CERTIFICATE-/{exit}' <(openssl s_client -connect 127.0.0.1:3923 </dev/null 2>/dev/null) >cert.pem
@@ -71,7 +71,7 @@ except:
     elif MACOS:
         libfuse = "install https://osxfuse.github.io/"
     else:
-        libfuse = "apt install libfuse\n    modprobe fuse"
+        libfuse = "apt install libfuse3-3\n    modprobe fuse"
 
     print(
         "\n  could not import fuse; these may help:"
@@ -393,15 +393,16 @@ class Gateway(object):
 
         rsp = json.loads(rsp.decode("utf-8"))
         ret = []
-        for is_dir, nodes in [[True, rsp["dirs"]], [False, rsp["files"]]]:
+        for statfun, nodes in [
+            [self.stat_dir, rsp["dirs"]],
+            [self.stat_file, rsp["files"]],
+        ]:
             for n in nodes:
-                fname = unquote(n["href"]).rstrip(b"/")
+                fname = unquote(n["href"].split("?")[0]).rstrip(b"/").decode("wtf-8")
-                fname = fname.decode("wtf-8")
                 if bad_good:
                     fname = enwin(fname)
 
-                fun = self.stat_dir if is_dir else self.stat_file
+                ret.append([fname, statfun(n["ts"], n["sz"]), 0])
-                ret.append([fname, fun(n["ts"], n["sz"]), 0])
 
         return ret
 

bin/mtag/README.md

@@ -1,11 +1,19 @@
 standalone programs which take an audio file as argument
 
+**NOTE:** these all require `-e2ts` to be functional, meaning you need to do at least one of these: `apt install ffmpeg` or `pip3 install mutagen`
+
 some of these rely on libraries which are not MIT-compatible
 
 * [audio-bpm.py](./audio-bpm.py) detects the BPM of music using the BeatRoot Vamp Plugin; imports GPL2
 * [audio-key.py](./audio-key.py) detects the melodic key of music using the Mixxx fork of keyfinder; imports GPL3
 * [media-hash.py](./media-hash.py) generates checksums for audio and video streams; uses FFmpeg (LGPL or GPL)
 
+these do not have any problematic dependencies:
+
+* [cksum.py](./cksum.py) computes various checksums
+* [exe.py](./exe.py) grabs metadata from .exe and .dll files (example for retrieving multiple tags with one parser)
+* [wget.py](./wget.py) lets you download files by POSTing URLs to copyparty
+
 
 # dependencies
 

bin/mtag/audio-bpm.py

@@ -25,6 +25,7 @@ def det(tf):
         "-v", "fatal",
         "-ss", "13",
         "-y", "-i", fsenc(sys.argv[1]),
+        "-map", "0:a:0",
         "-ac", "1",
         "-ar", "22050",
         "-t", "300",

bin/mtag/audio-key.py

@@ -28,6 +28,7 @@ def det(tf):
         "-hide_banner",
         "-v", "fatal",
         "-y", "-i", fsenc(sys.argv[1]),
+        "-map", "0:a:0",
         "-t", "300",
         "-sample_fmt", "s16",
         tf

bin/mtag/cksum.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+
+import sys
+import json
+import zlib
+import struct
+import base64
+import hashlib
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p
+
+
+"""
+calculates various checksums for uploads,
+usage: -mtp crc32,md5,sha1,sha256b=bin/mtag/cksum.py
+"""
+
+
+def main():
+    config = "crc32 md5 md5b sha1 sha1b sha256 sha256b sha512/240 sha512b/240"
+    # b suffix = base64 encoded
+    # slash = truncate to n bits
+
+    known = {
+        "md5": hashlib.md5,
+        "sha1": hashlib.sha1,
+        "sha256": hashlib.sha256,
+        "sha512": hashlib.sha512,
+    }
+    config = config.split()
+    hashers = {
+        k: v()
+        for k, v in known.items()
+        if k in [x.split("/")[0].rstrip("b") for x in known]
+    }
+    crc32 = 0 if "crc32" in config else None
+
+    with open(fsenc(sys.argv[1]), "rb", 512 * 1024) as f:
+        while True:
+            buf = f.read(64 * 1024)
+            if not buf:
+                break
+
+            for x in hashers.values():
+                x.update(buf)
+
+            if crc32 is not None:
+                crc32 = zlib.crc32(buf, crc32)
+
+    ret = {}
+    for s in config:
+        alg = s.split("/")[0]
+        b64 = alg.endswith("b")
+        alg = alg.rstrip("b")
+        if alg in hashers:
+            v = hashers[alg].digest()
+        elif alg == "crc32":
+            v = crc32
+            if v < 0:
+                v &= 2 ** 32 - 1
+            v = struct.pack(">L", v)
+        else:
+            raise Exception("what is {}".format(s))
+
+        if "/" in s:
+            v = v[: int(int(s.split("/")[1]) / 8)]
+
+        if b64:
+            v = base64.b64encode(v).decode("ascii").rstrip("=")
+        else:
+            try:
+                v = v.hex()
+            except:
+                import binascii
+
+                v = binascii.hexlify(v)
+
+        ret[s] = v
+
+    print(json.dumps(ret, indent=4))
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/install-deps.sh

@@ -4,7 +4,8 @@ set -e
 
 # install dependencies for audio-*.py
 #
-# linux: requires {python3,ffmpeg,fftw}-dev py3-{wheel,pip} py3-numpy{,-dev} vamp-sdk-dev patchelf
+# linux/alpine: requires {python3,ffmpeg,fftw}-dev py3-{wheel,pip} py3-numpy{,-dev} vamp-sdk-dev patchelf cmake
+# linux/debian: requires libav{codec,device,filter,format,resample,util}-dev {libfftw3,python3}-dev python3-{numpy,pip} vamp-{plugin-sdk,examples} patchelf cmake
 # win64: requires msys2-mingw64 environment
 # macos: requires macports
 #

bin/mtag/wget.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+
+"""
+use copyparty as a file downloader by POSTing URLs as
+application/x-www-form-urlencoded (for example using the
+message/pager function on the website)
+
+example copyparty config to use this:
+  --urlform save,get -vsrv/wget:wget:rwmd,ed:c,e2ts,mtp=title=ebin,t300,ad,bin/mtag/wget.py
+
+explained:
+  for realpath srv/wget (served at /wget) with read-write-modify-delete for ed,
+  enable file analysis on upload (e2ts),
+  use mtp plugin "bin/mtag/wget.py" to provide metadata tag "title",
+  do this on all uploads with the file extension "bin",
+  t300 = 300 seconds timeout for each dwonload,
+  ad = parse file regardless if FFmpeg thinks it is audio or not
+
+PS: this requires e2ts to be functional,
+  meaning you need to do at least one of these:
+   * apt install ffmpeg
+   * pip3 install mutagen
+"""
+
+
+import os
+import sys
+import subprocess as sp
+from urllib.parse import unquote_to_bytes as unquote
+
+
+def main():
+    fp = os.path.abspath(sys.argv[1])
+    fdir = os.path.dirname(fp)
+    fname = os.path.basename(fp)
+    if not fname.startswith("put-") or not fname.endswith(".bin"):
+        raise Exception("not a post file")
+
+    buf = b""
+    with open(fp, "rb") as f:
+        while True:
+            b = f.read(4096)
+            buf += b
+            if len(buf) > 4096:
+                raise Exception("too big")
+
+            if not b:
+                break
+
+    if not buf:
+        raise Exception("file is empty")
+
+    buf = unquote(buf.replace(b"+", b" "))
+    url = buf.decode("utf-8")
+
+    if not url.startswith("msg="):
+        raise Exception("does not start with msg=")
+
+    url = url[4:]
+    if "://" not in url:
+        url = "https://" + url
+
+    os.chdir(fdir)
+
+    name = url.split("?")[0].split("/")[-1]
+    tfn = "-- DOWNLOADING " + name
+    open(tfn, "wb").close()
+
+    cmd = ["wget", "--trust-server-names", "--", url]
+
+    try:
+        sp.check_call(cmd)
+
+        # OPTIONAL:
+        #   on success, delete the .bin file which contains the URL
+        os.unlink(fp)
+    except:
+        open("-- FAILED TO DONWLOAD " + name, "wb").close()
+
+    os.unlink(tfn)
+    print(url)
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/yt-ipr.py

@@ -17,7 +17,7 @@ it's probably best to use this through a config file; see res/yt-ipr.conf
 
 but if you want to use plain arguments instead then:
   -v srv/ytm:ytm:w:rw,ed
-       :c,e2ts:c,e2dsa
+       :c,e2ts,e2dsa
        :c,sz=16k-1m:c,maxn=10,300:c,rotf=%Y-%m/%d-%H
        :c,mtp=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires=bin/mtag/yt-ipr.py
        :c,mte=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires

bin/up2k.py

@@ -0,0 +1,723 @@
+#!/usr/bin/env python3
+from __future__ import print_function, unicode_literals
+
+"""
+up2k.py: upload to copyparty
+2021-10-04, v0.7, ed <irc.rizon.net>, MIT-Licensed
+https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py
+
+- dependencies: requests
+- supports python 2.6, 2.7, and 3.3 through 3.10
+
+- almost zero error-handling
+- but if something breaks just try again and it'll autoresume
+"""
+
+import os
+import sys
+import stat
+import math
+import time
+import atexit
+import signal
+import base64
+import hashlib
+import argparse
+import platform
+import threading
+import requests
+import datetime
+
+
+# from copyparty/__init__.py
+PY2 = sys.version_info[0] == 2
+if PY2:
+    from Queue import Queue
+
+    sys.dont_write_bytecode = True
+    bytes = str
+else:
+    from queue import Queue
+
+    unicode = str
+
+VT100 = platform.system() != "Windows"
+
+
+req_ses = requests.Session()
+
+
+class File(object):
+    """an up2k upload task; represents a single file"""
+
+    def __init__(self, top, rel, size, lmod):
+        self.top = top  # type: bytes
+        self.rel = rel.replace(b"\\", b"/")  # type: bytes
+        self.size = size  # type: int
+        self.lmod = lmod  # type: float
+
+        self.abs = os.path.join(top, rel)  # type: bytes
+        self.name = self.rel.split(b"/")[-1].decode("utf-8", "replace")  # type: str
+
+        # set by get_hashlist
+        self.cids = []  # type: list[tuple[str, int, int]]  # [ hash, ofs, sz ]
+        self.kchunks = {}  # type: dict[str, tuple[int, int]]  # hash: [ ofs, sz ]
+
+        # set by handshake
+        self.ucids = []  # type: list[str]  # chunks which need to be uploaded
+        self.wark = None  # type: str
+        self.url = None  # type: str
+
+        # set by upload
+        self.up_b = 0  # type: int
+        self.up_c = 0  # type: int
+
+        # m = "size({}) lmod({}) top({}) rel({}) abs({}) name({})\n"
+        # eprint(m.format(self.size, self.lmod, self.top, self.rel, self.abs, self.name))
+
+
+class FileSlice(object):
+    """file-like object providing a fixed window into a file"""
+
+    def __init__(self, file, cid):
+        # type: (File, str) -> FileSlice
+
+        self.car, self.len = file.kchunks[cid]
+        self.cdr = self.car + self.len
+        self.ofs = 0  # type: int
+        self.f = open(file.abs, "rb", 512 * 1024)
+        self.f.seek(self.car)
+
+        # https://stackoverflow.com/questions/4359495/what-is-exactly-a-file-like-object-in-python
+        # IOBase, RawIOBase, BufferedIOBase
+        funs = "close closed __enter__ __exit__ __iter__ isatty __next__ readable seekable writable"
+        try:
+            for fun in funs.split():
+                setattr(self, fun, getattr(self.f, fun))
+        except:
+            pass  # py27 probably
+
+    def tell(self):
+        return self.ofs
+
+    def seek(self, ofs, wh=0):
+        if wh == 1:
+            ofs = self.ofs + ofs
+        elif wh == 2:
+            ofs = self.len + ofs  # provided ofs is negative
+
+        if ofs < 0:
+            ofs = 0
+        elif ofs >= self.len:
+            ofs = self.len - 1
+
+        self.ofs = ofs
+        self.f.seek(self.car + ofs)
+
+    def read(self, sz):
+        sz = min(sz, self.len - self.ofs)
+        ret = self.f.read(sz)
+        self.ofs += len(ret)
+        return ret
+
+
+def eprint(*a, **ka):
+    ka["file"] = sys.stderr
+    ka["end"] = ""
+    if not PY2:
+        ka["flush"] = True
+
+    print(*a, **ka)
+    if PY2:
+        sys.stderr.flush()
+
+
+def termsize():
+    import os
+
+    env = os.environ
+
+    def ioctl_GWINSZ(fd):
+        try:
+            import fcntl, termios, struct, os
+
+            cr = struct.unpack("hh", fcntl.ioctl(fd, termios.TIOCGWINSZ, "1234"))
+        except:
+            return
+        return cr
+
+    cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
+    if not cr:
+        try:
+            fd = os.open(os.ctermid(), os.O_RDONLY)
+            cr = ioctl_GWINSZ(fd)
+            os.close(fd)
+        except:
+            pass
+    if not cr:
+        try:
+            cr = (env["LINES"], env["COLUMNS"])
+        except:
+            cr = (25, 80)
+    return int(cr[1]), int(cr[0])
+
+
+class CTermsize(object):
+    def __init__(self):
+        self.ev = False
+        self.margin = None
+        self.g = None
+        self.w, self.h = termsize()
+
+        try:
+            signal.signal(signal.SIGWINCH, self.ev_sig)
+        except:
+            return
+
+        thr = threading.Thread(target=self.worker)
+        thr.daemon = True
+        thr.start()
+
+    def worker(self):
+        while True:
+            time.sleep(0.5)
+            if not self.ev:
+                continue
+
+            self.ev = False
+            self.w, self.h = termsize()
+
+            if self.margin is not None:
+                self.scroll_region(self.margin)
+
+    def ev_sig(self, *a, **ka):
+        self.ev = True
+
+    def scroll_region(self, margin):
+        self.margin = margin
+        if margin is None:
+            self.g = None
+            eprint("\033[s\033[r\033[u")
+        else:
+            self.g = 1 + self.h - margin
+            m = "{0}\033[{1}A".format("\n" * margin, margin)
+            eprint("{0}\033[s\033[1;{1}r\033[u".format(m, self.g - 1))
+
+
+ss = CTermsize()
+
+
+def statdir(top):
+    """non-recursive listing of directory contents, along with stat() info"""
+    if hasattr(os, "scandir"):
+        with os.scandir(top) as dh:
+            for fh in dh:
+                yield [os.path.join(top, fh.name), fh.stat()]
+    else:
+        for name in os.listdir(top):
+            abspath = os.path.join(top, name)
+            yield [abspath, os.stat(abspath)]
+
+
+def walkdir(top):
+    """recursive statdir"""
+    for ap, inf in sorted(statdir(top)):
+        if stat.S_ISDIR(inf.st_mode):
+            for x in walkdir(ap):
+                yield x
+        else:
+            yield ap, inf
+
+
+def walkdirs(tops):
+    """recursive statdir for a list of tops, yields [top, relpath, stat]"""
+    for top in tops:
+        if os.path.isdir(top):
+            for ap, inf in walkdir(top):
+                yield top, ap[len(top) + 1 :], inf
+        else:
+            sep = "{0}".format(os.sep).encode("ascii")
+            d, n = top.rsplit(sep, 1)
+            yield d, n, os.stat(top)
+
+
+# from copyparty/util.py
+def humansize(sz, terse=False):
+    """picks a sensible unit for the given extent"""
+    for unit in ["B", "KiB", "MiB", "GiB", "TiB"]:
+        if sz < 1024:
+            break
+
+        sz /= 1024.0
+
+    ret = " ".join([str(sz)[:4].rstrip("."), unit])
+
+    if not terse:
+        return ret
+
+    return ret.replace("iB", "").replace(" ", "")
+
+
+# from copyparty/up2k.py
+def up2k_chunksize(filesize):
+    """gives The correct chunksize for up2k hashing"""
+    chunksize = 1024 * 1024
+    stepsize = 512 * 1024
+    while True:
+        for mul in [1, 2]:
+            nchunks = math.ceil(filesize * 1.0 / chunksize)
+            if nchunks <= 256 or chunksize >= 32 * 1024 * 1024:
+                return chunksize
+
+            chunksize += stepsize
+            stepsize *= mul
+
+
+# mostly from copyparty/up2k.py
+def get_hashlist(file, pcb):
+    # type: (File, any) -> None
+    """generates the up2k hashlist from file contents, inserts it into `file`"""
+
+    chunk_sz = up2k_chunksize(file.size)
+    file_rem = file.size
+    file_ofs = 0
+    ret = []
+    with open(file.abs, "rb", 512 * 1024) as f:
+        while file_rem > 0:
+            hashobj = hashlib.sha512()
+            chunk_sz = chunk_rem = min(chunk_sz, file_rem)
+            while chunk_rem > 0:
+                buf = f.read(min(chunk_rem, 64 * 1024))
+                if not buf:
+                    raise Exception("EOF at " + str(f.tell()))
+
+                hashobj.update(buf)
+                chunk_rem -= len(buf)
+
+            digest = hashobj.digest()[:33]
+            digest = base64.urlsafe_b64encode(digest).decode("utf-8")
+
+            ret.append([digest, file_ofs, chunk_sz])
+            file_ofs += chunk_sz
+            file_rem -= chunk_sz
+
+            if pcb:
+                pcb(file, file_ofs)
+
+    file.cids = ret
+    file.kchunks = {}
+    for k, v1, v2 in ret:
+        file.kchunks[k] = [v1, v2]
+
+
+def handshake(req_ses, url, file, pw, search):
+    # type: (requests.Session, str, File, any, bool) -> List[str]
+    """
+    performs a handshake with the server; reply is:
+      if search, a list of search results
+      otherwise, a list of chunks to upload
+    """
+
+    req = {
+        "hash": [x[0] for x in file.cids],
+        "name": file.name,
+        "lmod": file.lmod,
+        "size": file.size,
+    }
+    if search:
+        req["srch"] = 1
+
+    headers = {"Content-Type": "text/plain"}  # wtf ed
+    if pw:
+        headers["Cookie"] = "=".join(["cppwd", pw])
+
+    if file.url:
+        url = file.url
+    elif b"/" in file.rel:
+        url += file.rel.rsplit(b"/", 1)[0].decode("utf-8", "replace")
+
+    while True:
+        try:
+            r = req_ses.post(url, headers=headers, json=req)
+            break
+        except:
+            eprint("handshake failed, retry...\n")
+            time.sleep(1)
+
+    try:
+        r = r.json()
+    except:
+        raise Exception(r.text)
+
+    if search:
+        return r["hits"]
+
+    try:
+        pre, url = url.split("://")
+        pre += "://"
+    except:
+        pre = ""
+
+    file.url = pre + url.split("/")[0] + r["purl"]
+    file.name = r["name"]
+    file.wark = r["wark"]
+
+    return r["hash"]
+
+
+def upload(req_ses, file, cid, pw):
+    # type: (requests.Session, File, str, any) -> None
+    """upload one specific chunk, `cid` (a chunk-hash)"""
+
+    headers = {
+        "X-Up2k-Hash": cid,
+        "X-Up2k-Wark": file.wark,
+        "Content-Type": "application/octet-stream",
+    }
+    if pw:
+        headers["Cookie"] = "=".join(["cppwd", pw])
+
+    f = FileSlice(file, cid)
+    try:
+        r = req_ses.post(file.url, headers=headers, data=f)
+        if not r:
+            raise Exception(repr(r))
+
+        _ = r.content
+    finally:
+        f.f.close()
+
+
+class Daemon(threading.Thread):
+    def __init__(self, *a, **ka):
+        threading.Thread.__init__(self, *a, **ka)
+        self.daemon = True
+
+
+class Ctl(object):
+    """
+    this will be the coordinator which runs everything in parallel
+    (hashing, handshakes, uploads)  but right now it's p dumb
+    """
+
+    def __init__(self, ar):
+        self.ar = ar
+        ar.files = [
+            os.path.abspath(os.path.realpath(x.encode("utf-8"))) for x in ar.files
+        ]
+        ar.url = ar.url.rstrip("/") + "/"
+        if "://" not in ar.url:
+            ar.url = "http://" + ar.url
+
+        eprint("\nscanning {0} locations\n".format(len(ar.files)))
+
+        nfiles = 0
+        nbytes = 0
+        for _, _, inf in walkdirs(ar.files):
+            nfiles += 1
+            nbytes += inf.st_size
+
+        eprint("found {0} files, {1}\n\n".format(nfiles, humansize(nbytes)))
+        self.nfiles = nfiles
+        self.nbytes = nbytes
+
+        if ar.td:
+            req_ses.verify = False
+        if ar.te:
+            req_ses.verify = ar.te
+
+        self.filegen = walkdirs(ar.files)
+        if ar.safe:
+            self.safe()
+        else:
+            self.fancy()
+
+    def safe(self):
+        """minimal basic slow boring fallback codepath"""
+        search = self.ar.s
+        for nf, (top, rel, inf) in enumerate(self.filegen):
+            file = File(top, rel, inf.st_size, inf.st_mtime)
+            upath = file.abs.decode("utf-8", "replace")
+
+            print("{0} {1}\n  hash...".format(self.nfiles - nf, upath))
+            get_hashlist(file, None)
+
+            while True:
+                print("  hs...")
+                hs = handshake(req_ses, self.ar.url, file, self.ar.a, search)
+                if search:
+                    if hs:
+                        for hit in hs:
+                            print("  found: {0}{1}".format(self.ar.url, hit["rp"]))
+                    else:
+                        print("  NOT found")
+                    break
+
+                file.ucids = hs
+                if not hs:
+                    break
+
+                print("{0} {1}".format(self.nfiles - nf, upath))
+                ncs = len(hs)
+                for nc, cid in enumerate(hs):
+                    print("  {0} up {1}".format(ncs - nc, cid))
+                    upload(req_ses, file, cid, self.ar.a)
+
+            print("  ok!")
+
+    def fancy(self):
+        self.hash_f = 0
+        self.hash_c = 0
+        self.hash_b = 0
+        self.up_f = 0
+        self.up_c = 0
+        self.up_b = 0
+        self.up_br = 0
+        self.hasher_busy = 1
+        self.handshaker_busy = 0
+        self.uploader_busy = 0
+
+        self.t0 = time.time()
+        self.t0_up = None
+        self.spd = None
+
+        self.mutex = threading.Lock()
+        self.q_handshake = Queue()  # type: Queue[File]
+        self.q_recheck = Queue()  # type: Queue[File]  # partial upload exists [...]
+        self.q_upload = Queue()  # type: Queue[tuple[File, str]]
+
+        self.st_hash = [None, "(idle, starting...)"]  # type: tuple[File, int]
+        self.st_up = [None, "(idle, starting...)"]  # type: tuple[File, int]
+        if VT100:
+            atexit.register(self.cleanup_vt100)
+            ss.scroll_region(3)
+
+        Daemon(target=self.hasher).start()
+        for _ in range(self.ar.j):
+            Daemon(target=self.handshaker).start()
+            Daemon(target=self.uploader).start()
+
+        idles = 0
+        while idles < 3:
+            time.sleep(0.07)
+            with self.mutex:
+                if (
+                    self.q_handshake.empty()
+                    and self.q_upload.empty()
+                    and not self.hasher_busy
+                    and not self.handshaker_busy
+                    and not self.uploader_busy
+                ):
+                    idles += 1
+                else:
+                    idles = 0
+
+            if VT100:
+                maxlen = ss.w - len(str(self.nfiles)) - 14
+                txt = "\033[s\033[{0}H".format(ss.g)
+                for y, k, st, f in [
+                    [0, "hash", self.st_hash, self.hash_f],
+                    [1, "send", self.st_up, self.up_f],
+                ]:
+                    txt += "\033[{0}H{1}:".format(ss.g + y, k)
+                    file, arg = st
+                    if not file:
+                        txt += " {0}\033[K".format(arg)
+                    else:
+                        if y:
+                            p = 100 * file.up_b / file.size
+                        else:
+                            p = 100 * arg / file.size
+
+                        name = file.abs.decode("utf-8", "replace")[-maxlen:]
+                        if "/" in name:
+                            name = "\033[36m{0}\033[0m/{1}".format(*name.rsplit("/", 1))
+
+                        m = "{0:6.1f}% {1} {2}\033[K"
+                        txt += m.format(p, self.nfiles - f, name)
+
+                txt += "\033[{0}H ".format(ss.g + 2)
+            else:
+                txt = " "
+
+            if not self.up_br:
+                spd = self.hash_b / (time.time() - self.t0)
+                eta = (self.nbytes - self.hash_b) / (spd + 1)
+            else:
+                spd = self.up_br / (time.time() - self.t0_up)
+                spd = self.spd = (self.spd or spd) * 0.9 + spd * 0.1
+                eta = (self.nbytes - self.up_b) / (spd + 1)
+
+            spd = humansize(spd)
+            eta = str(datetime.timedelta(seconds=int(eta)))
+            left = humansize(self.nbytes - self.up_b)
+            tail = "\033[K\033[u" if VT100 else "\r"
+
+            m = "eta: {0} @ {1}/s, {2} left".format(eta, spd, left)
+            eprint(txt + "\033]0;{0}\033\\\r{1}{2}".format(m, m, tail))
+
+    def cleanup_vt100(self):
+        ss.scroll_region(None)
+        eprint("\033[J\033]0;\033\\")
+
+    def cb_hasher(self, file, ofs):
+        self.st_hash = [file, ofs]
+
+    def hasher(self):
+        for top, rel, inf in self.filegen:
+            file = File(top, rel, inf.st_size, inf.st_mtime)
+            while True:
+                with self.mutex:
+                    if (
+                        self.hash_b - self.up_b < 1024 * 1024 * 128
+                        and self.hash_c - self.up_c < 64
+                        and (
+                            not self.ar.nh
+                            or (
+                                self.q_upload.empty()
+                                and self.q_handshake.empty()
+                                and not self.uploader_busy
+                            )
+                        )
+                    ):
+                        break
+
+                time.sleep(0.05)
+
+            get_hashlist(file, self.cb_hasher)
+            with self.mutex:
+                self.hash_f += 1
+                self.hash_c += len(file.cids)
+                self.hash_b += file.size
+
+            self.q_handshake.put(file)
+
+        self.hasher_busy = 0
+        self.st_hash = [None, "(finished)"]
+
+    def handshaker(self):
+        search = self.ar.s
+        q = self.q_handshake
+        while True:
+            file = q.get()
+            if not file:
+                if q == self.q_handshake:
+                    q = self.q_recheck
+                    q.put(None)
+                    continue
+
+                self.q_upload.put(None)
+                break
+
+            with self.mutex:
+                self.handshaker_busy += 1
+
+            upath = file.abs.decode("utf-8", "replace")
+
+            try:
+                hs = handshake(req_ses, self.ar.url, file, self.ar.a, search)
+            except Exception as ex:
+                if q == self.q_handshake and "<pre>partial upload exists" in str(ex):
+                    self.q_recheck.put(file)
+                    hs = []
+                else:
+                    raise
+
+            if search:
+                if hs:
+                    for hit in hs:
+                        m = "found: {0}\n  {1}{2}\n"
+                        print(m.format(upath, self.ar.url, hit["rp"]), end="")
+                else:
+                    print("NOT found: {0}\n".format(upath), end="")
+
+                with self.mutex:
+                    self.up_f += 1
+                    self.up_c += len(file.cids)
+                    self.up_b += file.size
+                    self.handshaker_busy -= 1
+
+                continue
+
+            with self.mutex:
+                if not hs:
+                    # all chunks done
+                    self.up_f += 1
+                    self.up_c += len(file.cids) - file.up_c
+                    self.up_b += file.size - file.up_b
+
+                if hs and file.up_c:
+                    # some chunks failed
+                    self.up_c -= len(hs)
+                    file.up_c -= len(hs)
+                    for cid in hs:
+                        sz = file.kchunks[cid][1]
+                        self.up_b -= sz
+                        file.up_b -= sz
+
+                file.ucids = hs
+                self.handshaker_busy -= 1
+
+            if not hs:
+                print("uploaded {0}".format(upath))
+            for cid in hs:
+                self.q_upload.put([file, cid])
+
+    def uploader(self):
+        while True:
+            task = self.q_upload.get()
+            if not task:
+                self.st_up = [None, "(finished)"]
+                break
+
+            with self.mutex:
+                self.uploader_busy += 1
+                self.t0_up = self.t0_up or time.time()
+
+            file, cid = task
+            try:
+                upload(req_ses, file, cid, self.ar.a)
+            except:
+                eprint("upload failed, retry...\n")
+                pass  # handshake will fix it
+
+            with self.mutex:
+                sz = file.kchunks[cid][1]
+                file.ucids = [x for x in file.ucids if x != cid]
+                if not file.ucids:
+                    self.q_handshake.put(file)
+
+                self.st_up = [file, cid]
+                file.up_b += sz
+                self.up_b += sz
+                self.up_br += sz
+                file.up_c += 1
+                self.up_c += 1
+                self.uploader_busy -= 1
+
+
+def main():
+    time.strptime("19970815", "%Y%m%d")  # python#7980
+    if not VT100:
+        os.system("rem")  # enables colors
+
+    # fmt: off
+    ap = app = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    ap.add_argument("url", type=unicode, help="server url, including destination folder")
+    ap.add_argument("files", type=unicode, nargs="+", help="files and/or folders to process")
+    ap.add_argument("-a", metavar="PASSWORD", help="password")
+    ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
+    ap = app.add_argument_group("performance tweaks")
+    ap.add_argument("-j", type=int, metavar="THREADS", default=4, help="parallel connections")
+    ap.add_argument("-nh", action="store_true", help="disable hashing while uploading")
+    ap.add_argument("--safe", action="store_true", help="use simple fallback approach")
+    ap = app.add_argument_group("tls")
+    ap.add_argument("-te", metavar="PEM_FILE", help="certificate to expect/verify")
+    ap.add_argument("-td", action="store_true", help="disable certificate check")
+    # fmt: on
+
+    Ctl(app.parse_args())
+
+
+if __name__ == "__main__":
+    main()

bin/up2k.sh

@@ -8,7 +8,7 @@ set -e
 ##
 ## config
 
-datalen=$((2*1024*1024*1024))
+datalen=$((128*1024*1024))
 target=127.0.0.1
 posturl=/inc
 passwd=wark
@@ -37,10 +37,10 @@ gendata() {
 # pipe a chunk, get the base64 checksum
 gethash() {
     printf $(
-        sha512sum | cut -c-64 |
+        sha512sum | cut -c-66 |
         sed -r 's/ .*//;s/(..)/\\x\1/g'
     ) |
-    base64 -w0 | cut -c-43 |
+    base64 -w0 | cut -c-44 |
     tr '+/' '-_'
 }
 
@@ -123,7 +123,7 @@ printf '\033[36m'
 {
     {
         cat <<EOF
-POST $posturl/handshake.php HTTP/1.1
+POST $posturl/ HTTP/1.1
 Connection: Close
 Cookie: cppwd=$passwd
 Content-Type: text/plain;charset=UTF-8
@@ -145,14 +145,16 @@ printf '\033[0m\nwark: %s\n' $wark
 ##
 ## wait for signal to continue
 
-w8=/dev/shm/$salt.w8
+true || {
-touch $w8
+    w8=/dev/shm/$salt.w8
+    touch $w8
 
-echo "ready;  rm -f $w8"
+    echo "ready;  rm -f $w8"
 
-while [ -e $w8 ]; do
+    while [ -e $w8 ]; do
         sleep 0.2
-done
+    done
+}
 
 
 ##
@@ -175,7 +177,7 @@ while [ $remains -gt 0 ]; do
     
     {
         cat <<EOF
-POST $posturl/chunkpit.php HTTP/1.1
+POST $posturl/ HTTP/1.1
 Connection: Keep-Alive
 Cookie: cppwd=$passwd
 Content-Type: application/octet-stream

contrib/README.md

@@ -29,7 +29,8 @@ however if your copyparty is behind a reverse-proxy, you may want to use [`share
 
 # OS integration
 init-scripts to start copyparty as a service
-* [`systemd/copyparty.service`](systemd/copyparty.service)
+* [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
+* [`systemd/prisonparty.service`](systemd/prisonparty.service) runs the sfx in a chroot
 * [`openrc/copyparty`](openrc/copyparty)
 
 # Reverse-proxy

contrib/cfssl.sh

@@ -1,13 +1,14 @@
 #!/bin/bash
 set -e
 
-# ca-name and server-name
+# ca-name and server-fqdn
 ca_name="$1"
-srv_name="$2"
+srv_fqdn="$2"
 
-[ -z "$srv_name" ] && {
+[ -z "$srv_fqdn" ] && {
 	echo "need arg 1: ca name"
-	echo "need arg 2: server name"
+	echo "need arg 2: server fqdn"
+	echo "optional arg 3: if set, write cert into copyparty cfg"
 	exit 1
 }
 
@@ -31,15 +32,15 @@ EOF
 gen_srv() {
 	(tee /dev/stderr <<EOF
 {"key": {"algo":"rsa", "size":4096},
-"names": [{"O":"$ca_name - $srv_name"}]}
+"names": [{"O":"$ca_name - $srv_fqdn"}]}
 EOF
 	)|
 	cfssl gencert -ca ca.pem -ca-key ca.key \
-		-profile=www -hostname="$srv_name.$ca_name" - |
+		-profile=www -hostname="$srv_fqdn" - |
-	cfssljson -bare "$srv_name"
+	cfssljson -bare "$srv_fqdn"
 
-	mv "$srv_name-key.pem" "$srv_name.key"
+	mv "$srv_fqdn-key.pem" "$srv_fqdn.key"
-	rm "$srv_name.csr"
+	rm "$srv_fqdn.csr"
 }
 
 
@@ -57,13 +58,13 @@ show() {
 	awk '!o; {o=0} /[0-9a-f:]{16}/{o=1}'
 }
 show ca.pem
-show "$srv_name.pem"
+show "$srv_fqdn.pem"
 
 
 # write cert into copyparty config
 [ -z "$3" ] || {
 	mkdir -p ~/.config/copyparty
-	cat "$srv_name".{key,pem} ca.pem >~/.config/copyparty/cert.pem 
+	cat "$srv_fqdn".{key,pem} ca.pem >~/.config/copyparty/cert.pem 
 }
 
 

copyparty/__main__.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
@@ -104,7 +104,7 @@ def ensure_cert():
     cert_insec = os.path.join(E.mod, "res/insecure.pem")
     cert_cfg = os.path.join(E.cfg, "cert.pem")
     if not os.path.exists(cert_cfg):
-        shutil.copy2(cert_insec, cert_cfg)
+        shutil.copy(cert_insec, cert_cfg)
 
     try:
         if filecmp.cmp(cert_cfg, cert_insec):
@@ -203,6 +203,11 @@ def run_argparse(argv, formatter):
         description="http file sharing hub v{} ({})".format(S_VERSION, S_BUILD_DT),
     )
 
+    try:
+        fk_salt = unicode(os.path.getmtime(os.path.join(E.cfg, "cert.pem")))
+    except:
+        fk_salt = "hunter2"
+
     sects = [
         [
             "accounts",
@@ -211,14 +216,15 @@ def run_argparse(argv, formatter):
                 """
             -a takes username:password,
             -v takes src:dst:perm1:perm2:permN:volflag1:volflag2:volflagN:...
-               where "perm" is "accesslevels,username1,username2,..."
+               where "perm" is "permissions,username1,username2,..."
                and "volflag" is config flags to set on this volume
             
-            list of accesslevels:
+            list of permissions:
               "r" (read):   list folder contents, download files
               "w" (write):  upload files; need "r" to see the uploads
               "m" (move):   move files and folders; need "w" at destination
               "d" (delete): permanently delete files and folders
+              "g" (get):    download files, but cannot see folder contents
 
             too many volflags to list here, see the other sections
 
@@ -279,6 +285,10 @@ def run_argparse(argv, formatter):
               \033[36mmtp=.bpm=f,audio-bpm.py\033[35m uses the "audio-bpm.py" program to
                 generate ".bpm" tags from uploads (f = overwrite tags)
               \033[36mmtp=ahash,vhash=media-hash.py\033[35m collects two tags at once
+            
+            \033[0mothers:
+              \033[36mfk=8\033[35m generates per-file accesskeys,
+                which will then be required at the "g" permission
             \033[0m"""
             ),
         ],
@@ -334,6 +344,9 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="up2k min.size threshold (mswin-only)")
     ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled")
+    ap2.add_argument("--no-fpool", action="store_true", help="disable file-handle pooling -- instead, repeatedly close and reopen files during upload")
+    ap2.add_argument("--use-fpool", action="store_true", help="force file-handle pooling, even if copyparty thinks you're better off without")
+    ap2.add_argument("--no-symlink", action="store_true", help="duplicate file contents instead")
 
     ap2 = ap.add_argument_group('network options')
     ap2.add_argument("-i", metavar="IP", type=u, default="0.0.0.0", help="ip to bind (comma-sep.)")
@@ -360,6 +373,12 @@ def run_argparse(argv, formatter):
     ap2 = ap.add_argument_group('safety options')
     ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="scan all volumes; arguments USER,VOL,FLAGS; example [**,*,ln,p,r]")
     ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt")
+    ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt")
+    ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
+    ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to make something a dotfile")
+    ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings")
+    ap2.add_argument("--no-readme", action="store_true", help="disable rendering readme.md into directory listings")
+    ap2.add_argument("--vague-403", action="store_true", help="send 404 instead of 403 (security through ambiguity, very enterprise)")
 
     ap2 = ap.add_argument_group('logging options')
     ap2.add_argument("-q", action="store_true", help="quiet")
@@ -378,6 +397,7 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--no-thumb", action="store_true", help="disable all thumbnails")
     ap2.add_argument("--no-vthumb", action="store_true", help="disable video thumbnails")
     ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res")
+    ap2.add_argument("--th-mt", metavar="CORES", type=int, default=0, help="max num cpu cores to use, 0=all")
     ap2.add_argument("--th-no-crop", action="store_true", help="dynamic height; show full image")
     ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
     ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
@@ -488,7 +508,7 @@ def main(argv=None):
             if re.match("c[^,]", opt):
                 mod = True
                 na.append("c," + opt[1:])
-            elif re.sub("^[rwmd]*", "", opt) and "," not in opt:
+            elif re.sub("^[rwmdg]*", "", opt) and "," not in opt:
                 mod = True
                 perm = opt[0]
                 if perm == "a":

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 13, 2)
+VERSION = (1, 0, 8)
-CODENAME = "future-proof"
+CODENAME = "sufficient"
-BUILD_DT = (2021, 8, 12)
+BUILD_DT = (2021, 10, 4)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -29,17 +29,18 @@ LEELOO_DALLAS = "leeloo_dallas"
 
 
 class AXS(object):
-    def __init__(self, uread=None, uwrite=None, umove=None, udel=None):
+    def __init__(self, uread=None, uwrite=None, umove=None, udel=None, uget=None):
         self.uread = {} if uread is None else {k: 1 for k in uread}
         self.uwrite = {} if uwrite is None else {k: 1 for k in uwrite}
         self.umove = {} if umove is None else {k: 1 for k in umove}
         self.udel = {} if udel is None else {k: 1 for k in udel}
+        self.uget = {} if uget is None else {k: 1 for k in uget}
 
     def __repr__(self):
         return "AXS({})".format(
             ", ".join(
                 "{}={!r}".format(k, self.__dict__[k])
-                for k in "uread uwrite umove udel".split()
+                for k in "uread uwrite umove udel uget".split()
             )
         )
 
@@ -215,6 +216,7 @@ class VFS(object):
             self.awrite = {}
             self.amove = {}
             self.adel = {}
+            self.aget = {}
         else:
             self.histpath = None
             self.all_vols = None
@@ -222,6 +224,7 @@ class VFS(object):
             self.awrite = None
             self.amove = None
             self.adel = None
+            self.aget = None
 
     def __repr__(self):
         return "VFS({})".format(
@@ -308,7 +311,7 @@ class VFS(object):
 
     def can_access(self, vpath, uname):
         # type: (str, str) -> tuple[bool, bool, bool, bool]
-        """can Read,Write,Move,Delete"""
+        """can Read,Write,Move,Delete,Get"""
         vn, _ = self._find(vpath)
         c = vn.axs
         return [
@@ -316,10 +319,20 @@ class VFS(object):
             uname in c.uwrite or "*" in c.uwrite,
             uname in c.umove or "*" in c.umove,
             uname in c.udel or "*" in c.udel,
+            uname in c.uget or "*" in c.uget,
         ]
 
-    def get(self, vpath, uname, will_read, will_write, will_move=False, will_del=False):
+    def get(
-        # type: (str, str, bool, bool, bool, bool) -> tuple[VFS, str]
+        self,
+        vpath,
+        uname,
+        will_read,
+        will_write,
+        will_move=False,
+        will_del=False,
+        will_get=False,
+    ):
+        # type: (str, str, bool, bool, bool, bool, bool) -> tuple[VFS, str]
         """returns [vfsnode,fs_remainder] if user has the requested permissions"""
         vn, rem = self._find(vpath)
         c = vn.axs
@@ -329,6 +342,7 @@ class VFS(object):
             [will_write, c.uwrite, "write"],
             [will_move, c.umove, "move"],
             [will_del, c.udel, "delete"],
+            [will_get, c.uget, "get"],
         ]:
             if req and (uname not in d and "*" not in d) and uname != LEELOO_DALLAS:
                 m = "you don't have {}-access for this location"
@@ -342,7 +356,7 @@ class VFS(object):
         if not dbv:
             return self, vrem
 
-        vrem = [self.vpath[len(dbv.vpath) + 1 :], vrem]
+        vrem = [self.vpath[len(dbv.vpath) :].lstrip("/"), vrem]
         vrem = "/".join([x for x in vrem if x])
         return dbv, vrem
 
@@ -368,7 +382,7 @@ class VFS(object):
             for name, vn2 in sorted(self.nodes.items()):
                 ok = False
                 axs = vn2.axs
-                axs = [axs.uread, axs.uwrite, axs.umove, axs.udel]
+                axs = [axs.uread, axs.uwrite, axs.umove, axs.udel, axs.uget]
                 for pset in permsets:
                     ok = True
                     for req, lst in zip(pset, axs):
@@ -434,7 +448,11 @@ class VFS(object):
         f2a = os.sep + "dir.txt"
         f2b = "{0}.hist{0}".format(os.sep)
 
-        g = self.walk("", vrem, [], uname, [[True]], dots, scandir, False)
+        # if multiselect: add all items to archive root
+        # if single folder: the folder itself is the top-level item
+        folder = "" if flt else (vrem.split("/")[-1] or "top")
+
+        g = self.walk(folder, vrem, [], uname, [[True]], dots, scandir, False)
         for _, _, vpath, apath, files, rd, vd in g:
             if flt:
                 files = [x for x in files if x[0] in flt]
@@ -557,13 +575,21 @@ class AuthSrv(object):
 
     def _read_vol_str(self, lvl, uname, axs, flags):
         # type: (str, str, AXS, any) -> None
-        if lvl.strip("crwmd"):
+        if lvl.strip("crwmdg"):
             raise Exception("invalid volume flag: {},{}".format(lvl, uname))
 
         if lvl == "c":
-            cval = True
+            try:
-            if "=" in uname:
+                # volume flag with arguments, possibly with a preceding list of bools
                 uname, cval = uname.split("=", 1)
+            except:
+                # just one or more bools
+                cval = True
+
+            while "," in uname:
+                # one or more bools before the final flag; eat them
+                n1, uname = uname.split(",", 1)
+                self._read_volflag(flags, n1, True, False)
 
             self._read_volflag(flags, uname, cval, False)
             return
@@ -584,6 +610,9 @@ class AuthSrv(object):
             if "d" in lvl:
                 axs.udel[un] = 1
 
+            if "g" in lvl:
+                axs.uget[un] = 1
+
     def _read_volflag(self, flags, name, value, is_list):
         if name not in ["mtp"]:
             flags[name] = value
@@ -621,7 +650,7 @@ class AuthSrv(object):
 
         if self.args.v:
             # list of src:dst:permset:permset:...
-            # permset is <rwmd>[,username][,username] or <c>,<flag>[=args]
+            # permset is <rwmdg>[,username][,username] or <c>,<flag>[=args]
             for v_str in self.args.v:
                 m = re_vol.match(v_str)
                 if not m:
@@ -688,20 +717,21 @@ class AuthSrv(object):
         vfs.all_vols = {}
         vfs.get_all_vols(vfs.all_vols)
 
-        for perm in "read write move del".split():
+        for perm in "read write move del get".split():
             axs_key = "u" + perm
             unames = ["*"] + list(acct.keys())
             umap = {x: [] for x in unames}
             for usr in unames:
                 for mp, vol in vfs.all_vols.items():
-                    if usr in getattr(vol.axs, axs_key):
+                    axs = getattr(vol.axs, axs_key)
+                    if usr in axs or "*" in axs:
                         umap[usr].append(mp)
             setattr(vfs, "a" + perm, umap)
 
         all_users = {}
         missing_users = {}
         for axs in daxs.values():
-            for d in [axs.uread, axs.uwrite, axs.umove, axs.udel]:
+            for d in [axs.uread, axs.uwrite, axs.umove, axs.udel, axs.uget]:
                 for usr in d.keys():
                     all_users[usr] = 1
                     if usr != "*" and usr not in acct:
@@ -812,6 +842,11 @@ class AuthSrv(object):
             if use:
                 vol.lim = lim
 
+        for vol in vfs.all_vols.values():
+            fk = vol.flags.get("fk")
+            if fk:
+                vol.flags["fk"] = int(fk) if fk is not True else 8
+
         for vol in vfs.all_vols.values():
             if "pk" in vol.flags and "gz" not in vol.flags and "xz" not in vol.flags:
                 vol.flags["gz"] = False  # def.pk
@@ -845,6 +880,10 @@ class AuthSrv(object):
             # default tag cfgs if unset
             if "mte" not in vol.flags:
                 vol.flags["mte"] = self.args.mte
+            elif vol.flags["mte"].startswith("+"):
+                vol.flags["mte"] = ",".join(
+                    x for x in [self.args.mte, vol.flags["mte"][1:]] if x
+                )
             if "mth" not in vol.flags:
                 vol.flags["mth"] = self.args.mth
 
@@ -926,6 +965,7 @@ class AuthSrv(object):
                 [" write", "uwrite"],
                 ["  move", "umove"],
                 ["delete", "udel"],
+                ["   get", "uget"],
             ]:
                 u = list(sorted(getattr(v.axs, attr).keys()))
                 u = ", ".join("\033[35meverybody\033[0m" if x == "*" else x for x in u)
@@ -993,10 +1033,10 @@ class AuthSrv(object):
                 raise Exception("volume not found: " + v)
 
         self.log({"users": users, "vols": vols, "flags": flags})
-        m = "/{}: read({}) write({}) move({}) del({})"
+        m = "/{}: read({}) write({}) move({}) del({}) get({})"
         for k, v in self.vfs.all_vols.items():
             vc = v.axs
-            self.log(m.format(k, vc.uread, vc.uwrite, vc.umove, vc.udel))
+            self.log(m.format(k, vc.uread, vc.uwrite, vc.umove, vc.udel, vc.uget))
 
         flag_v = "v" in flags
         flag_ln = "ln" in flags
@@ -1010,7 +1050,7 @@ class AuthSrv(object):
             for u in users:
                 self.log("checking /{} as {}".format(v, u))
                 try:
-                    vn, _ = self.vfs.get(v, u, True, False, False, False)
+                    vn, _ = self.vfs.get(v, u, True, False, False, False, False)
                 except:
                     continue
 

copyparty/httpcli.py

@@ -7,10 +7,12 @@ import gzip
 import time
 import copy
 import json
+import base64
 import string
 import socket
 import ctypes
 from datetime import datetime
+from operator import itemgetter
 import calendar
 
 try:
@@ -21,13 +23,12 @@ except:
 from .__init__ import E, PY2, WINDOWS, ANYWIN, unicode
 from .util import *  # noqa  # pylint: disable=unused-wildcard-import
 from .bos import bos
-from .authsrv import AuthSrv, Lim
+from .authsrv import AuthSrv
 from .szip import StreamZip
 from .star import StreamTar
 
 
 NO_CACHE = {"Cache-Control": "no-cache"}
-NO_STORE = {"Cache-Control": "no-store; max-age=0"}
 
 
 class HttpCli(object):
@@ -38,6 +39,7 @@ class HttpCli(object):
     def __init__(self, conn):
         self.t0 = time.time()
         self.conn = conn
+        self.mutex = conn.mutex
         self.s = conn.s  # type: socket
         self.sr = conn.sr  # type: Unrecv
         self.ip = conn.addr[0]
@@ -46,6 +48,7 @@ class HttpCli(object):
         self.asrv = conn.asrv  # type: AuthSrv
         self.ico = conn.ico
         self.thumbcli = conn.thumbcli
+        self.u2fh = conn.u2fh
         self.log_func = conn.log_func
         self.log_src = conn.log_src
         self.tls = hasattr(self.s, "cipher")
@@ -53,7 +56,10 @@ class HttpCli(object):
         self.bufsz = 1024 * 32
         self.hint = None
         self.absolute_urls = False
-        self.out_headers = {"Access-Control-Allow-Origin": "*"}
+        self.out_headers = {
+            "Access-Control-Allow-Origin": "*",
+            "Cache-Control": "no-store; max-age=0",
+        }
 
     def log(self, msg, c=0):
         ptn = self.asrv.re_pwd
@@ -192,12 +198,27 @@ class HttpCli(object):
         self.cookies = cookies
         self.vpath = unquotep(vpath)  # not query, so + means +
 
-        pwd = uparam.get("pw")
+        pwd = None
+        ba = self.headers.get("authorization")
+        if ba:
+            try:
+                ba = ba.split(" ")[1].encode("ascii")
+                ba = base64.b64decode(ba).decode("utf-8")
+                # try "pwd", "x:pwd", "pwd:x"
+                for ba in [ba] + ba.split(":", 1)[::-1]:
+                    if self.asrv.iacct.get(ba):
+                        pwd = ba
+                        break
+            except:
+                pass
+
+        pwd = uparam.get("pw") or pwd
         self.uname = self.asrv.iacct.get(pwd, "*")
         self.rvol = self.asrv.vfs.aread[self.uname]
         self.wvol = self.asrv.vfs.awrite[self.uname]
         self.mvol = self.asrv.vfs.amove[self.uname]
         self.dvol = self.asrv.vfs.adel[self.uname]
+        self.gvol = self.asrv.vfs.aget[self.uname]
 
         if pwd and "pw" in self.ouparam and pwd != cookies.get("cppwd"):
             self.out_headers["Set-Cookie"] = self.get_pwd_cookie(pwd)[0]
@@ -212,6 +233,9 @@ class HttpCli(object):
 
         self.do_log = not self.conn.lf_url or not self.conn.lf_url.search(self.req)
 
+        x = self.asrv.vfs.can_access(self.vpath, self.uname)
+        self.can_read, self.can_write, self.can_move, self.can_delete, self.can_get = x
+
         try:
             if self.mode in ["GET", "HEAD"]:
                 return self.handle_get() and self.keepalive
@@ -336,8 +360,7 @@ class HttpCli(object):
         ).encode("utf-8", "replace")
 
         if use302:
-            h = {"Location": "/" + vpath, "Cache-Control": "no-cache"}
+            self.reply(html, status=302, headers={"Location": "/" + vpath})
-            self.reply(html, status=302, headers=h)
         else:
             self.reply(html, status=status)
 
@@ -363,12 +386,10 @@ class HttpCli(object):
             static_path = os.path.join(E.mod, "web/", self.vpath[5:])
             return self.tx_file(static_path)
 
-        x = self.asrv.vfs.can_access(self.vpath, self.uname)
+        if not self.can_read and not self.can_write and not self.can_get:
-        self.can_read, self.can_write, self.can_move, self.can_delete = x
-        if not self.can_read and not self.can_write:
             if self.vpath:
                 self.log("inaccessible: [{}]".format(self.vpath))
-                raise Pebkac(404)
+                return self.tx_404(True)
 
             self.uparam["h"] = False
 
@@ -490,7 +511,7 @@ class HttpCli(object):
             if "get" in opt:
                 return self.handle_get()
 
-            raise Pebkac(405, "POST({}) is disabled".format(ctype))
+            raise Pebkac(405, "POST({}) is disabled in server config".format(ctype))
 
         raise Pebkac(405, "don't know how to handle POST({})".format(ctype))
 
@@ -816,7 +837,18 @@ class HttpCli(object):
 
         reader = read_socket(self.sr, remains)
 
-        with open(fsenc(path), "rb+", 512 * 1024) as f:
+        f = None
+        fpool = not self.args.no_fpool
+        if fpool:
+            with self.mutex:
+                try:
+                    f = self.u2fh.pop(path)
+                except:
+                    pass
+
+        f = f or open(fsenc(path), "rb+", 512 * 1024)
+
+        try:
             f.seek(cstart[0])
             post_sz, _, sha_b64 = hashcopy(reader, f)
 
@@ -846,22 +878,36 @@ class HttpCli(object):
                     ofs += len(buf)
 
                 self.log("clone {} done".format(cstart[0]))
+        finally:
+            if not fpool:
+                f.close()
+            else:
+                with self.mutex:
+                    self.u2fh.put(path, f)
 
         x = self.conn.hsrv.broker.put(True, "up2k.confirm_chunk", ptop, wark, chash)
         x = x.get()
         try:
-            num_left, path = x
+            num_left, fin_path = x
         except:
             self.loud_reply(x, status=500)
             return False
 
-        if not ANYWIN and num_left == 0:
+        if not num_left and fpool:
+            with self.mutex:
+                self.u2fh.close(path)
+
+        # windows cant rename open files
+        if ANYWIN and path != fin_path and not self.args.nw:
+            self.conn.hsrv.broker.put(True, "up2k.finish_upload", ptop, wark).get()
+
+        if not ANYWIN and not num_left:
             times = (int(time.time()), int(lastmod))
             self.log("no more chunks, setting times {}".format(times))
             try:
-                bos.utime(path, times)
+                bos.utime(fin_path, times)
             except:
-                self.log("failed to utime ({}, {})".format(path, times))
+                self.log("failed to utime ({}, {})".format(fin_path, times))
 
         spd = self._spd(post_sz)
         self.log("{} thank".format(spd))
@@ -872,8 +918,12 @@ class HttpCli(object):
         pwd = self.parser.require("cppwd", 64)
         self.parser.drop()
 
+        dst = "/?h"
+        if self.vpath:
+            dst = "/" + quotep(self.vpath)
+
         ck, msg = self.get_pwd_cookie(pwd)
-        html = self.j2("msg", h1=msg, h2='<a href="/">ack</a>', redir="/")
+        html = self.j2("msg", h1=msg, h2='<a href="' + dst + '">ack</a>', redir=dst)
         self.reply(html.encode("utf-8"), headers={"Set-Cookie": ck})
         return True
 
@@ -956,10 +1006,14 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
+        upload_vpath = self.vpath
         lim = vfs.get_dbv(rem)[0].lim
         fdir_base = os.path.join(vfs.realpath, rem)
         if lim:
             fdir_base, rem = lim.all(self.ip, rem, -1, fdir_base)
+            upload_vpath = "{}/{}".format(vfs.vpath, rem).strip("/")
+            if not nullwrite:
+                bos.makedirs(fdir_base)
 
         files = []
         errmsg = ""
@@ -986,8 +1040,6 @@ class HttpCli(object):
                 if lim:
                     lim.chk_bup(self.ip)
                     lim.chk_nup(self.ip)
-                    if not nullwrite:
-                        bos.makedirs(fdir)
 
                 try:
                     with ren_open(fname, "wb", 512 * 1024, **open_args) as f:
@@ -1007,7 +1059,7 @@ class HttpCli(object):
                             bos.unlink(abspath)
                             raise
 
-                    files.append([sz, sha512_hex, p_file, fname])
+                    files.append([sz, sha512_hex, p_file, fname, abspath])
                     dbv, vrem = vfs.get_dbv(rem)
                     self.conn.hsrv.broker.put(
                         False,
@@ -1038,7 +1090,9 @@ class HttpCli(object):
                     raise
 
         except Pebkac as ex:
-            errmsg = volsan(self.asrv.vfs.all_vols.values(), unicode(ex))
+            errmsg = vol_san(
+                self.asrv.vfs.all_vols.values(), unicode(ex).encode("utf-8")
+            ).decode("utf-8")
 
         td = max(0.1, time.time() - t0)
         sz_total = sum(x[0] for x in files)
@@ -1057,10 +1111,19 @@ class HttpCli(object):
             jmsg["error"] = errmsg
             errmsg = "ERROR: " + errmsg
 
-        for sz, sha512, ofn, lfn in files:
+        for sz, sha512, ofn, lfn, ap in files:
-            vpath = (self.vpath + "/" if self.vpath else "") + lfn
+            vsuf = ""
-            msg += 'sha512: {} // {} bytes // <a href="/{}">{}</a>\n'.format(
+            if self.can_read and "fk" in vfs.flags:
-                sha512[:56], sz, quotep(vpath), html_escape(ofn, crlf=True)
+                vsuf = "?k=" + gen_filekey(
+                    self.args.fk_salt,
+                    abspath,
+                    sz,
+                    0 if ANYWIN or not ap else bos.stat(ap).st_ino,
+                )[: vfs.flags["fk"]]
+
+            vpath = "{}/{}".format(upload_vpath, lfn).strip("/")
+            msg += 'sha512: {} // {} bytes // <a href="/{}">{}</a> {}\n'.format(
+                sha512[:56], sz, quotep(vpath) + vsuf, html_escape(ofn, crlf=True), vsuf
             )
             # truncated SHA-512 prevents length extension attacks;
             # using SHA-512/224, optionally SHA-512/256 = :64
@@ -1068,13 +1131,13 @@ class HttpCli(object):
                 "url": "{}://{}/{}".format(
                     "https" if self.tls else "http",
                     self.headers.get("host", "copyparty"),
-                    vpath,
+                    vpath + vsuf,
                 ),
                 "sha512": sha512[:56],
                 "sz": sz,
                 "fn": lfn,
                 "fn_orig": ofn,
-                "path": vpath,
+                "path": vpath + vsuf,
             }
             jmsg["files"].append(jpart)
 
@@ -1258,7 +1321,7 @@ class HttpCli(object):
                 break
 
         if not editions:
-            raise Pebkac(404)
+            return self.tx_404()
 
         #
         # if-modified
@@ -1369,7 +1432,11 @@ class HttpCli(object):
         #
         # send reply
 
-        if not is_compressed and "cache" not in self.uparam:
+        if is_compressed:
+            self.out_headers["Cache-Control"] = "max-age=573"
+        elif "cache" in self.uparam:
+            self.out_headers["Cache-Control"] = "max-age=69"
+        else:
             self.out_headers.update(NO_CACHE)
 
         self.out_headers["Accept-Ranges"] = "bytes"
@@ -1496,6 +1563,10 @@ class HttpCli(object):
     def tx_md(self, fs_path):
         logmsg = "{:4} {} ".format("", self.req)
 
+        if not self.can_write:
+            if "edit" in self.uparam or "edit2" in self.uparam:
+                return self.tx_404(True)
+
         tpl = "mde" if "edit2" in self.uparam else "md"
         html_path = os.path.join(E.mod, "web", "{}.html".format(tpl))
         template = self.j2(tpl)
@@ -1518,6 +1589,10 @@ class HttpCli(object):
         self.out_headers.update(NO_CACHE)
         status = 200 if do_send else 304
 
+        arg_base = "?"
+        if "k" in self.uparam:
+            arg_base = "?k={}&".format(self.uparam["k"])
+
         boundary = "\roll\tide"
         targs = {
             "edit": "edit" in self.uparam,
@@ -1527,6 +1602,7 @@ class HttpCli(object):
             "md_chk_rate": self.args.mcr,
             "md": boundary,
             "ts": self.conn.hsrv.cachebuster(),
+            "arg_base": arg_base,
         }
         html = template.render(**targs).encode("utf-8", "replace")
         html = html.split(boundary.encode("utf-8"))
@@ -1577,6 +1653,7 @@ class HttpCli(object):
         html = self.j2(
             "splash",
             this=self,
+            qvpath=quotep(self.vpath),
             rvol=rvol,
             wvol=wvol,
             avol=avol,
@@ -1587,7 +1664,19 @@ class HttpCli(object):
             mtpq=vs["mtpq"],
             url_suf=suf,
         )
-        self.reply(html.encode("utf-8"), headers=NO_STORE)
+        self.reply(html.encode("utf-8"))
+        return True
+
+    def tx_404(self, is_403=False):
+        if self.args.vague_403:
+            m = '<h1>404 not found &nbsp;┐( ´ -`)┌</h1><p>or maybe you don\'t have access -- try logging in or <a href="/?h">go home</a></p>'
+        elif is_403:
+            m = '<h1>403 forbiddena &nbsp;~┻━┻</h1><p>you\'ll have to log in or <a href="/?h">go home</a></p>'
+        else:
+            m = '<h1>404 not found &nbsp;┐( ´ -`)┌</h1><p><a href="/?h">go home</a></p>'
+
+        html = self.j2("splash", this=self, qvpath=quotep(self.vpath), msg=m)
+        self.reply(html.encode("utf-8"), status=404)
         return True
 
     def scanvol(self):
@@ -1595,7 +1684,7 @@ class HttpCli(object):
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_rescan:
-            raise Pebkac(403, "disabled by argv")
+            raise Pebkac(403, "the rescan feature is disabled in server config")
 
         vn, _ = self.asrv.vfs.get(self.vpath, self.uname, True, True)
 
@@ -1614,7 +1703,7 @@ class HttpCli(object):
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_stack:
-            raise Pebkac(403, "disabled by argv")
+            raise Pebkac(403, "the stackdump feature is disabled in server config")
 
         ret = "<pre>{}\n{}".format(time.time(), alltrace())
         self.reply(ret.encode("utf-8"))
@@ -1678,7 +1767,7 @@ class HttpCli(object):
 
     def tx_ups(self):
         if not self.args.unpost:
-            raise Pebkac(400, "the unpost feature was disabled by server config")
+            raise Pebkac(400, "the unpost feature is disabled in server config")
 
         filt = self.uparam.get("filter")
         lm = "ups [{}]".format(filt)
@@ -1695,11 +1784,11 @@ class HttpCli(object):
 
             q = "select sz, rd, fn, at from up where ip=? and at>?"
             for sz, rd, fn, at in cur.execute(q, (self.ip, lim)):
-                vp = "/" + "/".join([rd, fn]).strip("/")
+                vp = "/" + "/".join(x for x in [vol.vpath, rd, fn] if x)
                 if filt and filt not in vp:
                     continue
 
-                ret.append({"vp": vp, "sz": sz, "at": at})
+                ret.append({"vp": quotep(vp), "sz": sz, "at": at})
                 if len(ret) > 3000:
                     ret.sort(key=lambda x: x["at"], reverse=True)
                     ret = ret[:2000]
@@ -1716,7 +1805,7 @@ class HttpCli(object):
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_del:
-            raise Pebkac(403, "disabled by argv")
+            raise Pebkac(403, "the delete feature is disabled in server config")
 
         if not req:
             req = [self.vpath]
@@ -1729,7 +1818,7 @@ class HttpCli(object):
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_mv:
-            raise Pebkac(403, "disabled by argv")
+            raise Pebkac(403, "the rename/move feature is disabled in server config")
 
         # full path of new loc (incl filename)
         dst = self.uparam.get("move")
@@ -1763,22 +1852,22 @@ class HttpCli(object):
         try:
             st = bos.stat(abspath)
         except:
-            raise Pebkac(404)
+            return self.tx_404()
 
-        if self.can_read:
         if rem.startswith(".hist/up2k.") or (
             rem.endswith("/dir.txt") and rem.startswith(".hist/th/")
         ):
             raise Pebkac(403)
 
         is_dir = stat.S_ISDIR(st.st_mode)
+        if self.can_read:
             th_fmt = self.uparam.get("th")
             if th_fmt is not None:
                 if is_dir:
                     for fn in self.args.th_covers.split(","):
                         fp = os.path.join(abspath, fn)
                         if bos.path.exists(fp):
-                            vrem = "{}/{}".format(vrem.rstrip("/"), fn)
+                            vrem = "{}/{}".format(vrem.rstrip("/"), fn).strip("/")
                             is_dir = False
                             break
 
@@ -1796,12 +1885,24 @@ class HttpCli(object):
 
                 return self.tx_ico(rem)
 
-            if not is_dir:
+        if not is_dir and (self.can_read or self.can_get):
+            if not self.can_read and "fk" in vn.flags:
+                correct = gen_filekey(
+                    self.args.fk_salt, abspath, st.st_size, 0 if ANYWIN else st.st_ino
+                )[: vn.flags["fk"]]
+                got = self.uparam.get("k")
+                if got != correct:
+                    self.log("wrong filekey, want {}, got {}".format(correct, got))
+                    return self.tx_404()
+
             if abspath.endswith(".md") and "raw" not in self.uparam:
                 return self.tx_md(abspath)
 
             return self.tx_file(abspath)
 
+        elif is_dir and not self.can_read and not self.can_write:
+            return self.tx_404(True)
+
         srv_info = []
 
         try:
@@ -1840,6 +1941,8 @@ class HttpCli(object):
             perms.append("move")
         if self.can_delete:
             perms.append("delete")
+        if self.can_get:
+            perms.append("get")
 
         url_suf = self.urlq({}, [])
         is_ls = "ls" in self.uparam
@@ -1849,12 +1952,22 @@ class HttpCli(object):
             tpl = "browser2"
 
         logues = ["", ""]
+        if not self.args.no_logues:
             for n, fn in enumerate([".prologue.html", ".epilogue.html"]):
                 fn = os.path.join(abspath, fn)
                 if bos.path.exists(fn):
                     with open(fsenc(fn), "rb") as f:
                         logues[n] = f.read().decode("utf-8")
 
+        readme = ""
+        if not self.args.no_readme and not logues[1]:
+            for fn in ["README.md", "readme.md"]:
+                fn = os.path.join(abspath, fn)
+                if bos.path.exists(fn):
+                    with open(fsenc(fn), "rb") as f:
+                        readme = f.read().decode("utf-8")
+                        break
+
         ls_ret = {
             "dirs": [],
             "files": [],
@@ -1863,6 +1976,7 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": perms,
             "logues": logues,
+            "readme": readme,
         }
         j2a = {
             "vdir": quotep(self.vpath),
@@ -1881,24 +1995,24 @@ class HttpCli(object):
             "have_b_u": (self.can_write and self.uparam.get("b") == "u"),
             "url_suf": url_suf,
             "logues": logues,
+            "readme": readme,
             "title": html_escape(self.vpath, crlf=True),
             "srv_info": srv_info,
         }
         if not self.can_read:
             if is_ls:
                 ret = json.dumps(ls_ret)
-                self.reply(
+                self.reply(ret.encode("utf-8", "replace"), mime="application/json")
-                    ret.encode("utf-8", "replace"),
-                    mime="application/json",
-                    headers=NO_STORE,
-                )
                 return True
 
             if not stat.S_ISDIR(st.st_mode):
-                raise Pebkac(404)
+                return self.tx_404(True)
+
+            if "zip" in self.uparam or "tar" in self.uparam:
+                raise Pebkac(403)
 
             html = self.j2(tpl, **j2a)
-            self.reply(html.encode("utf-8", "replace"), headers=NO_STORE)
+            self.reply(html.encode("utf-8", "replace"))
             return True
 
         for k in ["zip", "tar"]:
@@ -1942,6 +2056,8 @@ class HttpCli(object):
             idx = self.conn.get_u2idx()
             icur = idx.get_cur(dbv.realpath)
 
+        add_fk = vn.flags.get("fk")
+
         dirs = []
         files = []
         for fn in vfs_ls:
@@ -1987,9 +2103,19 @@ class HttpCli(object):
             except:
                 ext = "%"
 
+            if add_fk:
+                href = "{}?k={}".format(
+                    quotep(href),
+                    gen_filekey(
+                        self.args.fk_salt, fspath, sz, 0 if ANYWIN else inf.st_ino
+                    )[:add_fk],
+                )
+            else:
+                href = quotep(href)
+
             item = {
                 "lead": margin,
-                "href": quotep(href),
+                "href": href,
                 "name": fn,
                 "sz": sz,
                 "ext": ext,
@@ -2057,13 +2183,14 @@ class HttpCli(object):
             ls_ret["files"] = files
             ls_ret["taglist"] = taglist
             ret = json.dumps(ls_ret)
-            self.reply(
+            self.reply(ret.encode("utf-8", "replace"), mime="application/json")
-                ret.encode("utf-8", "replace"),
-                mime="application/json",
-                headers=NO_STORE,
-            )
             return True
 
+        for d in dirs:
+            d["name"] += "/"
+
+        dirs.sort(key=itemgetter("name"))
+
         j2a["files"] = dirs + files
         j2a["logues"] = logues
         j2a["taglist"] = taglist
@@ -2075,5 +2202,5 @@ class HttpCli(object):
             j2a["css"] = self.args.css_browser
 
         html = self.j2(tpl, **j2a)
-        self.reply(html.encode("utf-8", "replace"), headers=NO_STORE)
+        self.reply(html.encode("utf-8", "replace"))
         return True

copyparty/httpconn.py

@@ -32,9 +32,11 @@ class HttpConn(object):
         self.addr = addr
         self.hsrv = hsrv
 
+        self.mutex = hsrv.mutex
         self.args = hsrv.args
         self.asrv = hsrv.asrv
         self.cert_path = hsrv.cert_path
+        self.u2fh = hsrv.u2fh
 
         enth = HAVE_PIL and not self.args.no_thumb
         self.thumbcli = ThumbCli(hsrv.broker) if enth else None

copyparty/httpsrv.py

@@ -27,7 +27,7 @@ except ImportError:
     sys.exit(1)
 
 from .__init__ import E, PY2, MACOS
-from .util import spack, min_ex, start_stackmon, start_log_thrs
+from .util import FHC, spack, min_ex, start_stackmon, start_log_thrs
 from .bos import bos
 from .httpconn import HttpConn
 
@@ -50,7 +50,10 @@ class HttpSrv(object):
         self.log = broker.log
         self.asrv = broker.asrv
 
-        self.name = "httpsrv" + ("-n{}-i{:x}".format(nid, os.getpid()) if nid else "")
+        nsuf = "-{}".format(nid) if nid else ""
+        nsuf2 = "-n{}-i{:x}".format(nid, os.getpid()) if nid else ""
+
+        self.name = "hsrv" + nsuf2
         self.mutex = threading.Lock()
         self.stopping = False
 
@@ -59,6 +62,7 @@ class HttpSrv(object):
         self.tp_time = None  # latest worker collect
         self.tp_q = None if self.args.no_htp else queue.LifoQueue()
 
+        self.u2fh = FHC()
         self.srvs = []
         self.ncli = 0  # exact
         self.clients = {}  # laggy
@@ -82,11 +86,6 @@ class HttpSrv(object):
         if self.tp_q:
             self.start_threads(4)
 
-            name = "httpsrv-scaler" + ("-{}".format(nid) if nid else "")
-            t = threading.Thread(target=self.thr_scaler, name=name)
-            t.daemon = True
-            t.start()
-
         if nid:
             if self.args.stackmon:
                 start_stackmon(self.args.stackmon, nid)
@@ -94,6 +93,10 @@ class HttpSrv(object):
             if self.args.log_thrs:
                 start_log_thrs(self.log, self.args.log_thrs, nid)
 
+        t = threading.Thread(target=self.periodic, name="hsrv-pt" + nsuf)
+        t.daemon = True
+        t.start()
+
     def start_threads(self, n):
         self.tp_nthr += n
         if self.args.log_htp:
@@ -115,10 +118,12 @@ class HttpSrv(object):
         for _ in range(n):
             self.tp_q.put(None)
 
-    def thr_scaler(self):
+    def periodic(self):
         while True:
-            time.sleep(2 if self.tp_ncli else 30)
+            time.sleep(2 if self.tp_ncli else 10)
             with self.mutex:
+                self.u2fh.clean()
+                if self.tp_q:
                     self.tp_ncli = max(self.ncli, self.tp_ncli - 2)
                     if self.tp_nthr > self.tp_ncli + 8:
                         self.stop_threads(4)

copyparty/stolen/surrogateescape.py

@@ -1,3 +1,5 @@
+# coding: utf-8
+
 """
 This is Victor Stinner's pure-Python implementation of PEP 383: the "surrogateescape" error
 handler of Python 3.
@@ -171,7 +173,7 @@ FS_ENCODING = sys.getfilesystemencoding()
 
 if WINDOWS and not PY3:
     # py2 thinks win* is mbcs, probably a bug? anyways this works
-    FS_ENCODING = 'utf-8'
+    FS_ENCODING = "utf-8"
 
 
 # normalize the filesystem encoding name.

copyparty/svchub.py

@@ -1,7 +1,6 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
-import re
 import os
 import sys
 import time
@@ -54,6 +53,17 @@ class SvcHub(object):
         if args.log_thrs:
             start_log_thrs(self.log, args.log_thrs, 0)
 
+        if not ANYWIN and not args.use_fpool:
+            args.no_fpool = True
+
+        if not args.no_fpool and args.j != 1:
+            m = "WARNING: --use-fpool combined with multithreading is untested and can probably cause undefined behavior"
+            if ANYWIN:
+                m = "windows cannot do multithreading without --no-fpool, so enabling that -- note that upload performance will suffer if you have microsoft defender \"real-time protection\" enabled, so you probably want to use -j 1 instead"
+                args.no_fpool = True
+            
+            self.log("root", m, c=3)
+
         # initiate all services to manage
         self.asrv = AuthSrv(self.args, self.log)
         if args.ls:
@@ -205,6 +215,8 @@ class SvcHub(object):
         if self.stopping:
             return
 
+        # start_log_thrs(print, 0.1, 1)
+
         self.stopping = True
         self.stop_req = True
         with self.stop_cond:

copyparty/tcpsrv.py

@@ -177,7 +177,7 @@ class TcpSrv(object):
             eps = self.ips_linux()
 
         if "0.0.0.0" not in listen_ips:
-            eps = {k: v for k, v in eps if k in listen_ips}
+            eps = {k: v for k, v in eps.items() if k in listen_ips}
 
         default_route = None
         s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

copyparty/th_srv.py

@@ -21,7 +21,7 @@ HAVE_AVIF = False
 HAVE_WEBP = False
 
 try:
-    from PIL import Image, ImageOps
+    from PIL import Image, ImageOps, ExifTags
 
     HAVE_PIL = True
     try:
@@ -105,7 +105,10 @@ class ThumbSrv(object):
         self.mutex = threading.Lock()
         self.busy = {}
         self.stopping = False
+        self.nthr = self.args.th_mt
+        if not self.nthr:
             self.nthr = os.cpu_count() if hasattr(os, "cpu_count") else 4
+
         self.q = Queue(self.nthr * 4)
         for n in range(self.nthr):
             t = threading.Thread(
@@ -221,21 +224,38 @@ class ThumbSrv(object):
         with self.mutex:
             self.nthr -= 1
 
-    def conv_pil(self, abspath, tpath):
+    def fancy_pillow(self, im):
-        with Image.open(fsenc(abspath)) as im:
+        # exif_transpose is expensive (loads full image + unconditional copy)
-            crop = not self.args.th_no_crop
+        r = max(*self.res) * 2
-            res2 = self.res
+        im.thumbnail((r, r), resample=Image.LANCZOS)
-            if crop:
-                res2 = (res2[0] * 2, res2[1] * 2)
-
         try:
-                im.thumbnail(res2, resample=Image.LANCZOS)
+            k = next(k for k, v in ExifTags.TAGS.items() if v == "Orientation")
-                if crop:
+            exif = im.getexif()
+            rot = int(exif[k])
+            del exif[k]
+        except:
+            rot = 1
+
+        rots = {8: Image.ROTATE_90, 3: Image.ROTATE_180, 6: Image.ROTATE_270}
+        if rot in rots:
+            im = im.transpose(rots[rot])
+
+        if self.args.th_no_crop:
+            im.thumbnail(self.res, resample=Image.LANCZOS)
+        else:
             iw, ih = im.size
             dw, dh = self.res
             res = (min(iw, dw), min(ih, dh))
             im = ImageOps.fit(im, res, method=Image.LANCZOS)
-            except:
+
+        return im
+
+    def conv_pil(self, abspath, tpath):
+        with Image.open(fsenc(abspath)) as im:
+            try:
+                im = self.fancy_pillow(im)
+            except Exception as ex:
+                self.log("fancy_pillow {}".format(ex), "1;30")
                 im.thumbnail(self.res)
 
             fmts = ["RGB", "L"]
@@ -250,13 +270,14 @@ class ThumbSrv(object):
                 fmts += ["RGBA", "LA"]
                 args["method"] = 6
             else:
-                pass  # default q = 75
+                # default q = 75
+                args["progressive"] = True
 
             if im.mode not in fmts:
                 # print("conv {}".format(im.mode))
                 im = im.convert("RGB")
 
-            im.save(tpath, quality=40, method=6)
+            im.save(tpath, **args)
 
     def conv_ffmpeg(self, abspath, tpath):
         ret, _ = ffprobe(abspath)
@@ -289,6 +310,7 @@ class ThumbSrv(object):
             b"-map", b"0:v:0",
             b"-vf", scale,
             b"-frames:v", b"1",
+            b"-metadata:s:v:0", b"rotate=0",
         ]
         # fmt: on
 
@@ -306,6 +328,7 @@ class ThumbSrv(object):
             ]
 
         cmd += [fsenc(tpath)]
+        # self.log((b" ".join(cmd)).decode("utf-8"))
 
         ret, sout, serr = runcmd(cmd)
         if ret != 0:

copyparty/u2idx.py

@@ -6,9 +6,10 @@ import os
 import time
 import threading
 from datetime import datetime
+from operator import itemgetter
 
-from .__init__ import unicode
+from .__init__ import ANYWIN, unicode
-from .util import s3dec, Pebkac, min_ex
+from .util import absreal, s3dec, Pebkac, min_ex, gen_filekey, quotep
 from .bos import bos
 from .up2k import up2k_wark_from_hashlist
 
@@ -242,6 +243,7 @@ class U2idx(object):
             self.active_cur = cur
 
             sret = []
+            fk = flags.get("fk")
             c = cur.execute(q, v)
             for hit in c:
                 w, ts, sz, rd, fn, ip, at = hit
@@ -252,7 +254,23 @@ class U2idx(object):
                 if rd.startswith("//") or fn.startswith("//"):
                     rd, fn = s3dec(rd, fn)
 
-                rp = "/".join([x for x in [vtop, rd, fn] if x])
+                if not fk:
+                    suf = ""
+                else:
+                    try:
+                        ap = absreal(os.path.join(ptop, rd, fn))
+                        inf = bos.stat(ap)
+                    except:
+                        continue
+
+                    suf = (
+                        "?k="
+                        + gen_filekey(
+                            self.args.fk_salt, ap, sz, 0 if ANYWIN else inf.st_ino
+                        )[:fk]
+                    )
+
+                rp = quotep("/".join([x for x in [vtop, rd, fn] if x])) + suf
                 sret.append({"ts": int(ts), "sz": sz, "rp": rp, "w": w[:16]})
 
             for hit in sret:
@@ -275,9 +293,13 @@ class U2idx(object):
         # undupe hits from multiple metadata keys
         if len(ret) > 1:
             ret = [ret[0]] + [
-                y for x, y in zip(ret[:-1], ret[1:]) if x["rp"] != y["rp"]
+                y
+                for x, y in zip(ret[:-1], ret[1:])
+                if x["rp"].split("?")[0] != y["rp"].split("?")[0]
             ]
 
+        ret.sort(key=itemgetter("rp"))
+
         return ret, list(taglist.keys())
 
     def terminator(self, identifier, done_flag):

copyparty/up2k.py

@@ -27,6 +27,7 @@ from .util import (
     sanitize_fn,
     ren_open,
     atomic_move,
+    quotep,
     vsplit,
     s3enc,
     s3dec,
@@ -66,6 +67,7 @@ class Up2k(object):
         self.n_tagq = 0
         self.volstate = {}
         self.need_rescan = {}
+        self.dupesched = {}
         self.registry = {}
         self.entags = {}
         self.flags = {}
@@ -236,7 +238,7 @@ class Up2k(object):
                         if vp:
                             fvp = "{}/{}".format(vp, fvp)
 
-                        self._handle_rm(LEELOO_DALLAS, None, fvp)
+                        self._handle_rm(LEELOO_DALLAS, None, fvp, True)
                         nrm += 1
 
                 if nrm:
@@ -838,6 +840,7 @@ class Up2k(object):
 
             cur.connection.commit()
             if n_done:
+                self.log("mtp: scanned {} files in {}".format(n_done, ptop), c=6)
                 cur.execute("vacuum")
 
             wcur.close()
@@ -939,7 +942,12 @@ class Up2k(object):
 
     def _tag_file(self, write_cur, entags, wark, abspath, tags=None):
         if tags is None:
+            try:
                 tags = self.mtag.get(abspath)
+            except Exception as ex:
+                msg = "failed to read tags from {}:\n{}"
+                self.log(msg.format(abspath, ex), c=3)
+                return
 
         if entags:
             tags = {k: v for k, v in tags.items() if k in entags}
@@ -1111,9 +1119,18 @@ class Up2k(object):
                     if dp_dir.startswith("//") or dp_fn.startswith("//"):
                         dp_dir, dp_fn = s3dec(dp_dir, dp_fn)
 
+                    if job and (dp_dir != cj["prel"] or dp_fn != cj["name"]):
+                        continue
+
                     dp_abs = "/".join([cj["ptop"], dp_dir, dp_fn])
-                    # relying on path.exists to return false on broken symlinks
+                    # relying on this to fail on broken symlinks
-                    if bos.path.exists(dp_abs):
+                    try:
+                        sz = bos.path.getsize(dp_abs)
+                    except:
+                        sz = 0
+
+                    if sz:
+                        # self.log("--- " + wark + "  " + dp_abs + " found file", 4)
                         job = {
                             "name": dp_fn,
                             "prel": dp_dir,
@@ -1126,9 +1143,9 @@ class Up2k(object):
                             "hash": [],
                             "need": [],
                         }
-                        break
 
                 if job and wark in reg:
+                    # self.log("pop " + wark + "  " + job["name"] + " handle_json db", 4)
                     del reg[wark]
 
             if job or wark in reg:
@@ -1156,11 +1173,20 @@ class Up2k(object):
                     if job["need"]:
                         self.log("unfinished:\n  {0}\n  {1}".format(src, dst))
                         err = "partial upload exists at a different location; please resume uploading here instead:\n"
-                        err += "/" + vsrc + " "
+                        err += "/" + quotep(vsrc) + " "
+
+                        dupe = [cj["prel"], cj["name"]]
+                        try:
+                            self.dupesched[src].append(dupe)
+                        except:
+                            self.dupesched[src] = [dupe]
+
                         raise Pebkac(400, err)
+
                     elif "nodupe" in self.flags[job["ptop"]]:
                         self.log("dupe-reject:\n  {0}\n  {1}".format(src, dst))
-                        err = "upload rejected, file already exists:\n/" + vsrc + " "
+                        err = "upload rejected, file already exists:\n"
+                        err += "/" + quotep(vsrc) + " "
                         raise Pebkac(400, err)
                     else:
                         # symlink to the client-provided name,
@@ -1223,7 +1249,8 @@ class Up2k(object):
 
                 self._new_upload(job)
 
-            purl = "/{}/".format("{}/{}".format(job["vtop"], job["prel"]).strip("/"))
+            purl = "{}/{}".format(job["vtop"], job["prel"]).strip("/")
+            purl = "/{}/".format(purl) if purl else "/"
 
             return {
                 "name": job["name"],
@@ -1252,6 +1279,9 @@ class Up2k(object):
             return
 
         try:
+            if self.args.no_symlink:
+                raise Exception("disabled in config")
+
             lsrc = src
             ldst = dst
             fs1 = bos.stat(os.path.dirname(src)).st_dev
@@ -1332,6 +1362,26 @@ class Up2k(object):
                 # del self.registry[ptop][wark]
                 return ret, dst
 
+            # windows cant rename open files
+            if not ANYWIN or src == dst:
+                self._finish_upload(ptop, wark)
+
+        return ret, dst
+
+    def finish_upload(self, ptop, wark):
+        with self.mutex:
+            self._finish_upload(ptop, wark)
+
+    def _finish_upload(self, ptop, wark):
+        try:
+            job = self.registry[ptop][wark]
+            pdir = os.path.join(job["ptop"], job["prel"])
+            src = os.path.join(pdir, job["tnam"])
+            dst = os.path.join(pdir, job["name"])
+        except Exception as ex:
+            return "finish_upload, wark, " + repr(ex)
+
+        # self.log("--- " + wark + "  " + dst + " finish_upload atomic " + dst, 4)
         atomic_move(src, dst)
 
         if ANYWIN:
@@ -1341,10 +1391,27 @@ class Up2k(object):
         a = [job[x] for x in "ptop wark prel name lmod size addr".split()]
         a += [job.get("at") or time.time()]
         if self.idx_wark(*a):
+            # self.log("pop " + wark + "  " + dst + " finish_upload idx_wark", 4)
             del self.registry[ptop][wark]
             # in-memory registry is reserved for unfinished uploads
 
-        return ret, dst
+        dupes = self.dupesched.pop(dst, [])
+        if not dupes:
+            return
+
+        cur = self.cur.get(ptop)
+        for rd, fn in dupes:
+            d2 = os.path.join(ptop, rd, fn)
+            if os.path.exists(d2):
+                continue
+
+            self._symlink(dst, d2)
+            if cur:
+                self.db_rm(cur, rd, fn)
+                self.db_add(cur, wark, rd, fn, *a[-4:])
+
+        if cur:
+            cur.connection.commit()
 
     def idx_wark(self, ptop, wark, rd, fn, lmod, sz, ip, at):
         cur = self.cur.get(ptop)
@@ -1383,7 +1450,7 @@ class Up2k(object):
         ok = {}
         ng = {}
         for vp in vpaths:
-            a, b, c = self._handle_rm(uname, ip, vp)
+            a, b, c = self._handle_rm(uname, ip, vp, False)
             n_files += a
             for k in b:
                 ok[k] = 1
@@ -1396,7 +1463,7 @@ class Up2k(object):
 
         return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
 
-    def _handle_rm(self, uname, ip, vpath):
+    def _handle_rm(self, uname, ip, vpath, rm_topdir):
         try:
             permsets = [[True, False, False, True]]
             vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
@@ -1404,7 +1471,7 @@ class Up2k(object):
         except:
             # unpost with missing permissions? try read+write and verify with db
             if not self.args.unpost:
-                raise Pebkac(400, "the unpost feature was disabled by server config")
+                raise Pebkac(400, "the unpost feature is disabled in server config")
 
             unpost = True
             permsets = [[True, True]]
@@ -1427,7 +1494,11 @@ class Up2k(object):
         ptop = vn.realpath
         atop = vn.canonical(rem, False)
         adir, fn = os.path.split(atop)
+        try:
             st = bos.lstat(atop)
+        except:
+            raise Pebkac(400, "file not found on disk (already deleted?)")
+
         scandir = not self.args.no_scandir
         if stat.S_ISLNK(st.st_mode) or stat.S_ISREG(st.st_mode):
             dbv, vrem = self.asrv.vfs.get(vpath, uname, *permsets[0])
@@ -1450,16 +1521,18 @@ class Up2k(object):
                 self.log("rm {}\n  {}".format(vpath, abspath))
                 _ = dbv.get(volpath, uname, *permsets[0])
                 with self.mutex:
+                    cur = None
                     try:
                         ptop = dbv.realpath
                         cur, wark, _, _, _, _ = self._find_from_vpath(ptop, volpath)
                         self._forget_file(ptop, volpath, cur, wark, True)
                     finally:
+                        if cur:
                             cur.connection.commit()
 
                 bos.unlink(abspath)
 
-        rm = rmdirs(self.log_func, scandir, True, atop)
+        rm = rmdirs(self.log_func, scandir, True, atop, 1 if rm_topdir else 0)
         return n_files, rm[0], rm[1]
 
     def handle_mv(self, uname, svp, dvp):
@@ -1501,7 +1574,7 @@ class Up2k(object):
                 with self.mutex:
                     self._mv_file(uname, svpf, dvpf)
 
-        rmdirs(self.log_func, scandir, True, sabs)
+        rmdirs(self.log_func, scandir, True, sabs, 1)
         return "k"
 
     def _mv_file(self, uname, svp, dvp):
@@ -1515,6 +1588,14 @@ class Up2k(object):
         dabs = dvn.canonical(drem)
         drd, dfn = vsplit(drem)
 
+        n1 = svp.split("/")[-1]
+        n2 = dvp.split("/")[-1]
+        if n1.startswith(".") or n2.startswith("."):
+            if self.args.no_dot_mv:
+                raise Pebkac(400, "moving dotfiles is disabled in server config")
+            elif self.args.no_dot_ren and n1 != n2:
+                raise Pebkac(400, "renaming dotfiles is disabled in server config")
+
         if bos.path.exists(dabs):
             raise Pebkac(400, "mv2: target file exists")
 
@@ -1570,7 +1651,7 @@ class Up2k(object):
     def _find_from_vpath(self, ptop, vrem):
         cur = self.cur.get(ptop)
         if not cur:
-            return None, None
+            return [None] * 6
 
         rd, fn = vsplit(vrem)
         q = "select w, mt, sz, ip, at from up where rd=? and fn=? limit 1"
@@ -1607,7 +1688,7 @@ class Up2k(object):
                 wark = [
                     x
                     for x, y in reg.items()
-                    if fn in [y["name"], y.get("tnam")] and y["prel"] == vrem
+                    if sfn in [y["name"], y.get("tnam")] and y["prel"] == vrem
                 ]
 
             if wark and wark in reg:
@@ -1690,7 +1771,13 @@ class Up2k(object):
         except:
             cj["lmod"] = int(time.time())
 
+        if cj["hash"]:
             wark = up2k_wark_from_hashlist(self.salt, cj["size"], cj["hash"])
+        else:
+            wark = up2k_wark_from_metadata(
+                self.salt, cj["size"], cj["lmod"], cj["prel"], cj["name"]
+            )
+
         return wark
 
     def _hashlist_from_file(self, path):
@@ -1733,6 +1820,8 @@ class Up2k(object):
 
         if self.args.nw:
             job["tnam"] = tnam
+            if not job["hash"]:
+                del self.registry[job["ptop"]][job["wark"]]
             return
 
         suffix = ".{:.6f}-{}".format(job["t0"], job["addr"])
@@ -1749,9 +1838,13 @@ class Up2k(object):
                 except:
                     self.log("could not sparse [{}]".format(fp), 3)
 
+            if job["hash"]:
                 f.seek(job["size"] - 1)
                 f.write(b"e")
 
+        if not job["hash"]:
+            self._finish_upload(job["ptop"], job["wark"])
+
     def _lastmodder(self):
         while True:
             ready = []
@@ -1848,11 +1941,16 @@ class Up2k(object):
 
             # self.log("\n  " + repr([ptop, rd, fn]))
             abspath = os.path.join(ptop, rd, fn)
+            try:
                 tags = self.mtag.get(abspath)
                 ntags1 = len(tags)
                 parsers = self._get_parsers(ptop, tags, abspath)
                 if parsers:
                     tags.update(self.mtag.get_bin(parsers, abspath))
+            except Exception as ex:
+                msg = "failed to read tags from {}:\n{}"
+                self.log(msg.format(abspath, ex), c=3)
+                continue
 
             with self.mutex:
                 cur = self.cur[ptop]

copyparty/util.py

@@ -19,7 +19,7 @@ import subprocess as sp  # nosec
 from datetime import datetime
 from collections import Counter
 
-from .__init__ import PY2, WINDOWS, ANYWIN
+from .__init__ import PY2, WINDOWS, ANYWIN, VT100, unicode
 from .stolen import surrogateescape
 
 FAKE_MP = False
@@ -58,7 +58,7 @@ except:
         return struct.unpack(f.decode("ascii"), *a, **ka)
 
 
-ansi_re = re.compile("\033\\[[^m]*m")
+ansi_re = re.compile("\033\\[[^mK]*[mK]")
 
 
 surrogateescape.register_surrogateescape()
@@ -169,7 +169,7 @@ class Cooldown(object):
             return ret
 
 
-class Unrecv(object):
+class _Unrecv(object):
     """
     undo any number of socket recv ops
     """
@@ -189,10 +189,117 @@ class Unrecv(object):
         except:
             return b""
 
+    def recv_ex(self, nbytes):
+        """read an exact number of bytes"""
+        ret = self.recv(nbytes)
+        while ret and len(ret) < nbytes:
+            buf = self.recv(nbytes - len(ret))
+            if not buf:
+                break
+
+            ret += buf
+
+        return ret
+
     def unrecv(self, buf):
         self.buf = buf + self.buf
 
 
+class _LUnrecv(object):
+    """
+    with expensive debug logging
+    """
+
+    def __init__(self, s):
+        self.s = s
+        self.buf = b""
+
+    def recv(self, nbytes):
+        if self.buf:
+            ret = self.buf[:nbytes]
+            self.buf = self.buf[nbytes:]
+            m = "\033[0;7mur:pop:\033[0;1;32m {}\n\033[0;7mur:rem:\033[0;1;35m {}\033[0m\n"
+            print(m.format(ret, self.buf), end="")
+            return ret
+
+        try:
+            ret = self.s.recv(nbytes)
+            m = "\033[0;7mur:recv\033[0;1;33m {}\033[0m\n"
+            print(m.format(ret), end="")
+            return ret
+        except:
+            return b""
+
+    def recv_ex(self, nbytes):
+        """read an exact number of bytes"""
+        ret = self.recv(nbytes)
+        while ret and len(ret) < nbytes:
+            buf = self.recv(nbytes - len(ret))
+            if not buf:
+                break
+
+            ret += buf
+
+        return ret
+
+    def unrecv(self, buf):
+        self.buf = buf + self.buf
+        m = "\033[0;7mur:push\033[0;1;31m {}\n\033[0;7mur:rem:\033[0;1;35m {}\033[0m\n"
+        print(m.format(buf, self.buf), end="")
+
+
+Unrecv = _Unrecv
+
+
+class FHC(object):
+    class CE(object):
+        def __init__(self, fh):
+            self.ts = 0
+            self.fhs = [fh]
+
+    def __init__(self):
+        self.cache = {}
+
+    def close(self, path):
+        try:
+            ce = self.cache[path]
+        except:
+            return
+
+        for fh in ce.fhs:
+            fh.close()
+
+        del self.cache[path]
+
+    def clean(self):
+        if not self.cache:
+            return
+
+        keep = {}
+        now = time.time()
+        for path, ce in self.cache.items():
+            if now < ce.ts + 5:
+                keep[path] = ce
+            else:
+                for fh in ce.fhs:
+                    fh.close()
+
+        self.cache = keep
+
+    def pop(self, path):
+        return self.cache[path].fhs.pop()
+
+    def put(self, path, fh):
+        try:
+            ce = self.cache[path]
+            ce.fhs.append(fh)
+        except:
+            ce = self.CE(fh)
+            self.cache[path] = ce
+
+        ce.ts = time.time()
+
+
 class ProgressPrinter(threading.Thread):
     """
     periodically print progress info without linefeeds
@@ -207,17 +314,22 @@ class ProgressPrinter(threading.Thread):
 
     def run(self):
         msg = None
+        fmt = " {}\033[K\r" if VT100 else " {} $\r"
         while not self.end:
             time.sleep(0.1)
             if msg == self.msg or self.end:
                 continue
 
             msg = self.msg
-            uprint(" {}\033[K\r".format(msg))
+            uprint(fmt.format(msg))
             if PY2:
                 sys.stdout.flush()
 
+        if VT100:
             print("\033[K", end="")
+        elif msg:
+            print("------------------------")
+
         sys.stdout.flush()  # necessary on win10 even w/ stderr btw
 
 
@@ -312,7 +424,7 @@ def stackmon(fp, ival, suffix):
 
 
 def start_log_thrs(logger, ival, nid):
-    ival = int(ival)
+    ival = float(ival)
     tname = lname = "log-thrs"
     if nid:
         tname = "logthr-n{}-i{:x}".format(nid, os.getpid())
@@ -346,8 +458,10 @@ def log_thrs(log, ival, name):
 
 def vol_san(vols, txt):
     for vol in vols:
+        txt = txt.replace(vol.realpath.encode("utf-8"), vol.vpath.encode("utf-8"))
         txt = txt.replace(
-            vol.realpath.encode("utf-8"), vol.vpath.encode("utf-8")
+            vol.realpath.encode("utf-8").replace(b"\\", b"\\\\"),
+            vol.vpath.encode("utf-8"),
         )
 
     return txt
@@ -447,8 +561,8 @@ class MultipartParser(object):
         self.log = log_func
         self.headers = http_headers
 
-        self.re_ctype = re.compile(r"^content-type: *([^;]+)", re.IGNORECASE)
+        self.re_ctype = re.compile(r"^content-type: *([^; ]+)", re.IGNORECASE)
-        self.re_cdisp = re.compile(r"^content-disposition: *([^;]+)", re.IGNORECASE)
+        self.re_cdisp = re.compile(r"^content-disposition: *([^; ]+)", re.IGNORECASE)
         self.re_cdisp_field = re.compile(
             r'^content-disposition:(?: *|.*; *)name="([^"]+)"', re.IGNORECASE
         )
@@ -584,19 +698,21 @@ class MultipartParser(object):
         yields [fieldname, unsanitized_filename, fieldvalue]
         where fieldvalue yields chunks of data
         """
-        while True:
+        run = True
+        while run:
             fieldname, filename = self._read_header()
             yield [fieldname, filename, self._read_data()]
 
-            tail = self.sr.recv(2)
+            tail = self.sr.recv_ex(2)
 
             if tail == b"--":
                 # EOF indicated by this immediately after final boundary
-                self.sr.recv(2)
+                tail = self.sr.recv_ex(2)
-                return
+                run = False
 
             if tail != b"\r\n":
-                raise Pebkac(400, "protocol error after field value")
+                m = "protocol error after field value: want b'\\r\\n', got {!r}"
+                raise Pebkac(400, m.format(tail))
 
     def _read_value(self, iterator, max_len):
         ret = b""
@@ -645,7 +761,7 @@ class MultipartParser(object):
 def get_boundary(headers):
     # boundaries contain a-z A-Z 0-9 ' ( ) + _ , - . / : = ?
     # (whitespace allowed except as the last char)
-    ptn = r"^multipart/form-data; *(.*; *)?boundary=([^;]+)"
+    ptn = r"^multipart/form-data *; *(.*; *)?boundary=([^;]+)"
     ct = headers["content-type"]
     m = re.match(ptn, ct, re.IGNORECASE)
     if not m:
@@ -682,6 +798,14 @@ def read_header(sr):
         return ret[:ofs].decode("utf-8", "surrogateescape").lstrip("\r\n").split("\r\n")
 
 
+def gen_filekey(salt, fspath, fsize, inode):
+    return base64.urlsafe_b64encode(
+        hashlib.sha512(
+            "{} {} {} {}".format(salt, fspath, fsize, inode).encode("utf-8", "replace")
+        ).digest()
+    ).decode("ascii")
+
+
 def humansize(sz, terse=False):
     for unit in ["B", "KiB", "MiB", "GiB", "TiB"]:
         if sz < 1024:
@@ -982,16 +1106,23 @@ def read_socket_chunked(sr, log=None):
             raise Pebkac(400, err)
 
         if chunklen == 0:
-            sr.recv(2)  # \r\n after final chunk
+            x = sr.recv_ex(2)
+            if x == b"\r\n":
                 return
 
+            m = "protocol error after final chunk: want b'\\r\\n', got {!r}"
+            raise Pebkac(400, m.format(x))
+
         if log:
             log("receiving {} byte chunk".format(chunklen))
 
         for chunk in read_socket(sr, chunklen):
             yield chunk
 
-        sr.recv(2)  # \r\n after each chunk too
+        x = sr.recv_ex(2)
+        if x != b"\r\n":
+            m = "protocol error in chunk separator: want b'\\r\\n', got {!r}"
+            raise Pebkac(400, m.format(x))
 
 
 def yieldfile(fn):
@@ -1086,7 +1217,7 @@ def statdir(logger, scandir, lstat, top):
         logger(src, "{} @ {}".format(repr(ex), top), 1)
 
 
-def rmdirs(logger, scandir, lstat, top):
+def rmdirs(logger, scandir, lstat, top, depth):
     if not os.path.exists(fsenc(top)) or not os.path.isdir(fsenc(top)):
         top = os.path.dirname(top)
 
@@ -1096,10 +1227,11 @@ def rmdirs(logger, scandir, lstat, top):
     ok = []
     ng = []
     for d in dirs[::-1]:
-        a, b = rmdirs(logger, scandir, lstat, d)
+        a, b = rmdirs(logger, scandir, lstat, d, depth + 1)
         ok += a
         ng += b
 
+    if depth:
         try:
             os.rmdir(fsenc(top))
             ok.append(top)

copyparty/web/baguettebox.js

@@ -22,7 +22,7 @@ window.baguetteBox = (function () {
             afterHide: null,
             onChange: null,
         },
-        overlay, slider, btnPrev, btnNext, btnHelp, btnSel, btnVmode, btnClose,
+        overlay, slider, btnPrev, btnNext, btnHelp, btnRotL, btnRotR, btnSel, btnVmode, btnClose,
         currentGallery = [],
         currentIndex = 0,
         isOverlayVisible = false,
@@ -49,7 +49,7 @@ window.baguetteBox = (function () {
     };
 
     var touchstartHandler = function (e) {
-        touch.count++;
+        touch.count = e.touches.length;
         if (touch.count > 1)
             touch.multitouch = true;
 
@@ -72,8 +72,11 @@ window.baguetteBox = (function () {
             hideOverlay();
         }
     };
-    var touchendHandler = function () {
+    var touchendHandler = function (e) {
         touch.count--;
+        if (e && e.touches)
+            touch.count = e.touches.length;
+
         if (touch.count <= 0)
             touch.multitouch = false;
 
@@ -175,6 +178,8 @@ window.baguetteBox = (function () {
                 '<button id="bbox-next" class="bbox-btn" type="button" aria-label="Next">&gt;</button>' +
                 '<div id="bbox-btns">' +
                 '<button id="bbox-help" type="button">?</button>' +
+                '<button id="bbox-rotl" type="button">↶</button>' +
+                '<button id="bbox-rotr" type="button">↷</button>' +
                 '<button id="bbox-tsel" type="button">sel</button>' +
                 '<button id="bbox-vmode" type="button" tt="a"></button>' +
                 '<button id="bbox-close" type="button" aria-label="Close">X</button>' +
@@ -188,6 +193,8 @@ window.baguetteBox = (function () {
         btnPrev = ebi('bbox-prev');
         btnNext = ebi('bbox-next');
         btnHelp = ebi('bbox-help');
+        btnRotL = ebi('bbox-rotl');
+        btnRotR = ebi('bbox-rotr');
         btnSel = ebi('bbox-tsel');
         btnVmode = ebi('bbox-vmode');
         btnClose = ebi('bbox-close');
@@ -205,12 +212,13 @@ window.baguetteBox = (function () {
             ['right, L', 'next file'],
             ['home', 'first file'],
             ['end', 'last file'],
+            ['R', 'rotate (shift=ccw)'],
             ['S', 'toggle file selection'],
             ['space, P, K', 'video: play / pause'],
             ['U', 'video: seek 10sec back'],
             ['P', 'video: seek 10sec ahead'],
             ['M', 'video: toggle mute'],
-            ['R', 'video: toggle loop'],
+            ['V', 'video: toggle loop'],
             ['C', 'video: toggle auto-next'],
             ['F', 'video: toggle fullscreen'],
         ],
@@ -252,7 +260,7 @@ window.baguetteBox = (function () {
             v.muted = vmute = !vmute;
             mp_ctl();
         }
-        else if (k == "KeyR" && v) {
+        else if (k == "KeyV" && v) {
             vloop = !vloop;
             vnext = vnext && !vloop;
             setVmode();
@@ -272,6 +280,8 @@ window.baguetteBox = (function () {
             catch (ex) { }
         else if (k == "KeyS")
             tglsel();
+        else if (k == "KeyR")
+            rotn(e.shiftKey ? -1 : 1);
     }
 
     function setVmode() {
@@ -284,7 +294,7 @@ window.baguetteBox = (function () {
         if (vloop) {
             lbl = 'Loop';
             msg += 'repeat it';
-            tts = '$NHotkey: R';
+            tts = '$NHotkey: V';
         }
         else if (vnext) {
             lbl = 'Cont';
@@ -333,7 +343,7 @@ window.baguetteBox = (function () {
     }
 
     function selbg() {
-        var img = imagesElements[currentIndex].querySelector('img, video'),
+        var img = vidimg(),
             thumb = currentGallery[currentIndex].imageElement,
             name = vsplit(thumb.href)[1],
             files = msel.getsel(),
@@ -346,11 +356,11 @@ window.baguetteBox = (function () {
         ebi('bbox-overlay').style.background = sel ?
             'rgba(153,34,85,0.7)' : '';
 
-        img.style.boxShadow = sel ? '0 0 3em #f4a' : '';
         img.style.borderRadius = sel ? '1em' : '';
         btnSel.style.color = sel ? '#fff' : '';
         btnSel.style.background = sel ? '#d48' : '';
         btnSel.style.textShadow = sel ? '1px 1px 0 #b38' : '';
+        btnSel.style.boxShadow = sel ? '.15em .15em 0 #502' : '';
     }
 
     function keyUpHandler(e) {
@@ -387,6 +397,8 @@ window.baguetteBox = (function () {
         bind(btnClose, 'click', hideOverlay);
         bind(btnVmode, 'click', tglVmode);
         bind(btnHelp, 'click', halp);
+        bind(btnRotL, 'click', rotl);
+        bind(btnRotR, 'click', rotr);
         bind(btnSel, 'click', tglsel);
         bind(slider, 'contextmenu', contextmenuHandler);
         bind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
@@ -402,12 +414,15 @@ window.baguetteBox = (function () {
         unbind(btnClose, 'click', hideOverlay);
         unbind(btnVmode, 'click', tglVmode);
         unbind(btnHelp, 'click', halp);
+        unbind(btnRotL, 'click', rotl);
+        unbind(btnRotR, 'click', rotr);
         unbind(btnSel, 'click', tglsel);
         unbind(slider, 'contextmenu', contextmenuHandler);
         unbind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
         unbind(overlay, 'touchmove', touchmoveHandler, passiveEvent);
         unbind(overlay, 'touchend', touchendHandler);
         unbind(document, 'focus', trapFocusInsideOverlay, true);
+        timer.rm(rotn);
     }
 
     function prepareOverlay(gallery, userOptions) {
@@ -658,10 +673,91 @@ window.baguetteBox = (function () {
         return true;
     }
 
+    var prev_cw = 0, prev_ch = 0, unrot_timer = null;
+    function rotn(n) {
+        var el = vidimg(),
+            orot = parseInt(el.getAttribute('rot') || 0),
+            frot = orot + (n || 0) * 90;
+
+        if (!frot && !orot)
+            return;  // reflow noop
+
+        var co = ebi('bbox-overlay'),
+            cw = co.clientWidth,
+            ch = co.clientHeight;
+
+        if (!n && prev_cw === cw && prev_ch === ch)
+            return;  // reflow noop
+
+        prev_cw = cw;
+        prev_ch = ch;
+        var rot = frot,
+            iw = el.naturalWidth || el.videoWidth,
+            ih = el.naturalHeight || el.videoHeight,
+            magic = 4,  // idk, works in enough browsers
+            dl = el.closest('div').querySelector('figcaption a'),
+            vw = cw,
+            vh = ch - dl.offsetHeight + magic,
+            pmag = Math.min(1, Math.min(vw / ih, vh / iw)),
+            wmag = Math.min(1, Math.min(vw / iw, vh / ih));
+
+        while (rot < 0) rot += 360;
+        while (rot >= 360) rot -= 360;
+        var q = rot == 90 || rot == 270 ? 1 : 0,
+            mag = q ? pmag : wmag;
+
+        el.style.cssText = 'max-width:none; max-height:none; position:absolute; display:block; margin:0';
+        if (!orot) {
+            el.style.width = iw * wmag + 'px';
+            el.style.height = ih * wmag + 'px';
+            el.style.left = (vw - iw * wmag) / 2 + 'px';
+            el.style.top = (vh - ih * wmag) / 2 - magic + 'px';
+            q = el.offsetHeight;
+        }
+        el.style.width = iw * mag + 'px';
+        el.style.height = ih * mag + 'px';
+        el.style.left = (vw - iw * mag) / 2 + 'px';
+        el.style.top = (vh - ih * mag) / 2 - magic + 'px';
+        el.style.transform = 'rotate(' + frot + 'deg)';
+        el.setAttribute('rot', frot);
+        timer.add(rotn);
+        if (!rot) {
+            clearTimeout(unrot_timer);
+            unrot_timer = setTimeout(unrot, 300);
+        }
+    }
+    function rotl() {
+        rotn(-1);
+    }
+    function rotr() {
+        rotn(1);
+    }
+    function unrot() {
+        var el = vidimg(),
+            orot = el.getAttribute('rot'),
+            rot = parseInt(orot || 0);
+
+        while (rot < 0) rot += 360;
+        while (rot >= 360) rot -= 360;
+        if (rot || orot === null)
+            return;
+
+        clmod(el, 'nt', 1);
+        el.removeAttribute('rot');
+        el.removeAttribute("style");
+        rot = el.offsetHeight;
+        clmod(el, 'nt');
+        timer.rm(rotn);
+    }
+
     function vid() {
         return imagesElements[currentIndex].querySelector('video');
     }
 
+    function vidimg() {
+        return imagesElements[currentIndex].querySelector('img, video');
+    }
+
     function playvid(play) {
         if (vid())
             vid()[play ? 'play' : 'pause']();
@@ -703,15 +799,21 @@ window.baguetteBox = (function () {
     }
 
     function updateOffset() {
-        var offset = -currentIndex * 100 + '%';
+        var offset = -currentIndex * 100 + '%',
+            xform = slider.style.perspective !== undefined;
+
         if (options.animation === 'fadeIn') {
             slider.style.opacity = 0;
             setTimeout(function () {
-                slider.style.transform = 'translate3d(' + offset + ',0,0)';
+                xform ?
+                    slider.style.transform = 'translate3d(' + offset + ',0,0)' :
+                    slider.style.left = offset;
                 slider.style.opacity = 1;
             }, 400);
         } else {
-            slider.style.transform = 'translate3d(' + offset + ',0,0)';
+            xform ?
+                slider.style.transform = 'translate3d(' + offset + ',0,0)' :
+                slider.style.left = offset;
         }
         playvid(false);
         var v = vid();
@@ -723,6 +825,18 @@ window.baguetteBox = (function () {
         selbg();
         mp_ctl();
         setVmode();
+
+        var el = vidimg();
+        if (el.getAttribute('rot'))
+            timer.add(rotn);
+        else
+            timer.rm(rotn);
+
+        var prev = QS('.full-image.vis');
+        if (prev)
+            clmod(prev, 'vis');
+
+        clmod(el.closest('div'), 'vis', 1);
     }
 
     function preloadNext(index) {

copyparty/web/browser.css

@@ -1,5 +1,7 @@
 :root {
 	--grid-sz: 10em;
+	--grid-ln: 3;
+	--nav-sz: 16em;
 }
 * {
 	line-height: 1.2em;
@@ -23,7 +25,7 @@ html, body {
 	padding: 0;
 }
 pre, code, tt {
-	font-family: monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 }
 #path,
 #path * {
@@ -113,7 +115,7 @@ a, #files tbody div a:last-child {
 	border-top: 1px solid #383838;
 }
 #files tbody td:nth-child(3) {
-	font-family: monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 	text-align: right;
 	padding-right: 1em;
 	white-space: nowrap;
@@ -163,6 +165,7 @@ a, #files tbody div a:last-child {
 .logue {
 	padding: .2em 1.5em;
 }
+.logue.hidden,
 .logue:empty {
 	display: none;
 }
@@ -172,6 +175,13 @@ a, #files tbody div a:last-child {
 #epi.logue {
 	margin: .8em 0;
 }
+.mdo {
+	max-width: 52em;
+}
+.mdo,
+.mdo * {
+	line-height: 1.4em;
+}
 #srv_info {
 	color: #a73;
 	background: #333;
@@ -199,6 +209,9 @@ a, #files tbody div a:last-child {
 	color: #f4c;
 	border-bottom: 1px solid rgba(255,68,204,0.6);
 }
+#repl {
+	padding: .33em;
+}
 #files tbody a.play {
 	color: #e70;
 	padding: .2em;
@@ -208,8 +221,8 @@ a, #files tbody div a:last-child {
 	color: #720;
 	text-shadow: 0 0 .3em #b80;
 }
-#ggrid a.play,
+#ggrid>a.play,
-html.light #ggrid a.play {
+html.light #ggrid>a.play {
 	color: #fff;
 	background: #750;
 	border-color: #c90;
@@ -217,36 +230,38 @@ html.light #ggrid a.play {
 	box-shadow: 0 .1em 1.2em #b83;
 }
 #files tbody tr.sel td,
-#ggrid a.sel,
+#ggrid>a.sel,
-html.light #ggrid a.sel {
+#ggrid>a[tt].sel,
+html.light #ggrid>a.sel,
+html.light #ggrid>a[tt].sel {
 	color: #fff;
 	background: #925;
 	border-color: #c37;
 }
 #files tbody tr.sel:hover td,
 #files tbody tr.sel:focus td,
-#ggrid a.sel:hover,
+#ggrid>a.sel:hover,
-html.light #ggrid a.sel:hover {
+html.light #ggrid>a.sel:hover {
 	color: #fff;
 	background: #d39;
 	border-color: #d48;
 	text-shadow: 1px 1px 0 #804;
 }
-#ggrid a.sel,
+#ggrid>a.sel,
-html.light #ggrid a.sel {
+html.light #ggrid>a.sel {
 	border-top: 1px solid #d48;
 	box-shadow: 0 .1em 1.2em #b36;
 	transition: all 0.2s cubic-bezier(.2, 2.2, .5, 1); /* https://cubic-bezier.com/#.4,2,.7,1 */
 }
-#ggrid a.sel img,
+#ggrid>a.sel img,
-#ggrid a.play img {
+#ggrid>a.play img {
 	opacity: .7;
 	filter: contrast(130%) brightness(107%);
 }
-#ggrid a.sel img {
+#ggrid>a.sel img {
 	box-shadow: 0 0 1em #b36;
 }
-#ggrid a.play img {
+#ggrid>a.play img {
 	box-shadow: 0 0 1em #b83;
 }
 #files tr.sel a {
@@ -310,7 +325,7 @@ html.light #ggrid a.sel {
 }
 #widget {
 	position: fixed;
-	font-size: 1.5em;
+	font-size: 1.4em;
 	left: 0;
 	right: 0;
 	bottom: -6em;
@@ -333,16 +348,19 @@ html.light #ggrid a.sel {
 }
 #wtgrid,
 #wtico {
-	cursor: url(/.cpr/dd/4.png), pointer;
-	animation: cursor 500ms;
 	position: relative;
-	top: -.06em;
+	font-size: .9em;
+	top: -.04em;
 }
 #wtgrid {
-	font-size: .8em;
+	font-size: .75em;
+	padding: .1em;
 	top: -.12em;
 }
-#wtgrid:hover,
+#wtico {
+	cursor: url(/.cpr/dd/4.png), pointer;
+	animation: cursor 500ms;
+}
 #wtico:hover {
 	animation: cursor 500ms infinite;
 }
@@ -359,7 +377,8 @@ html.light #ggrid a.sel {
 #wtoggle {
 	position: absolute;
 	white-space: nowrap;
-	top: -1.2em;
+	font-size: .8em;
+	top: -1em;
 	right: 0;
 	height: 1em;
 	font-size: 2em;
@@ -369,23 +388,26 @@ html.light #ggrid a.sel {
 	background: #3c3c3c;
 	box-shadow: 0 0 .5em #222;
 	border-radius: .3em 0 0 0;
-	padding: .2em .2em;
+	padding: 0 0 0 .1em;
 	color: #fff;
 }
-#wzip, #wnp {
+#wfm, #wzip, #wnp {
 	display: none;
-	margin-right: .3em;
-	padding-right: .3em;
-	border-right: .1em solid #555;
 }
-#wnp a {
+#wzip, #wnp {
-	position: relative;
+	margin-right: .2em;
-	font-size: .47em;
+	padding-right: .2em;
-	margin: 0 .1em;
+	border: 1px solid #555;
-	top: -.4em;
+	border-width: 0 .1em 0 0;
 }
-#wnp a+a {
+#wfm.act+#wzip,
-	margin-left: .33em;
+#wfm.act+#wzip+#wnp {
+	margin-left: .2em;
+	padding-left: .2em;
+	border-left-width: .1em;
+}
+#wfm.act {
+	display: inline-block;
 }
 #wtoggle,
 #wtoggle * {
@@ -399,17 +421,22 @@ html.light #ggrid a.sel {
 	display: none;
 }
 #wfm a,
+#wnp a,
 #wzip a {
 	font-size: .5em;
 	padding: 0 .3em;
-	margin: -.3em .2em;
+	margin: -.3em .1em;
 	position: relative;
 	display: inline-block;
 }
-#wfm span {
+#wfm span,
+#wnp span {
 	font-size: .6em;
 	display: block;
 }
+#wnp span {
+	font-size: .7em;
+}
 #wfm a:not(.en) {
 	opacity: .3;
 	color: #f6c;
@@ -417,23 +444,16 @@ html.light #ggrid a.sel {
 html.light #wfm a:not(.en) {
 	color: #c4a;
 }
-#files tbody tr.c1 td {
+#files tbody tr.fcut td {
-	animation: fcut1 .5s ease-out;
+	animation: fcut .5s ease-out;
 }
-#files tbody tr.c2 td {
+@keyframes fcut {
-	animation: fcut2 .5s ease-out;
-}
-@keyframes fcut1 {
-	0% {opacity:0}
-	100% {opacity:1}
-}
-@keyframes fcut2 {
 	0% {opacity:0}
 	100% {opacity:1}
 }
 #wzip a {
 	font-size: .4em;
-	margin: -.3em .3em;
+	margin: -.3em .1em;
 }
 #wtoggle.sel #wzip #selzip {
 	top: -.6em;
@@ -533,24 +553,9 @@ html.light #wfm a:not(.en) {
 	box-shadow: 0 -.15em .2em #000 inset;
 	padding-bottom: .3em;
 }
-#ops i {
-	font-size: 1.5em;
-}
-#ops i:before {
-	content: 'x';
-	color: #282828;
-	text-shadow: 0 0 .08em #01a7e1;
-	position: relative;
-}
-#ops i:after {
-	content: 'x';
-	color: #282828;
-	text-shadow: 0 0 .08em #ff3f1a;
-	margin-left: -.35em;
-	font-size: 1.05em;
-}
 #ops,
-.opbox {
+.opbox,
+#u2etas {
 	border: 1px solid #3a3a3a;
 	box-shadow: 0 0 1em #222 inset;
 }
@@ -568,7 +573,7 @@ html.light #wfm a:not(.en) {
 	padding: .5em;
 	border-radius: 0 1em 1em 0;
 	border-width: .15em .3em .3em 0;
-	max-width: 40em;
+	max-width: 41em;
 }
 .opbox input {
 	margin: .5em;
@@ -699,6 +704,7 @@ input.eq_gain {
 	left: 0;
 	bottom: 0;
 	top: 7em;
+	overflow-x: hidden;
 	overflow-y: auto;
 	-ms-scroll-chaining: none;
 	overscroll-behavior-y: none;
@@ -709,6 +715,10 @@ input.eq_gain {
 	position: sticky;
 	z-index: 1;
 	top: 0;
+	height: 2.2em;
+	line-height: 2.2em;
+	border-bottom: 1px solid #555;
+	overflow: hidden;
 }
 #thx_ff {
 	padding: 5em 0;
@@ -726,8 +736,8 @@ input.eq_gain {
 }
 #treeul {
 	position: relative;
-	left: -1.7em;
+	left: -2em;
-	width: calc(100% + 1.3em);
+	width: calc(100% + 2em);
 }
 .btn {
 	padding: .2em .4em;
@@ -748,6 +758,9 @@ input.eq_gain {
 	color: #400;
 	text-shadow: none;
 }
+.tgl.btn.on:hover {
+	background: #fe8;
+}
 #detree {
 	padding: .3em .5em;
 	font-size: 1.5em;
@@ -783,12 +796,27 @@ input.eq_gain {
 	width: calc(100% - 2em);
 	line-height: 1em;
 }
+#tree.nowrap #treeul li {
+	min-height: 1.4em;
+	white-space: nowrap;
+}
+#tree.nowrap #treeul a+a:hover {
+	background: rgba(34, 34, 34, 0.67);
+	min-width: calc(var(--nav-sz) - 2em);
+	width: auto;
+}
+html.light #tree.nowrap #treeul a+a:hover {
+	background: rgba(255, 255, 255, 0.67);
+	color: #000;
+}
 #treeul a+a:hover {
 	background: #222;
 	color: #fff;
 }
 #treeul a:first-child {
-	font-family: monospace, monospace;
+	font-family: 'scp', monospace, monospace;
+	font-size: 1.2em;
+	line-height: 0;
 }
 .dumb_loader_thing {
 	display: inline-block;
@@ -809,8 +837,6 @@ input.eq_gain {
 	width: 1em;
 	border-radius: .2em;
 	margin: -1.2em auto 0 auto;
-	top: 2em;
-	position: relative;
 	background: #444;
 }
 #files th span {
@@ -874,14 +900,19 @@ input.eq_gain {
 #u2turbo.on+#u2tdate {
 	opacity: 1;
 }
+#wraptree.on+#hovertree {
+	display: none;
+}
 #ghead {
 	background: #3c3c3c;
 	border: 1px solid #444;
 	border-radius: .3em;
-	padding: .5em;
+	padding: .2em .5em;
+	line-height: 2.3em;
 	margin: 0 1.5em 1em .4em;
 	position: sticky;
 	top: -.3em;
+	z-index: 1;
 }
 html.light #ghead {
 	background: #f7f7f7;
@@ -891,11 +922,24 @@ html.light #ghead {
 	position: relative;
 	top: 0;
 }
+#ghead>span {
+	white-space: pre;
+	padding-left: .3em;
+}
 #ggrid {
 	padding-top: .5em;
 }
-#ggrid a {
+#ggrid>a>span {
+	overflow: hidden;
+	display: block;
+	display: -webkit-box;
+	-webkit-line-clamp: var(--grid-ln);
+	-webkit-box-orient: vertical;
+	padding-top: .3em;
+}
+#ggrid>a {
 	display: inline-block;
+	width: 10em;
 	width: var(--grid-sz);
 	vertical-align: top;
 	overflow-wrap: break-word;
@@ -907,34 +951,45 @@ html.light #ghead {
 	padding: .3em;
 	margin: .5em;
 }
-#ggrid a img {
+#ggrid>a[tt] {
+	background: linear-gradient(135deg, #383838 95%, #555 95%);
+}
+#ggrid>a img {
 	border-radius: .2em;
+	max-width: 10em;
 	max-width: var(--grid-sz);
+	max-height: 8em;
 	max-height: calc(var(--grid-sz)/1.25);
 	margin: 0 auto;
 	display: block;
 }
-#ggrid a span {
+#ggrid>a.dir:before {
-	padding: .2em .3em;
-	display: block;
-}
-#ggrid span.dir:before {
 	content: '📂';
-	line-height: 0;
-	font-size: 2em;
-	margin: -.7em .1em -.5em -.3em;
 }
-#ggrid a:hover {
+#ggrid>a:before {
+	display: block;
+	position: absolute;
+	padding: .3em 0;
+	margin: -.4em;
+	text-shadow: 0 0 .1em #000;
+	background: linear-gradient(135deg,rgba(255,255,255,0) 50%,rgba(255,255,255,0.2));
+	border-radius: .3em;
+	font-size: 2em;
+}
+#ggrid>a:hover {
 	background: #444;
 	border-color: #555;
 	color: #fd9;
 }
-html.light #ggrid a {
+html.light #ggrid>a {
 	background: #f7f7f7;
 	border-color: #ddd;
 	box-shadow: 0 .1em .2em #ddd;
 }
-html.light #ggrid a:hover {
+html.light #ggrid>a[tt] {
+	background: linear-gradient(135deg, #f7f7f7 95%, #ccc 95%);
+}
+html.light #ggrid>a:hover {
 	background: #fff;
 	border-color: #ccc;
 	color: #015;
@@ -978,7 +1033,7 @@ html.light #rui {
 	padding: .2em 0 .2em .5em;
 }
 #rn_vadv input {
-	font-family: monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 }
 #rui td+td,
 #rui td input[type="text"] {
@@ -1011,8 +1066,12 @@ html.light #rui {
 #pvol,
 #barbuf,
 #barpos,
+a.btn,
+#u2btn,
 #u2conf label,
 #rui label,
+#modal-ok,
+#modal-ng,
 #ops {
 	-webkit-user-select: none;
 	-moz-user-select: none;
@@ -1051,7 +1110,8 @@ html.light {
 }
 html.light #ops,
 html.light .opbox,
-html.light #srch_form {
+html.light #srch_form,
+html.light #u2etas {
 	background: #f7f7f7;
 	box-shadow: 0 0 .3em #ddd;
 	border-color: #f7f7f7;
@@ -1077,6 +1137,9 @@ html.light .tgl.btn.on {
 	background: #4a0;
 	color: #fff;
 }
+html.light .tgl.btn.on:hover {
+	background: #5c0;
+}
 html.light #srv_info {
 	color: #c83;
 	background: #eee;
@@ -1102,7 +1165,7 @@ html.light #treeul a.hl:hover {
 html.light #tree li {
 	border-color: #f7f7f7 #fff #ddd #fff;
 }
-html.light #tree a:hover {
+html.light #treeul a:hover {
 	background: #fff;
 }
 html.light #tree ul {
@@ -1251,6 +1314,7 @@ html.light #files tr.sel a:hover {
 }
 html.light #treeh {
 	background: #eee;
+	border-color: #ddd;
 }
 html.light #tree {
 	scrollbar-color: #a70 #ddd;
@@ -1294,7 +1358,7 @@ html.light #tree::-webkit-scrollbar {
 	opacity: 0;
 	position: fixed;
 	overflow: hidden;
-	touch-action: none;
+	touch-action: pinch-zoom;
 	top: 0;
 	left: 0;
 	width: 100%;
@@ -1328,6 +1392,14 @@ html.light #tree::-webkit-scrollbar {
 	max-height: calc(100% - 1.4em);
 	margin-bottom: 1.4em;
 	vertical-align: middle;
+	transition: transform .23s, left .23s, top .23s, width .23s, height .23s;
+}
+.full-image img.nt,
+.full-image video.nt {
+	transition: none;
+}
+.full-image.vis img,
+.full-image.vis video {
 	box-shadow: 0 0 8px rgba(0, 0, 0, 0.6);
 }
 .full-image video {
@@ -1341,6 +1413,7 @@ html.light #tree::-webkit-scrollbar {
 	text-align: center;
 	white-space: normal;
 	color: #ccc;
+	z-index: 1;
 }
 #bbox-overlay figcaption a {
 	background: rgba(0, 0, 0, 0.6);
@@ -1497,6 +1570,98 @@ html.light #bbox-overlay figcaption a {
 
 #op_up2k {
 	padding: 0 1em 1em 1em;
+	min-height: 0;
+	transition: min-height .2s;
+}
+#drops {
+	display: none;
+	z-index: 3;
+	background: rgba(48, 48, 48, 0.7);
+}
+#drops.vis,
+#drops .dropzone {
+	display: block;
+	position: fixed;
+	top: 0;
+	left: 0;
+	right: 0;
+	bottom: 0;
+}
+#drops .dropdesc {
+	position: fixed;
+	display: table;
+	left: 10%;
+	width: 78%;
+	height: 30%;
+	margin: 0;
+	font-size: 3em;
+	font-weight: bold;
+	text-shadow: .05em .05em .1em #000;
+	background: rgba(224, 224, 224, 0.2);
+	box-shadow: 0 0 0 #999;
+	border: .5em solid #999;
+	border-radius: .5em;
+	border-width: 1vw;
+	color: #fff;
+	transition: all 0.12s;
+}
+#drops .dropdesc.hl.ok {
+	border-color: #fff;
+	box-shadow: 0 0 1em #cf5;
+	background: rgba(24, 24, 24, 0.7);
+	left: 8%;
+	width: 82%;
+	height: 32%;
+	margin: -1vh 0;
+}
+#drops .dropdesc.hl.err {
+	background: rgba(224, 32, 65, 0.2);
+	box-shadow: 0 0 1em #f26;
+	border-color: #fab;
+}
+#drops .dropdesc>div {
+	display: table-cell;
+	vertical-align: middle;
+	text-align: center;
+}
+#drops .dropdesc>div>div {
+	position: absolute;
+	top: 40%;
+	top: calc(50% - .5em);
+	left: -.8em;
+}
+#drops .dropdesc>div>div+div {
+	left: auto;
+	right: -.8em;
+}
+#drops .dropzone {
+	z-index: 80386;
+	height: 50%;
+}
+#drops #up_dz {
+	bottom: 50%;
+}
+#drops #srch_dz {
+	top: 50%;
+}
+#up_zd {
+	top: 12%;
+}
+#srch_zd {
+	bottom: 12%;
+}
+.dropdesc span {
+	color: #fff;
+	background: #490;
+	border-bottom: .3em solid #6d2;
+	text-shadow: 1px 1px 1px #000;
+	border-radius: .3em;
+	padding: .4em .8em;
+	font-size: .4em;
+}
+.dropdesc.err span {
+	background: #904;
+	border-color: #d26;
 }
 #u2form {
 	position: absolute;
@@ -1520,6 +1685,9 @@ html.light #bbox-overlay figcaption a {
 	padding: .5em;
 	font-size: .9em;
 }
+html.light #u2err.err {
+	color: #f07;
+}
 #u2btn {
 	color: #eee;
 	background: #555;
@@ -1545,7 +1713,7 @@ html.light #bbox-overlay figcaption a {
 	color: #333;
 }
 #u2conf #u2btn {
-	margin: -1.5em 0;
+	margin: -2.4em 0;
 	padding: .8em 0;
 	width: 100%;
 	max-width: 12em;
@@ -1564,6 +1732,7 @@ html.light #bbox-overlay figcaption a {
 	line-height: 1.3em;
 }
 #u2tab {
+	border-collapse: collapse;
 	margin: 3em auto;
 	width: calc(100% - 2em);
 	max-width: 100em;
@@ -1574,7 +1743,7 @@ html.light #bbox-overlay figcaption a {
 #u2tab td {
 	border: 1px solid #ccc;
 	border-width: 0 0px 1px 0;
-	padding: .1em .3em;
+	padding: .2em .3em;
 }
 #u2tab td:nth-child(2) {
 	width: 5em;
@@ -1591,17 +1760,51 @@ html.light #bbox-overlay figcaption a {
 #u2tab tbody tr:hover td {
 	background: #222;
 }
+#u2etas {
+	background: #333;
+	padding: .2em .5em;
+	border-radius: .5em;
+	border-width: .25em 0;
+	width: 17em;
+	text-align: center;
+	white-space: nowrap;
+	display: inline-block;
+	font-family: 'scp', monospace, monospace;
+}
+#u2etas.o {
+	width: 20em;
+}
+#u2etas .o {
+	display: none;
+}
+#u2etas.o .o {
+	display: unset;
+}
+#u2etaw {
+	width: 18em;
+	font-size: .94em;
+	margin: 1.8em auto .5em auto;
+}
+#u2etas.o #u2etaw {
+	width: 21em;
+}
 #u2cards {
-	padding: 1em 0 .3em 1em;
+	padding: 1em 1em .3em 1em;
-	margin: 1.5em auto -2.5em auto;
+	margin: 0 auto -1.5em auto;
 	white-space: nowrap;
 	text-align: center;
 	overflow: hidden;
+	min-width: 24em;
 }
 #u2cards.w {
-	width: 45em;
+	width: 44em;
 	text-align: left;
 }
+#u2etaw.w {
+	width: 52em;
+	text-align: right;
+	margin: 3em auto -2.7em auto;
+}
 #u2cards a {
 	padding: .2em 1em;
 	border: 1px solid #777;
@@ -1708,6 +1911,14 @@ html.light #bbox-overlay figcaption a {
 	box-shadow: none;
 	opacity: .2;
 }
+#u2flagblock {
+	display: none;
+	margin: 2em auto 4em auto;
+	padding: .5em;
+	max-width: 27em;
+	text-align: center;
+	border: .2em dashed rgba(128, 128, 128, 0.3);
+}
 #u2foot {
 	color: #fff;
 	font-style: italic;
@@ -1742,9 +1953,30 @@ html.light #u2foot .warn span {
 	margin-bottom: -1em;
 }
 .prog {
-	font-family: monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 }
-#u2tab a>span {
+#u2tab span.inf,
+#u2tab span.ok,
+#u2tab span.err {
+	color: #fff;
+	padding: .22em;
+	border-radius: .2em;
+	border: 2px solid #f0f;
+}
+#u2tab span.inf {
+	background: #07a;
+	border-color: #0be;
+}
+#u2tab span.ok {
+	background: #380;
+	border-color: #8e4;
+}
+#u2tab span.err {
+	background: #900;
+	border-color: #d06;
+}
+#u2tab a>span,
+#unpost a>span {
 	font-weight: bold;
 	font-style: italic;
 	color: #fff;

copyparty/web/browser.html

@@ -114,6 +114,8 @@
 
 	<h2><a href="/?h">control-panel</a></h2>
 	
+	<a href="#" id="repl">π</a>
+
 </div>
 
 	{%- if srv_info %}
@@ -131,7 +133,8 @@
 			have_mv = {{ have_mv|tojson }},
 			have_del = {{ have_del|tojson }},
 			have_unpost = {{ have_unpost|tojson }},
-			have_zip = {{ have_zip|tojson }};
+			have_zip = {{ have_zip|tojson }},
+			readme = {{ readme|tojson }};
 	</script>
 	<script src="/.cpr/util.js?_={{ ts }}"></script>
 	<script src="/.cpr/browser.js?_={{ ts }}"></script>

copyparty/web/md.css

@@ -1,13 +1,17 @@
-@font-face {
-	font-family: 'scp';
-	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
-}
 html, body {
 	color: #333;
 	background: #eee;
 	font-family: sans-serif;
 	line-height: 1.5em;
 }
+#repl {
+	position: absolute;
+	top: 0;
+	right: .5em;
+	border: none;
+	color: inherit;
+	background: none;
+}
 #mtw {
 	display: none;
 }
@@ -19,122 +23,8 @@ html, body {
 	bottom: auto;
 	top: 1.4em;
 }
-pre, code, a {
+a {
-	color: #480;
+	text-decoration: none;
-	background: #f7f7f7;
-	border: .07em solid #ddd;
-	border-radius: .2em;
-	padding: .1em .3em;
-	margin: 0 .1em;
-}
-code {
-	font-size: .96em;
-}
-pre, code, tt {
-	font-family: 'scp', monospace, monospace;
-	white-space: pre-wrap;
-	word-break: break-all;
-}
-pre {
-	counter-reset: precode;
-}
-pre code {
-	counter-increment: precode;
-	display: inline-block;
-	margin: 0 -.3em;
-	padding: .4em .5em;
-	border: none;
-	border-bottom: 1px solid #cdc;
-	min-width: calc(100% - .6em);
-	line-height: 1.1em;
-}
-pre code:last-child {
-	border-bottom: none;
-}
-pre code::before {
-	content: counter(precode);
-	-webkit-user-select: none;
-	-moz-user-select: none;
-	-ms-user-select: none;
-	user-select: none;
-	display: inline-block;
-	text-align: right;
-	font-size: .75em;
-	color: #48a;
-	width: 4em;
-	padding-right: 1.5em;
-	margin-left: -5.5em;
-}
-pre code:hover {
-	background: #fec;
-	color: #360;
-}
-h1, h2 {
-	line-height: 1.5em;
-}
-h1 {
-	font-size: 1.7em;
-	text-align: center;
-	border: 1em solid #777;
-	border-width: .05em 0;
-	margin: 3em 0;
-}
-h2 {
-	font-size: 1.5em;
-	font-weight: normal;
-	background: #f7f7f7;
-	border-top: .07em solid #fff;
-	border-bottom: .07em solid #bbb;
-	border-radius: .5em .5em 0 0;
-	padding-left: .4em;
-	margin-top: 3em;
-}
-h3 {
-	border-bottom: .1em solid #999;
-}
-h1 a, h3 a, h5 a,
-h2 a, h4 a, h6 a {
-	color: inherit;
-	display: block;
-	background: none;
-	border: none;
-	padding: 0;
-	margin: 0;
-}
-#mp ul,
-#mp ol {
-	border-left: .3em solid #ddd;
-}
-#m>ul,
-#m>ol {
-	border-color: #bbb;
-}
-#mp ul>li {
-	list-style-type: disc;
-}
-#mp ul>li,
-#mp ol>li {
-	margin: .7em 0;
-}
-strong {
-	color: #000;
-}
-p>em,
-li>em,
-td>em {
-	color: #c50;
-	padding: .1em;
-	border-bottom: .1em solid #bbb;
-}
-blockquote {
-	font-family: serif;
-	background: #f7f7f7;
-	border: .07em dashed #ccc;
-	padding: 0 2em;
-	margin: 1em 0;
-}
-small {
-	opacity: .8;
 }
 #toc {
 	margin: 0 1em;
@@ -182,14 +72,6 @@ small {
 	color: #6b3;
 	text-shadow: .02em 0 0 #6b3;
 }
-table {
-	border-collapse: collapse;
-	margin: 1em 0;
-}
-th, td {
-	padding: .2em .5em;
-	border: .12em solid #aaa;
-}
 blink {
 	animation: blinker .7s cubic-bezier(.9, 0, .1, 1) infinite;
 }
@@ -202,6 +84,36 @@ blink {
 	}
 }
 
+
+.mdo pre {
+	counter-reset: precode;
+}
+.mdo pre code {
+	counter-increment: precode;
+	display: inline-block;
+	border: none;
+	border-bottom: 1px solid #cdc;
+	min-width: calc(100% - .6em);
+}
+.mdo pre code:last-child {
+	border-bottom: none;
+}
+.mdo pre code::before {
+	content: counter(precode);
+	-webkit-user-select: none;
+	-moz-user-select: none;
+	-ms-user-select: none;
+	user-select: none;
+	display: inline-block;
+	text-align: right;
+	font-size: .75em;
+	color: #48a;
+	width: 4em;
+	padding-right: 1.5em;
+	margin-left: -5.5em;
+}
+
+
 @media screen {
 	html, body {
 		margin: 0;
@@ -218,34 +130,6 @@ blink {
 	#mp {
 		max-width: 52em;
 		margin-bottom: 6em;
-		word-break: break-word;
-		overflow-wrap: break-word;
-		word-wrap: break-word; /*ie*/
-	}
-	a {
-		color: #fff;
-		background: #39b;
-		text-decoration: none;
-		padding: 0 .3em;
-		border: none;
-		border-bottom: .07em solid #079;
-	}
-	h2 {
-		color: #fff;
-		background: #555;
-		margin-top: 2em;
-		border-bottom: .22em solid #999;
-		border-top: none;
-	}
-	h1 {
-		color: #fff;
-		background: #444;
-		font-weight: normal;
-		border-top: .4em solid #fb0;
-		border-bottom: .4em solid #777;
-		border-radius: 0 1em 0 1em;
-		margin: 3em 0 1em 0;
-		padding: .5em 0;
 	}
 	#mn {
 		padding: 1.3em 0 .7em 1em;
@@ -298,6 +182,8 @@ blink {
 		color: #444;
 		background: none;
 		text-decoration: underline;
+		margin: 0 .1em;
+		padding: 0 .3em;
 		border: none;
 	}
 	#mh a:hover {
@@ -326,6 +212,10 @@ blink {
 	#toolsbox a+a {
 		text-decoration: none;
 	}
+	#lno {
+		position: absolute;
+		right: 0;
+	}
 
 
 
@@ -346,55 +236,6 @@ blink {
 	html.dark #toc li {
 		border-width: 0;
 	}
-	html.dark #mp a {
-		background: #057;
-	}
-	html.dark #mp h1 a, html.dark #mp h4 a,
-	html.dark #mp h2 a, html.dark #mp h5 a,
-	html.dark #mp h3 a, html.dark #mp h6 a {
-		color: inherit;
-		background: none;
-	}
-	html.dark pre,
-	html.dark code {
-		color: #8c0;
-		background: #1a1a1a;
-		border: .07em solid #333;
-	}
-	html.dark #mp ul,
-	html.dark #mp ol {
-		border-color: #444;
-	}
-	html.dark #m>ul,
-	html.dark #m>ol {
-		border-color: #555;
-	}
-	html.dark strong {
-		color: #fff;
-	}
-	html.dark p>em,
-	html.dark li>em,
-	html.dark td>em {
-		color: #f94;
-		border-color: #666;
-	}
-	html.dark h1 {
-		background: #383838;
-		border-top: .4em solid #b80;
-		border-bottom: .4em solid #4c4c4c;
-	}
-	html.dark h2 {
-		background: #444;
-		border-bottom: .22em solid #555;
-	}
-	html.dark td,
-	html.dark th {
-		border-color: #444;
-	}
-	html.dark blockquote {
-		background: #282828;
-		border: .07em dashed #444;
-	}
 	html.dark #mn a:not(:last-child)::after {
 		border-color: rgba(255,255,255,0.3);
 	}
@@ -500,12 +341,15 @@ blink {
 		mso-footer-margin: .6in;
 		mso-paper-source: 0;
 	}
-	a {
+	.mdo a {
 		color: #079;
 		text-decoration: none;
 		border-bottom: .07em solid #4ac;
 		padding: 0 .3em;
 	}
+	#repl {
+		display: none;
+	}
 	#toc>ul {
 		border-left: .1em solid #84c4dd;
 	}
@@ -530,18 +374,20 @@ blink {
 	a[ctr]::before {
 		content: attr(ctr) '. ';
 	}
-	h1 {
+	.mdo h1 {
 		margin: 2em 0;
 	}
-	h2 {
+	.mdo h2 {
 		margin: 2em 0 0 0;
 	}
-	h1, h2, h3 {
+	.mdo h1,
+	.mdo h2,
+	.mdo h3 {
 		page-break-inside: avoid;
 	}
-	h1::after,
+	.mdo h1::after,
-	h2::after,
+	.mdo h2::after,
-	h3::after {
+	.mdo h3::after {
 		content: 'orz';
 		color: transparent;
 		display: block;
@@ -549,20 +395,20 @@ blink {
 		padding: 4em 0 0 0;
 		margin: 0 0 -5em 0;
 	}
-	p {
+	.mdo p {
 		page-break-inside: avoid;
 	}
-	table {
+	.mdo table {
 		page-break-inside: auto;
 	}
-	tr {
+	.mdo tr {
 		page-break-inside: avoid;
 		page-break-after: auto;
 	}
-	thead {
+	.mdo thead {
 		display: table-header-group;
 	}
-	tfoot {
+	.mdo tfoot {
 		display: table-footer-group;
 	}
 	#mp a.vis::after {
@@ -570,31 +416,32 @@ blink {
 		border-bottom: 1px solid #bbb;
 		color: #444;
 	}
-	blockquote {
+	.mdo blockquote {
 		border-color: #555;
 	}
-	code {
+	.mdo code {
 		border-color: #bbb;
 	}
-	pre, pre code {
+	.mdo pre,
+	.mdo pre code {
 		border-color: #999;
 	}
-	pre code::before {
+	.mdo pre code::before {
 		color: #058;
 	}
 
 
 	
-	html.dark a {
+	html.dark .mdo a {
 		color: #000;
 	}
-	html.dark pre,
+	html.dark .mdo pre,
-	html.dark code {
+	html.dark .mdo code {
 		color: #240;
 	}
-	html.dark p>em,
+	html.dark .mdo p>em,
-	html.dark li>em,
+	html.dark .mdo li>em,
-	html.dark td>em {
+	html.dark .mdo td>em {
 		color: #940;
 	}
 }

copyparty/web/md.html

@@ -15,7 +15,7 @@
 		<a id="lightswitch" href="#">go dark</a>
 		<a id="navtoggle" href="#">hide nav</a>
 		{%- if edit %}
-			<a id="save" href="?edit" tt="Hotkey: ctrl-s">save</a>
+			<a id="save" href="{{ arg_base }}edit" tt="Hotkey: ctrl-s">save</a>
 			<a id="sbs" href="#" tt="editor and preview side by side">sbs</a>
 			<a id="nsbs" href="#" tt="switch between editor and preview$NHotkey: ctrl-e">editor</a>
 			<div id="toolsbox">
@@ -26,10 +26,11 @@
 				<a id="cfg_uni" href="#">non-ascii: whitelist</a>
 				<a id="help" href="#">help</a>
 			</div>
+			<span id="lno">L#</span>
 		{%- else %}
-			<a href="?edit" tt="good: higher performance$Ngood: same document width as viewer$Nbad: assumes you know markdown">edit (basic)</a>
+			<a href="{{ arg_base }}edit" tt="good: higher performance$Ngood: same document width as viewer$Nbad: assumes you know markdown">edit (basic)</a>
-			<a href="?edit2" tt="not in-house so probably less buggy">edit (fancy)</a>
+			<a href="{{ arg_base }}edit2" tt="not in-house so probably less buggy">edit (fancy)</a>
-			<a href="?raw">view raw</a>
+			<a href="{{ arg_base }}raw">view raw</a>
 		{%- endif %}
 	</div>
 	<div id="toc"></div>
@@ -43,8 +44,9 @@
 				if you're still reading this, check that javascript is allowed
 			</div>
 		</div>
-		<div id="mp"></div>
+		<div id="mp" class="mdo"></div>
 	</div>
+	<a href="#" id="repl">π</a>
 	
 	{%- if edit %}
 	<div id="helpbox">

copyparty/web/md.js

@@ -24,23 +24,6 @@ var dbg = function () { };
 var md_plug = {};
 
 
-function hesc(txt) {
-    return txt.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
-}
-
-
-function cls(dom, name, add) {
-    var re = new RegExp('(^| )' + name + '( |$)');
-    var lst = (dom.getAttribute('class') + '').replace(re, "$1$2").replace(/  /, "");
-    dom.setAttribute('class', lst + (add ? ' ' + name : ''));
-}
-
-
-function statify(obj) {
-    return JSON.parse(JSON.stringify(obj));
-}
-
-
 // dodge browser issues
 (function () {
     var ua = navigator.userAgent;
@@ -65,7 +48,7 @@ function statify(obj) {
         if (a > 0)
             loc.push(n[a]);
 
-        var dec = hesc(uricom_dec(n[a])[0]);
+        var dec = esc(uricom_dec(n[a])[0]);
 
         nav.push('<a href="/' + loc.join('/') + '">' + dec + '</a>');
     }
@@ -73,6 +56,26 @@ function statify(obj) {
 })();
 
 
+// image load handler
+var img_load = (function () {
+    var r = {};
+    r.callbacks = [];
+
+    function fire() {
+        for (var a = 0; a < r.callbacks.length; a++)
+            r.callbacks[a]();
+    }
+
+    var timeout = null;
+    r.done = function () {
+        clearTimeout(timeout);
+        timeout = setTimeout(fire, 500);
+    };
+
+    return r;
+})();
+
+
 // faster than replacing the entire html (chrome 1.8x, firefox 1.6x)
 function copydom(src, dst, lv) {
     var sc = src.childNodes,
@@ -185,7 +188,7 @@ function md_plug_err(ex, js) {
     errbox.style.cssText = 'position:absolute;top:0;left:0;padding:1em .5em;background:#2b2b2b;color:#fc5'
     errbox.textContent = msg;
     errbox.onclick = function () {
-        modal.alert('<pre>' + ex.stack + '</pre>');
+        modal.alert('<pre>' + esc(ex.stack) + '</pre>');
     };
     if (o) {
         errbox.appendChild(o);
@@ -264,7 +267,14 @@ function convert_markdown(md_text, dest_dom) {
 
         throw ex;
     }
-    var md_dom = new DOMParser().parseFromString(md_html, "text/html").body;
+    var md_dom = dest_dom;
+    try {
+        md_dom = new DOMParser().parseFromString(md_html, "text/html").body;
+    }
+    catch (ex) {
+        md_dom.innerHTML = md_html;
+        window.copydom = noop;
+    }
 
     var nodes = md_dom.getElementsByTagName('a');
     for (var a = nodes.length - 1; a >= 0; a--) {
@@ -356,6 +366,10 @@ function convert_markdown(md_text, dest_dom) {
 
     copydom(md_dom, dest_dom, 0);
 
+    var imgs = dest_dom.getElementsByTagName('img');
+    for (var a = 0, aa = imgs.length; a < aa; a++)
+        imgs[a].onload = img_load.done;
+
     if (ext && ext[0].render2)
         try {
             ext[0].render2(dest_dom);
@@ -490,13 +504,16 @@ function init_toc() {
 // "main" :p
 convert_markdown(dom_src.value, dom_pre);
 var toc = init_toc();
+img_load.callbacks = [toc.refresh];
 
 
 // scroll handler
 var redraw = (function () {
-    var sbs = false;
+    var sbs = true;
     function onresize() {
+        if (window.matchMedia)
             sbs = window.matchMedia('(min-width: 64em)').matches;
+
         var y = (dom_hbar.offsetTop + dom_hbar.offsetHeight) + 'px';
         if (sbs) {
             dom_toc.style.top = y;

copyparty/web/md2.css

@@ -50,7 +50,7 @@
 	outline: none;
 	padding: 0;
 	margin: 0;
-	font-family: 'consolas', monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 	white-space: pre-wrap;
 	word-break: break-word;
 	overflow-wrap: break-word;

copyparty/web/md2.js

@@ -98,7 +98,7 @@ var draw_md = (function () {
         var src = dom_src.value;
         convert_markdown(src, dom_pre);
 
-        var lines = hesc(src).replace(/\r/g, "").split('\n');
+        var lines = esc(src).replace(/\r/g, "").split('\n');
         nlines = lines.length;
         var html = [];
         for (var a = 0; a < lines.length; a++)
@@ -108,7 +108,7 @@ var draw_md = (function () {
         map_src = genmap(dom_ref, map_src);
         map_pre = genmap(dom_pre, map_pre);
 
-        cls(ebi('save'), 'disabled', src == server_md);
+        clmod(ebi('save'), 'disabled', src == server_md);
 
         var t1 = Date.now();
         delay = t1 - t0 > 100 ? 25 : 1;
@@ -127,6 +127,12 @@ var draw_md = (function () {
 })();
 
 
+// discard TOC callback, just regen editor scroll map
+img_load.callbacks = [function () {
+    map_pre = genmap(dom_pre, map_pre);
+}];
+
+
 // resize handler
 redraw = (function () {
     function onresize() {
@@ -136,7 +142,6 @@ redraw = (function () {
         dom_ref.style.width = getComputedStyle(dom_src).offsetWidth + 'px';
         map_src = genmap(dom_ref, map_src);
         map_pre = genmap(dom_pre, map_pre);
-        dbg(document.body.clientWidth + 'x' + document.body.clientHeight);
     }
     function setsbs() {
         dom_wrap.setAttribute('class', '');
@@ -225,44 +230,40 @@ redraw = (function () {
 
 // modification checker
 function Modpoll() {
-    this.skip_one = true;
+    var r = {
-    this.disabled = false;
+        skip_one: true,
-
+        disabled: false
-    this.periodic = function () {
+    };
-        var that = this;
-        setTimeout(function () {
-            that.periodic();
-        }, 1000 * md_opt.modpoll_freq);
 
+    r.periodic = function () {
         var skip = null;
 
         if (toast.visible)
             skip = 'toast';
 
-        else if (this.skip_one)
+        else if (r.skip_one)
             skip = 'saved';
 
-        else if (this.disabled)
+        else if (r.disabled)
             skip = 'disabled';
 
         if (skip) {
             console.log('modpoll skip, ' + skip);
-            this.skip_one = false;
+            r.skip_one = false;
             return;
         }
 
         console.log('modpoll...');
         var url = (document.location + '').split('?')[0] + '?raw&_=' + Date.now();
         var xhr = new XMLHttpRequest();
-        xhr.modpoll = this;
         xhr.open('GET', url, true);
         xhr.responseType = 'text';
-        xhr.onreadystatechange = this.cb;
+        xhr.onreadystatechange = r.cb;
         xhr.send();
-    }
+    };
 
-    this.cb = function () {
+    r.cb = function () {
-        if (this.modpoll.disabled || this.modpoll.skip_one) {
+        if (r.disabled || r.skip_one) {
             console.log('modpoll abort');
             return;
         }
@@ -283,7 +284,7 @@ function Modpoll() {
 
         if (server_ref != server_now) {
             console.log("modpoll diff |" + server_ref.length + "|, |" + server_now.length + "|");
-            this.modpoll.disabled = true;
+            r.disabled = true;
             var msg = [
                 "The document has changed on the server.",
                 "The changes will NOT be loaded into your editor automatically.",
@@ -297,12 +298,12 @@ function Modpoll() {
         }
 
         console.log('modpoll eq');
-    }
+    };
 
     if (md_opt.modpoll_freq > 0)
-        this.periodic();
+        setInterval(r.periodic, 1000 * md_opt.modpoll_freq);
 
-    return this;
+    return r;
 }
 var modpoll = new Modpoll();
 
@@ -370,8 +371,8 @@ function save_cb() {
     }
 
     if (!r.ok) {
-        if (!this.btn.classList.contains('force-save')) {
+        if (!clgot(this.btn, 'force-save')) {
-            this.btn.classList.add('force-save');
+            clmod(this.btn, 'force-save', 1);
             var msg = [
                 'This file has been modified since you started editing it!\n',
                 'if you really want to overwrite, press save again.\n',
@@ -387,7 +388,7 @@ function save_cb() {
             return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
     }
 
-    this.btn.classList.remove('force-save');
+    clmod(this.btn, 'force-save');
     //alert('save OK -- wrote ' + r.size + ' bytes.\n\nsha512: ' + r.sha512);
 
     run_savechk(r.lastmod, this.txt, this.btn, 0);
@@ -874,6 +875,40 @@ function cfg_uni(e) {
 }
 
 
+var set_lno = (function () {
+    var t = null,
+        pi = null,
+        pv = null,
+        lno = ebi('lno');
+
+    function poke() {
+        clearTimeout(t);
+        t = setTimeout(fire, 20);
+    }
+
+    function fire() {
+        try {
+            clearTimeout(t);
+
+            var i = dom_src.selectionStart;
+            if (i === pi)
+                return;
+
+            var v = 'L' + dom_src.value.slice(0, i).split('\n').length;
+            if (v != pv)
+                lno.innerHTML = v;
+
+            pi = i;
+            pv = v;
+        }
+        catch (e) { }
+    }
+
+    timer.add(fire);
+    return poke;
+})();
+
+
 // hotkeys / toolbar
 (function () {
     function keydown(ev) {
@@ -892,6 +927,8 @@ function cfg_uni(e) {
         if (document.activeElement != dom_src)
             return true;
 
+        set_lno();
+
         if (ctrl(ev)) {
             if (ev.code == "KeyH" || kc == 72) {
                 md_header(ev.shiftKey);
@@ -1086,9 +1123,9 @@ action_stack = (function () {
         ref = newtxt;
         dbg('undos(%d) redos(%d)', hist.un.length, hist.re.length);
         if (hist.un.length > 0)
-            dbg(statify(hist.un.slice(-1)[0]));
+            dbg(jcp(hist.un.slice(-1)[0]));
         if (hist.re.length > 0)
-            dbg(statify(hist.re.slice(-1)[0]));
+            dbg(jcp(hist.re.slice(-1)[0]));
     }
 
     return {

copyparty/web/mde.css

@@ -7,6 +7,8 @@ html .editor-toolbar>button.active { border-color: rgba(0,0,0,0.4); background:
 html .editor-toolbar>i.separator { border-left: 1px solid #ccc; }
 html .editor-toolbar.disabled-for-preview>button:not(.no-disable) { opacity: .35 }
 
+
+
 html {
 	line-height: 1.5em;
 }
@@ -22,6 +24,18 @@ html, body {
 	bottom: auto;
 	top: 1.4em;
 }
+#repl {
+	position: absolute;
+	top: 0;
+	right: .5em;
+	border: none;
+	color: inherit;
+	background: none;
+	text-decoration: none;
+}
+
+
+
 #mn {
 	font-weight: normal;
 	margin: 1.3em 0 .7em 1em;
@@ -63,148 +77,12 @@ html .editor-toolbar>button.disabled {
 html .editor-toolbar>button.save.force-save {
 	background: #f97;
 }
-
-
-
-
-
-/* copied from md.css for now */
-.mdo pre,
-.mdo code,
-.mdo a {
-	color: #480;
-	background: #f7f7f7;
-	border: .07em solid #ddd;
-	border-radius: .2em;
-	padding: .1em .3em;
-	margin: 0 .1em;
-}
-.mdo code {
-	font-size: .96em;
-}
-.mdo pre,
-.mdo code {
-	font-family: monospace, monospace;
-	white-space: pre-wrap;
-	word-break: break-all;
-}
-.mdo pre code {
-	display: block;
-	margin: 0 -.3em;
-	padding: .4em .5em;
-	line-height: 1.1em;
-}
-.mdo a {
-	color: #fff;
-	background: #39b;
-	text-decoration: none;
-	padding: 0 .3em;
-	border: none;
-	border-bottom: .07em solid #079;
-}
-.mdo h2 {
-	color: #fff;
-	background: #555;
-	margin-top: 2em;
-	border-bottom: .22em solid #999;
-	border-top: none;
-}
-.mdo h1 {
-	color: #fff;
-	background: #444;
-	font-weight: normal;
-	border-top: .4em solid #fb0;
-	border-bottom: .4em solid #777;
-	border-radius: 0 1em 0 1em;
-	margin: 3em 0 1em 0;
-	padding: .5em 0;
-}
-h1, h2 {
-	line-height: 1.5em;
-}
-h1 {
-	font-size: 1.7em;
-	text-align: center;
-	border: 1em solid #777;
-	border-width: .05em 0;
-	margin: 3em 0;
-}
-h2 {
-	font-size: 1.5em;
-	font-weight: normal;
-	background: #f7f7f7;
-	border-top: .07em solid #fff;
-	border-bottom: .07em solid #bbb;
-	border-radius: .5em .5em 0 0;
-	padding-left: .4em;
-	margin-top: 3em;
-}
-.mdo ul,
-.mdo ol {
-	border-left: .3em solid #ddd;
-}
-.mdo>ul,
-.mdo>ol {
-	border-color: #bbb;
-}
-.mdo ul>li {
-	list-style-type: disc;
-}
-.mdo ul>li,
-.mdo ol>li {
-	margin: .7em 0;
-}
-strong {
-	color: #000;
-}
-p>em,
-li>em,
-td>em {
-	color: #c50;
-	padding: .1em;
-	border-bottom: .1em solid #bbb;
-}
-blockquote {
-	font-family: serif;
-	background: #f7f7f7;
-	border: .07em dashed #ccc;
-	padding: 0 2em;
-	margin: 1em 0;
-}
-small {
-	opacity: .8;
-}
-table {
-	border-collapse: collapse;
-}
-td {
-	padding: .2em .5em;
-	border: .12em solid #aaa;
-}
-th {
-	border: .12em solid #aaa;
-}
-
-
-
-
-
-/* mde support */
-.mdo {
-	padding: 1em;
-	background: #f7f7f7;
-}
-html.dark .mdo {
-	background: #1c1c1c;
-}
 .CodeMirror {
 	background: #f7f7f7;
 }
 
 
 
-
-
 /* darkmode */
 html.dark .mdo,
 html.dark .CodeMirror {
@@ -228,55 +106,6 @@ html.dark .CodeMirror-selectedtext {
 	background: #246;
 	color: #fff;
 }
-html.dark .mdo a {
-	background: #057;
-}
-html.dark .mdo h1 a, html.dark .mdo h4 a,
-html.dark .mdo h2 a, html.dark .mdo h5 a,
-html.dark .mdo h3 a, html.dark .mdo h6 a {
-	color: inherit;
-	background: none;
-}
-html.dark pre,
-html.dark code {
-	color: #8c0;
-	background: #1a1a1a;
-	border: .07em solid #333;
-}
-html.dark .mdo ul,
-html.dark .mdo ol {
-	border-color: #444;
-}
-html.dark .mdo>ul,
-html.dark .mdo>ol {
-	border-color: #555;
-}
-html.dark strong {
-	color: #fff;
-}
-html.dark p>em,
-html.dark li>em,
-html.dark td>em {
-	color: #f94;
-	border-color: #666;
-}
-html.dark h1 {
-	background: #383838;
-	border-top: .4em solid #b80;
-	border-bottom: .4em solid #4c4c4c;
-}
-html.dark h2 {
-	background: #444;
-	border-bottom: .22em solid #555;
-}
-html.dark td,
-html.dark th {
-	border-color: #444;
-}
-html.dark blockquote {
-	background: #282828;
-	border: .07em dashed #444;
-}
 
 
 
@@ -313,3 +142,14 @@ html.dark .editor-toolbar::after,
 html.dark .editor-toolbar::before {
 	background: none;
 }
+
+
+
+/* ui.css overrides */
+.mdo {
+	padding: 1em;
+	background: #f7f7f7;
+}
+html.dark .mdo {
+	background: #1c1c1c;
+}

copyparty/web/mde.html

@@ -21,6 +21,7 @@
 			<textarea id="mt" style="display:none" autocomplete="off">{{ md }}</textarea>
 		</div>
 	</div>
+	<a href="#" id="repl">π</a>
 	<script>
 
 var last_modified = {{ lastmod }};
@@ -44,6 +45,7 @@ l.setItem('lightmode', drk? 0:1);
 
 	</script>
     <script src="/.cpr/util.js?_={{ ts }}"></script>
+	<script src="/.cpr/deps/marked.js?_={{ ts }}"></script>
 	<script src="/.cpr/deps/easymde.js?_={{ ts }}"></script>
 	<script src="/.cpr/mde.js?_={{ ts }}"></script>
 </body></html>

copyparty/web/mde.js

@@ -96,20 +96,16 @@ function md_changed(mde, on_srv) {
     var md_now = mde.value();
     var save_btn = QS('.editor-toolbar button.save');
 
-    if (md_now == window.md_saved)
+    clmod(save_btn, 'disabled', md_now == window.md_saved);
-        save_btn.classList.add('disabled');
-    else
-        save_btn.classList.remove('disabled');
-
     set_jumpto();
 }
 
 function save(mde) {
     var save_btn = QS('.editor-toolbar button.save');
-    if (save_btn.classList.contains('disabled'))
+    if (clgot(save_btn, 'disabled'))
         return toast.inf(2, 'no changes');
 
-    var force = save_btn.classList.contains('force-save');
+    var force = clgot(save_btn, 'force-save');
     function save2() {
         var txt = mde.value();
 
@@ -153,8 +149,8 @@ function save_cb() {
     }
 
     if (!r.ok) {
-        if (!this.btn.classList.contains('force-save')) {
+        if (!clgot(this.btn, 'force-save')) {
-            this.btn.classList.add('force-save');
+            clmod(this.btn, 'force-save', 1);
             var msg = [
                 'This file has been modified since you started editing it!\n',
                 'if you really want to overwrite, press save again.\n',
@@ -170,7 +166,7 @@ function save_cb() {
             return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
     }
 
-    this.btn.classList.remove('force-save');
+    clmod(this.btn, 'force-save');
     //alert('save OK -- wrote ' + r.size + ' bytes.\n\nsha512: ' + r.sha512);
 
     // download the saved doc from the server and compare

copyparty/web/msg.css

@@ -11,6 +11,7 @@ html {
 	background: #333;
 	font-family: sans-serif;
 	text-shadow: 1px 1px 0px #000;
+	touch-action: manipulation;
 }
 html, body {
 	margin: 0;

copyparty/web/splash.css

@@ -3,6 +3,9 @@ html, body, #wrap {
 	background: #f7f7f7;
 	font-family: sans-serif;
 }
+html {
+	touch-action: manipulation;
+}
 #wrap {
 	max-width: 40em;
 	margin: 2em auto;
@@ -26,6 +29,12 @@ a {
 	border-radius: .2em;
 	padding: .2em .8em;
 }
+#repl {
+	border: none;
+	background: none;
+	color: inherit;
+	padding: 0;
+}
 table {
 	border-collapse: collapse;
 }
@@ -46,6 +55,16 @@ table {
 .btns {
 	margin: 1em 0;
 }
+#msg {
+	margin: 3em 0;
+}
+#msg h1 {
+	margin-bottom: 0;
+}
+#msg h1 + p {
+	margin-top: .3em;
+	text-align: right;
+}
 
 
 html.dark,
@@ -64,8 +83,8 @@ html.dark a {
 }
 html.dark input {
 	color: #fff;
-	background: #624;
+	background: #626;
-	border: 1px solid #c27;
+	border: 1px solid #c2c;
 	border-width: 1px 0 0 0;
 	border-radius: .5em;
 	padding: .5em .7em;

copyparty/web/splash.html

@@ -7,11 +7,22 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
     <link rel="stylesheet" media="screen" href="/.cpr/splash.css?_={{ ts }}">
+    <link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
 </head>
 
 <body>
     <div id="wrap">
-        <p>hello {{ this.uname }}</p>
+        {%- if this.uname == '*' %}
+            <p>howdy stranger &nbsp; <small>(you're not logged in)</small></p>
+        {%- else %}
+            <p>welcome back, <strong>{{ this.uname }}</strong></p>
+        {%- endif %}
+
+        {%- if msg %}
+        <div id="msg">
+            {{ msg }}
+        </div>
+        {%- endif %}
 
         {%- if avol %}
         <h1>admin panel:</h1>
@@ -59,18 +70,20 @@
 
         <h1>login for more:</h1>
         <ul>
-            <form method="post" enctype="multipart/form-data" action="/">
+            <form method="post" enctype="multipart/form-data" action="/{{ qvpath }}">
                 <input type="hidden" name="act" value="login" />
                 <input type="password" name="cppwd" />
                 <input type="submit" value="Login" />
             </form>
         </ul>
     </div>
+	<a href="#" id="repl">π</a>
     <script>
 
 if (localStorage.getItem('lightmode') != 1)
     document.documentElement.setAttribute("class", "dark");
 
 </script>
+<script src="/.cpr/util.js?_={{ ts }}"></script>
 </body>
 </html>

copyparty/web/ui.css

@@ -1,6 +1,16 @@
+@font-face {
+	font-family: 'scp';
+	font-display: swap;
+	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
+}
+html {
+	touch-action: manipulation;
+}
 #tt, #toast {
 	position: fixed;
 	max-width: 34em;
+	max-width: min(34em, 90%);
+	max-width: min(34em, calc(100% - 7em));
 	background: #222;
 	border: 0 solid #777;
 	box-shadow: 0 .2em .5em #222;
@@ -8,8 +18,9 @@
 	z-index: 9001;
 }
 #tt {
+	max-width: min(34em, calc(100% - 3.3em));
 	overflow: hidden;
-	margin-top: 1em;
+	margin: .7em 0;
 	padding: 0 1.3em;
 	height: 0;
 	opacity: .1;
@@ -20,7 +31,9 @@
 	right: -1em;
 	line-height: 1.5em;
 	padding: 1em 1.3em;
+	margin-left: 3em;
 	border-width: .4em 0;
+	overflow-wrap: break-word;
 	transform: translateX(100%);
 	transition:
 		transform .4s cubic-bezier(.2, 1.2, .5, 1),
@@ -28,7 +41,14 @@
 	text-shadow: 1px 1px 0 #000;
 	color: #fff;
 }
-#toastc {
+#toast a {
+	color: inherit;
+	text-shadow: inherit;
+	background: rgba(0, 0, 0, 0.4);
+	border-radius: .3em;
+	padding: .2em .3em;
+}
+#toast a#toastc {
 	display: inline-block;
 	position: absolute;
 	overflow: hidden;
@@ -37,7 +57,7 @@
 	opacity: 0;
 	padding: .3em 0;
 	margin: -.3em 0 0 0;
-	line-height: 1.5em;
+	line-height: 1.3em;
 	color: #000;
 	border: none;
 	outline: none;
@@ -45,6 +65,15 @@
 	border-radius: .5em 0 0 .5em;
 	transition: left .3s, width .3s, padding .3s, opacity .3s;
 }
+#toastb {
+	max-height: 70vh;
+	overflow-y: auto;
+}
+#toast.scroll #toastb {
+	overflow-y: scroll;
+	margin-right: -1.2em;
+	padding-right: .7em;
+}
 #toast pre {
 	margin: 0;
 }
@@ -73,7 +102,7 @@
 	background: #8e4;
 }
 #toast.warn {
-	background: #970;
+	background: #960;
 	border-color: #fc0;
 }
 #toast.warn #toastc {
@@ -101,6 +130,7 @@
 	padding: 1.5em 2em;
 	border-width: .5em 0;
 }
+#modalc code,
 #tt code {
 	background: #3c3c3c;
 	padding: .1em .3em;
@@ -119,6 +149,7 @@ html.light #tt,
 html.light #toast {
 	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
 }
+#modalc code,
 html.light #tt code {
 	background: #060;
 	color: #fff;
@@ -165,6 +196,16 @@ html.light #tt em {
         min-width: 30em;
     }
 }
+#modalc li {
+	margin: 1em 0;
+}
+#modalc h6 {
+	font-size: 1.3em;
+	border-bottom: 1px solid #999;
+	margin: 0;
+	padding: .3em;
+	text-align: center;
+}
 #modalb {
 	position: sticky;
 	text-align: right;
@@ -191,6 +232,8 @@ html.light #tt em {
 }
 #modali {
 	display: block;
+	background: #fff;
+	color: #000;
 	width: calc(100% - 1.25em);
     margin: 1em -.1em 0 -.1em;
 	padding: .5em;
@@ -201,3 +244,221 @@ html.light #tt em {
 #modali:focus {
 	border-color: #06d;
 }
+#repl_pre {
+	max-width: 24em;
+}
+*:focus,
+#pctl *:focus,
+.btn:focus {
+	box-shadow: 0 .1em .2em #fc0 inset;
+	border-radius: .2em;
+}
+html.light *:focus,
+html.light #pctl *:focus,
+html.light .btn:focus {
+	box-shadow: 0 .1em .2em #037 inset;
+}
+input[type="text"]:focus,
+input:not([type]):focus,
+textarea:focus {
+	box-shadow: 0 .1em .3em #fc0, 0 -.1em .3em #fc0;
+}
+html.light input[type="text"]:focus,
+html.light input:not([type]):focus,
+html.light textarea:focus {
+	box-shadow: 0 .1em .3em #037, 0 -.1em .3em #037;
+}
+
+
+
+
+
+
+
+
+
+
+
+.mdo pre,
+.mdo code,
+.mdo a {
+	color: #480;
+	background: #f7f7f7;
+	border: .07em solid #ddd;
+	border-radius: .2em;
+	padding: .1em .3em;
+	margin: 0 .1em;
+}
+.mdo pre,
+.mdo code,
+.mdo tt {
+	font-family: 'scp', monospace, monospace;
+	white-space: pre-wrap;
+	word-break: break-all;
+}
+.mdo code {
+	font-size: .96em;
+}
+.mdo h1,
+.mdo h2 {
+	line-height: 1.5em;
+}
+.mdo h1 {
+	font-size: 1.7em;
+	text-align: center;
+	border: 1em solid #777;
+	border-width: .05em 0;
+	margin: 3em 0;
+}
+.mdo h2 {
+	font-size: 1.5em;
+	font-weight: normal;
+	background: #f7f7f7;
+	border-top: .07em solid #fff;
+	border-bottom: .07em solid #bbb;
+	border-radius: .5em .5em 0 0;
+	padding-left: .4em;
+	margin-top: 3em;
+}
+.mdo h3 {
+	border-bottom: .1em solid #999;
+}
+.mdo h1 a, .mdo h3 a, .mdo h5 a,
+.mdo h2 a, .mdo h4 a, .mdo h6 a {
+	color: inherit;
+	display: block;
+	background: none;
+	border: none;
+	padding: 0;
+	margin: 0;
+}
+.mdo ul,
+.mdo ol {
+	border-left: .3em solid #ddd;
+}
+.mdo ul>li,
+.mdo ol>li {
+	margin: .7em 0;
+	list-style-type: disc;
+}
+.mdo strong {
+	color: #000;
+}
+.mdo p>em,
+.mdo li>em,
+.mdo td>em {
+	color: #c50;
+	padding: .1em;
+	border-bottom: .1em solid #bbb;
+}
+.mdo blockquote {
+	font-family: serif;
+	background: #f7f7f7;
+	border: .07em dashed #ccc;
+	padding: 0 2em;
+	margin: 1em 0;
+}
+.mdo small {
+	opacity: .8;
+}
+.mdo pre code {
+	display: block;
+	margin: 0 -.3em;
+	padding: .4em .5em;
+	line-height: 1.1em;
+}
+.mdo pre code:hover {
+	background: #fec;
+	color: #360;
+}
+.mdo table {
+	border-collapse: collapse;
+	margin: 1em 0;
+}
+.mdo th,
+.mdo td {
+	padding: .2em .5em;
+	border: .12em solid #aaa;
+}
+
+@media screen {
+	.mdo {
+		word-break: break-word;
+		overflow-wrap: break-word;
+		word-wrap: break-word; /*ie*/
+	}
+	html.light .mdo a,
+	.mdo a {
+		color: #fff;
+		background: #39b;
+		text-decoration: none;
+		padding: 0 .3em;
+		border: none;
+		border-bottom: .07em solid #079;
+	}
+	.mdo h1 {
+		color: #fff;
+		background: #444;
+		font-weight: normal;
+		border-top: .4em solid #fb0;
+		border-bottom: .4em solid #777;
+		border-radius: 0 1em 0 1em;
+		margin: 3em 0 1em 0;
+		padding: .5em 0;
+	}
+	.mdo h2 {
+		color: #fff;
+		background: #555;
+		margin-top: 2em;
+		border-bottom: .22em solid #999;
+		border-top: none;
+	}
+
+
+
+	html.dark .mdo a {
+		background: #057;
+	}
+	html.dark .mdo h1 a, html.dark .mdo h4 a,
+	html.dark .mdo h2 a, html.dark .mdo h5 a,
+	html.dark .mdo h3 a, html.dark .mdo h6 a {
+		color: inherit;
+		background: none;
+	}
+	html.dark .mdo pre,
+	html.dark .mdo code {
+		color: #8c0;
+		background: #1a1a1a;
+		border: .07em solid #333;
+	}
+	html.dark .mdo ul,
+	html.dark .mdo ol {
+		border-color: #444;
+	}
+	html.dark .mdo strong {
+		color: #fff;
+	}
+	html.dark .mdo p>em,
+	html.dark .mdo li>em,
+	html.dark .mdo td>em {
+		color: #f94;
+		border-color: #666;
+	}
+	html.dark .mdo h1 {
+		background: #383838;
+		border-top: .4em solid #b80;
+		border-bottom: .4em solid #4c4c4c;
+	}
+	html.dark .mdo h2 {
+		background: #444;
+		border-bottom: .22em solid #555;
+	}
+	html.dark .mdo td,
+	html.dark .mdo th {
+		border-color: #444;
+	}
+	html.dark .mdo blockquote {
+		background: #282828;
+		border: .07em dashed #444;
+	}
+}

copyparty/web/up2k.js

@@ -141,6 +141,15 @@ function U2pvis(act, btns) {
     r.tail = -1;
     r.wsz = 3;
 
+    var markup = {
+        '404': '<span class="err">404</span>',
+        'ERROR': '<span class="err">ERROR</span>',
+        'OS-error': '<span class="err">OS-error</span>',
+        'found': '<span class="inf">found</span>',
+        'YOLO': '<span class="inf">YOLO</span>',
+        'done': '<span class="ok">done</span>',
+    };
+
     r.addfile = function (entry, sz, draw) {
         r.tab.push({
             "hn": entry[0],
@@ -185,7 +194,7 @@ function U2pvis(act, btns) {
             return;
 
         var obj = ebi('f{0}{1}'.format(nfile, field.slice(1)));
-        obj.innerHTML = html;
+        obj.innerHTML = field == 'ht' ? (markup[html] || html) : html;
         if (field == 'hp') {
             obj.style.color = '';
             obj.style.background = '';
@@ -227,7 +236,7 @@ function U2pvis(act, btns) {
             p = r.perc(nb, 0, fobj.size, fobj.t_hashing);
 
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
-            p[0].toFixed(2), p[1], p[2].toFixed(2)
+            f2f(p[0], 2), p[1], f2f(p[2], 2)
         );
         if (!r.is_act(fo.in))
             return;
@@ -249,7 +258,7 @@ function U2pvis(act, btns) {
 
         var p = r.perc(fo.bd, fo.bd0, fo.bt, fobj.t_uploading);
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
-            p[0].toFixed(2), p[1], p[2].toFixed(2)
+            f2f(p[0], 2), p[1], f2f(p[2], 2)
         );
 
         if (!r.is_act(fo.in))
@@ -424,7 +433,7 @@ function U2pvis(act, btns) {
             td1 = '<td id="f' + nfile,
             td = '</td>' + td1,
             ret = td1 + 'n">' + row.hn +
-                td + 't">' + row.ht +
+                td + 't">' + (markup[row.ht] || row.ht) +
                 td + 'p" class="prog">' + row.hp + '</td>';
 
         if (as_html)
@@ -503,23 +512,27 @@ function up2k_init(subtle) {
         // chrome<37 firefox<34 edge<12 opera<24 safari<7
         shame = 'your browser is impressively ancient';
 
-    var got_deps = false;
+    function got_deps() {
+        return subtle || window.asmCrypto || window.hashwasm;
+    }
+
+    var loading_deps = false;
     function init_deps() {
-        if (!got_deps && !subtle && !window.asmCrypto) {
+        if (!loading_deps && !got_deps()) {
             var fn = 'sha512.' + sha_js + '.js';
             showmodal('<h1>loading ' + fn + '</h1><h2>since ' + shame + '</h2><h4>thanks chrome</h4>');
             import_js('/.cpr/deps/' + fn, unmodal);
 
             if (is_https)
-                ebi('u2foot').innerHTML = shame + ' so <em>this</em> uploader will do like 500kB/s at best';
+                ebi('u2foot').innerHTML = shame + ' so <em>this</em> uploader will do like 500 KiB/s at best';
             else
                 ebi('u2foot').innerHTML = 'seems like ' + shame + ' so do that if you want more performance <span style="color:#' +
                     (sha_js == 'ac' ? 'c84">(expecting 20' : '8a5">(but dont worry too much, expect 100') + ' MiB/s)</span>';
         }
-        got_deps = true;
+        loading_deps = true;
     }
 
-    if (perms.length && !has(perms, 'read'))
+    if (perms.length && !has(perms, 'read') && has(perms, 'write'))
         goto('up2k');
 
     function setmsg(msg, type) {
@@ -563,15 +576,17 @@ function up2k_init(subtle) {
     }
 
     var parallel_uploads = icfg_get('nthread'),
-        multitask = bcfg_get('multitask', true),
+        uc = {},
-        ask_up = bcfg_get('ask_up', true),
-        flag_en = bcfg_get('flag_en', false),
-        fsearch = bcfg_get('fsearch', false),
-        turbo = bcfg_get('u2turbo', false),
-        datechk = bcfg_get('u2tdate', true),
         fdom_ctr = 0,
         min_filebuf = 0;
 
+    bcfg_bind(uc, 'multitask', 'multitask', true, null, false);
+    bcfg_bind(uc, 'ask_up', 'ask_up', true, null, false);
+    bcfg_bind(uc, 'flag_en', 'flag_en', false, apply_flag_cfg);
+    bcfg_bind(uc, 'fsearch', 'fsearch', false, set_fsearch, false);
+    bcfg_bind(uc, 'turbo', 'u2turbo', false, draw_turbo, false);
+    bcfg_bind(uc, 'datechk', 'u2tdate', true, null, false);
+
     var st = {
         "files": [],
         "todo": {
@@ -587,8 +602,15 @@ function up2k_init(subtle) {
             "upload": []
         },
         "bytes": {
+            "total": 0,
             "hashed": 0,
-            "uploaded": 0
+            "uploaded": 0,
+            "finished": 0
+        },
+        "time": {
+            "hashing": 0,
+            "uploading": 0,
+            "busy": 0
         }
     };
 
@@ -617,20 +639,97 @@ function up2k_init(subtle) {
     function nav() {
         ebi('file' + fdom_ctr).click();
     }
-    ebi('u2btn').addEventListener('click', nav, false);
+    ebi('u2btn').onclick = nav;
 
+    var nenters = 0;
     function ondrag(e) {
-        e.stopPropagation();
+        if (++nenters <= 0)
-        e.preventDefault();
+            nenters = 1;
+
+        //console.log(nenters, Date.now(), 'enter', this, e.target);
+        if (onover.bind(this)(e))
+            return true;
+
+        var mup, up = QS('#up_zd');
+        var msr, sr = QS('#srch_zd');
+        if (!has(perms, 'write'))
+            mup = 'you do not have write-access to this folder';
+        if (!has(perms, 'read'))
+            msr = 'you do not have read-access to this folder';
+        if (!have_up2k_idx)
+            msr = 'file-search is not enabled in server config';
+
+        up.querySelector('span').textContent = mup || 'drop it here';
+        sr.querySelector('span').textContent = msr || 'drop it here';
+        clmod(up, 'err', mup);
+        clmod(sr, 'err', msr);
+        clmod(up, 'ok', !mup);
+        clmod(sr, 'ok', !msr);
+        ebi('up_dz').setAttribute('err', mup || '');
+        ebi('srch_dz').setAttribute('err', msr || '');
+    }
+    function onover(e) {
+        try {
+            var ok = false, dt = e.dataTransfer.types;
+            for (var a = 0; a < dt.length; a++)
+                if (dt[a] == 'Files')
+                    ok = true;
+
+            if (!ok)
+                return true;
+        }
+        catch (ex) { }
+
+        ev(e);
         e.dataTransfer.dropEffect = 'copy';
         e.dataTransfer.effectAllowed = 'copy';
+        clmod(ebi('drops'), 'vis', 1);
+        var v = this.getAttribute('v');
+        if (v)
+            clmod(ebi(v), 'hl', 1);
     }
-    ebi('u2btn').addEventListener('dragover', ondrag, false);
+    function offdrag(e) {
-    ebi('u2btn').addEventListener('dragenter', ondrag, false);
+        ev(e);
+
+        var v = this.getAttribute('v');
+        if (v)
+            clmod(ebi(v), 'hl');
+
+        if (--nenters <= 0) {
+            clmod(ebi('drops'), 'vis');
+            clmod(ebi('up_dz'), 'hl');
+            clmod(ebi('srch_dz'), 'hl');
+        }
+
+        //console.log(nenters, Date.now(), 'leave', this, e && e.target);
+    }
+    document.body.ondragenter = ondrag;
+    document.body.ondragleave = offdrag;
+
+    var drops = [ebi('up_dz'), ebi('srch_dz')];
+    for (var a = 0; a < 2; a++) {
+        drops[a].ondragenter = ondrag;
+        drops[a].ondragover = onover;
+        drops[a].ondragleave = offdrag;
+        drops[a].ondrop = gotfile;
+    }
+    ebi('drops').onclick = offdrag;  // old ff
 
     function gotfile(e) {
-        e.stopPropagation();
+        ev(e);
-        e.preventDefault();
+        nenters = 0;
+        offdrag.bind(this)();
+        var dz = (this && this.getAttribute('id'));
+
+        var err = this.getAttribute('err');
+        if (err)
+            return modal.alert('sorry, ' + err);
+
+        if ((dz == 'up_dz' && uc.fsearch) || (dz == 'srch_dz' && !uc.fsearch))
+            tgl_fsearch();
+
+        if (!QS('#op_up2k.act'))
+            goto('up2k');
 
         var files,
             is_itemlist = false;
@@ -649,11 +748,14 @@ function up2k_init(subtle) {
 
         more_one_file();
         var bad_files = [],
+            nil_files = [],
             good_files = [],
             dirs = [];
 
         for (var a = 0; a < files.length; a++) {
-            var fobj = files[a];
+            var fobj = files[a],
+                dst = good_files;
+
             if (is_itemlist) {
                 if (fobj.kind !== 'file')
                     continue;
@@ -670,16 +772,15 @@ function up2k_init(subtle) {
             }
             try {
                 if (fobj.size < 1)
-                    throw 1;
+                    dst = nil_files;
             }
             catch (ex) {
-                bad_files.push(fobj.name);
+                dst = bad_files;
-                continue;
             }
-            good_files.push([fobj, fobj.name]);
+            dst.push([fobj, fobj.name]);
         }
         if (dirs) {
-            return read_dirs(null, [], dirs, good_files, bad_files);
+            return read_dirs(null, [], dirs, good_files, nil_files, bad_files);
         }
     }
 
@@ -693,7 +794,7 @@ function up2k_init(subtle) {
     }
 
     var rd_missing_ref = [];
-    function read_dirs(rd, pf, dirs, good, bad, spins) {
+    function read_dirs(rd, pf, dirs, good, nil, bad, spins) {
         spins = spins || 0;
         if (++spins == 5)
             rd_missing_ref = rd_flatten(pf, dirs);
@@ -709,12 +810,12 @@ function up2k_init(subtle) {
                     match = false;
 
             if (match) {
-                var msg = ['directory iterator got stuck on the following {0} items; good chance your browser is about to spinlock:'.format(missing.length)];
+                var msg = ['directory iterator got stuck on the following {0} items; good chance your browser is about to spinlock:<ul>'.format(missing.length)];
                 for (var a = 0; a < Math.min(20, missing.length); a++)
-                    msg.push(missing[a]);
+                    msg.push('<li>' + esc(missing[a]) + '</li>');
 
-                return modal.alert(msg.join('\n-- '), function () {
+                return modal.alert(msg.join('') + '</ul>', function () {
-                    read_dirs(rd, [], [], good, bad, spins);
+                    read_dirs(rd, [], [], good, nil, bad, spins);
                 });
             }
             spins = 0;
@@ -722,11 +823,11 @@ function up2k_init(subtle) {
 
         if (!dirs.length) {
             if (!pf.length)
-                return gotallfiles(good, bad);
+                return gotallfiles(good, nil, bad);
 
             console.log("retry pf, " + pf.length);
             setTimeout(function () {
-                read_dirs(rd, pf, dirs, good, bad, spins);
+                read_dirs(rd, pf, dirs, good, nil, bad, spins);
             }, 50);
             return;
         }
@@ -748,14 +849,15 @@ function up2k_init(subtle) {
                     pf.push(name);
                     dn.file(function (fobj) {
                         apop(pf, name);
+                        var dst = good;
                         try {
-                            if (fobj.size > 0) {
+                            if (fobj.size < 1)
-                                good.push([fobj, name]);
+                                dst = nil;
-                                return;
                         }
+                        catch (ex) {
+                            dst = bad;
                         }
-                        catch (ex) { }
+                        dst.push([fobj, name]);
-                        bad.push(name);
                     });
                 }
                 ngot += 1;
@@ -764,32 +866,48 @@ function up2k_init(subtle) {
                 dirs.shift();
                 rd = null;
             }
-            return read_dirs(rd, pf, dirs, good, bad, spins);
+            return read_dirs(rd, pf, dirs, good, nil, bad, spins);
         });
     }
 
-    function gotallfiles(good_files, bad_files) {
+    function gotallfiles(good_files, nil_files, bad_files) {
+        var ntot = good_files.concat(nil_files, bad_files).length;
         if (bad_files.length) {
-            var ntot = bad_files.length + good_files.length,
+            var msg = 'These {0} files (of {1} total) were skipped, possibly due to filesystem permissions:\n'.format(bad_files.length, ntot);
-                msg = 'These {0} files (of {1} total) were skipped because they are empty:\n'.format(bad_files.length, ntot);
-
             for (var a = 0, aa = Math.min(20, bad_files.length); a < aa; a++)
-                msg += '-- ' + bad_files[a] + '\n';
+                msg += '-- ' + bad_files[a][1] + '\n';
-
-            if (good_files.length - bad_files.length <= 1 && ANDROID)
-                msg += '\nFirefox-Android has a bug which prevents selecting multiple files. Try selecting one file at a time. For more info, see firefox bug 1456557';
 
+            msg += '\nMaybe it works better if you select just one file';
             return modal.alert(msg, function () {
-                gotallfiles(good_files, []);
+                gotallfiles(good_files, nil_files, []);
             });
         }
 
-        var msg = ['upload these ' + good_files.length + ' files?'];
+        if (nil_files.length) {
-        for (var a = 0, aa = Math.min(20, good_files.length); a < aa; a++)
+            var msg = 'These {0} files (of {1} total) are blank/empty; upload them anyways?\n'.format(nil_files.length, ntot);
-            msg.push(good_files[a][1]);
+            for (var a = 0, aa = Math.min(20, nil_files.length); a < aa; a++)
+                msg += '-- ' + nil_files[a][1] + '\n';
 
-        if (ask_up && !fsearch)
+            msg += '\nMaybe it works better if you select just one file';
-            return modal.confirm(msg.join('\n'), function () { up_them(good_files); }, null);
+            return modal.confirm(msg, function () {
+                gotallfiles(good_files.concat(nil_files), [], []);
+            }, function () {
+                gotallfiles(good_files, [], []);
+            });
+        }
+
+        good_files.sort(function (a, b) {
+            a = a[1];
+            b = b[1];
+            return a < b ? -1 : a > b ? 1 : 0;
+        });
+
+        var msg = ['{0} these {1} files?<ul>'.format(uc.fsearch ? 'search' : 'upload', good_files.length)];
+        for (var a = 0, aa = Math.min(20, good_files.length); a < aa; a++)
+            msg.push('<li>' + esc(good_files[a][1]) + '</li>');
+
+        if (uc.ask_up && !uc.fsearch)
+            return modal.confirm(msg.join('') + '</ul>', function () { up_them(good_files); }, null);
 
         up_them(good_files);
     }
@@ -805,13 +923,13 @@ function up2k_init(subtle) {
         for (var a = 0; a < good_files.length; a++) {
             var fobj = good_files[a][0],
                 name = good_files[a][1],
-                fdir = '',
+                fdir = evpath,
                 now = Date.now(),
                 lmod = fobj.lastModified || now,
                 ofs = name.lastIndexOf('/') + 1;
 
             if (ofs) {
-                fdir = name.slice(0, ofs);
+                fdir += url_enc(name.slice(0, ofs));
                 name = name.slice(ofs);
             }
 
@@ -820,7 +938,7 @@ function up2k_init(subtle) {
                 "t0": now,
                 "fobj": fobj,
                 "name": name,
-                "size": fobj.size,
+                "size": fobj.size || 0,
                 "lmod": lmod / 1000,
                 "purl": fdir,
                 "done": false,
@@ -828,20 +946,26 @@ function up2k_init(subtle) {
             },
                 key = entry.name + '\n' + entry.size;
 
+            if (uc.fsearch)
+                entry.srch = 1;
+
             if (seen[key])
                 continue;
 
             seen[key] = 1;
 
             pvis.addfile([
-                fsearch ? esc(entry.name) : linksplit(
+                uc.fsearch ? esc(entry.name) : linksplit(
-                    uricom_dec(entry.purl)[0] + entry.name).join(' '),
+                    entry.purl + uricom_enc(entry.name)).join(' '),
                 '📐 hash',
                 ''
             ], fobj.size, draw_each);
 
+            st.bytes.total += fobj.size;
             st.files.push(entry);
-            if (turbo)
+            if (!entry.size)
+                push_t(st.todo.handshake, entry);
+            else if (uc.turbo)
                 push_t(st.todo.head, entry);
             else
                 push_t(st.todo.hash, entry);
@@ -851,14 +975,13 @@ function up2k_init(subtle) {
             pvis.changecard(pvis.act);
         }
     }
-    ebi('u2btn').addEventListener('drop', gotfile, false);
 
     function more_one_file() {
         fdom_ctr++;
         var elm = mknod('div');
         elm.innerHTML = '<input id="file{0}" type="file" name="file{0}[]" multiple="multiple" />'.format(fdom_ctr);
         ebi('u2form').appendChild(elm);
-        ebi('file' + fdom_ctr).addEventListener('change', gotfile, false);
+        ebi('file' + fdom_ctr).onchange = gotfile;
     }
     more_one_file();
 
@@ -878,6 +1001,81 @@ function up2k_init(subtle) {
     }
     ebi('u2cleanup').onclick = u2cleanup;
 
+    var etaref = 0, etaskip = 0, op_minh = 0;
+    function etafun() {
+        var nhash = st.busy.head.length + st.busy.hash.length + st.todo.head.length + st.todo.hash.length,
+            nsend = st.busy.upload.length + st.todo.upload.length,
+            now = Date.now(),
+            td = (now - (etaref || now)) / 1000.0;
+
+        etaref = now;
+        if (td > 1.2)
+            td = 0.05;
+
+        //ebi('acc_info').innerHTML = humantime(st.time.busy) + ' ' + f2f(now / 1000, 1);
+
+        var op = ebi('op_up2k'),
+            uff = ebi('u2footfoot'),
+            minh = QS('#op_up2k.act') ? Math.max(op_minh, uff.offsetTop + uff.offsetHeight - op.offsetTop + 32) : 0;
+
+        if (minh > op_minh || !op_minh) {
+            op_minh = minh;
+            op.style.minHeight = op_minh + 'px';
+        }
+
+        if (!nhash)
+            ebi('u2etah').innerHTML = 'Done ({0}, {1} files)'.format(humansize(st.bytes.hashed), pvis.ctr["ok"] + pvis.ctr["ng"]);
+
+        if (!nsend && !nhash)
+            ebi('u2etau').innerHTML = ebi('u2etat').innerHTML = (
+                'Done ({0}, {1} files)'.format(humansize(st.bytes.uploaded), pvis.ctr["ok"] + pvis.ctr["ng"]));
+
+        if (!st.busy.hash.length && !hashing_permitted())
+            nhash = 0;
+
+        if (!parallel_uploads || !(nhash + nsend) || (flag && flag.owner && !flag.ours))
+            return;
+
+        var t = [];
+        if (nhash) {
+            st.time.hashing += td;
+            t.push(['u2etah', st.bytes.hashed, st.bytes.hashed, st.time.hashing]);
+            if (uc.fsearch)
+                t.push(['u2etat', st.bytes.hashed, st.bytes.hashed, st.time.hashing]);
+        }
+        if (nsend) {
+            st.time.uploading += td;
+            t.push(['u2etau', st.bytes.uploaded, st.bytes.finished, st.time.uploading]);
+        }
+        if ((nhash || nsend) && !uc.fsearch) {
+            if (!st.bytes.finished) {
+                ebi('u2etat').innerHTML = '(preparing to upload)';
+            }
+            else {
+                st.time.busy += td;
+                t.push(['u2etat', st.bytes.finished, st.bytes.finished, st.time.busy]);
+            }
+        }
+        for (var a = 0; a < t.length; a++) {
+            var rem = st.bytes.total - t[a][2],
+                bps = t[a][1] / t[a][3],
+                eta = Math.floor(rem / bps);
+
+            if (t[a][1] < 1024 || t[a][3] < 0.1) {
+                ebi(t[a][0]).innerHTML = '(preparing to upload)';
+                continue;
+            }
+
+            if (etaskip)
+                continue;
+
+            ebi(t[a][0]).innerHTML = '{0}, {1}/s, {2}'.format(
+                humansize(rem), humansize(bps, 1), humantime(eta));
+        }
+        if (++etaskip > 2)
+            etaskip = 0;
+    }
+
     /////
     ////
     ///   actuator
@@ -902,12 +1100,7 @@ function up2k_init(subtle) {
             st.busy.handshake.length)
             return false;
 
-        if (st.busy.handshake.length)
+        if ((uc.multitask ? 1 : 0) <
-            for (var n = t.n - 1; n >= t.n - parallel_uploads && n >= 0; n--)
-                if (st.files[n].t_uploading)
-                    return false;
-
-        if ((multitask ? 1 : 0) <
             st.todo.upload.length +
             st.busy.upload.length)
             return false;
@@ -916,8 +1109,11 @@ function up2k_init(subtle) {
     }
 
     function hashing_permitted() {
-        if (multitask) {
+        if (!parallel_uploads)
-            var ahead = st.bytes.hashed - st.bytes.uploaded;
+            return false;
+
+        if (uc.multitask) {
+            var ahead = st.bytes.hashed - st.bytes.finished;
             return ahead < 1024 * 1024 * 1024 * 4 &&
                 st.todo.handshake.length + st.busy.handshake.length < 16;
         }
@@ -940,7 +1136,7 @@ function up2k_init(subtle) {
             if (running)
                 return;
 
-            if (crashed)
+            if (crashed || !got_deps())
                 return defer();
 
             running = true;
@@ -956,16 +1152,60 @@ function up2k_init(subtle) {
                         st.busy.handshake.length +
                         st.busy.upload.length;
 
+                if (was_busy && !is_busy) {
+                    for (var a = 0; a < st.files.length; a++) {
+                        var t = st.files[a];
+                        if (t.want_recheck) {
+                            t.rechecks++;
+                            t.want_recheck = false;
+                            push_t(st.todo.handshake, t);
+                        }
+                    }
+                    is_busy = st.todo.handshake.length;
+                }
+
                 if (was_busy != is_busy) {
                     was_busy = is_busy;
 
                     window[(is_busy ? "add" : "remove") +
                         "EventListener"]("beforeunload", warn_uploader_busy);
+
+                    if (!is_busy) {
+                        var k = uc.fsearch ? 'searches' : 'uploads',
+                            ks = uc.fsearch ? 'Search' : 'Upload',
+                            tok = uc.fsearch ? 'successful (found on server)' : 'completed successfully',
+                            tng = uc.fsearch ? 'failed (NOT found on server)' : 'failed, sorry',
+                            ok = pvis.ctr["ok"],
+                            ng = pvis.ctr["ng"],
+                            t = uc.ask_up ? 0 : 10;
+
+                        if (ok && ng)
+                            toast.warn(t, 'Finished, but some {0} failed:\n{1} {2},\n{3} {4}'.format(k, ok, tok, ng, tng));
+                        else if (ok > 1)
+                            toast.ok(t, 'All {1} {0} {2}'.format(k, ok, tok));
+                        else if (ok)
+                            toast.ok(t, '{0} {1}'.format(ks, tok));
+                        else if (ng > 1)
+                            toast.err(t, 'All {1} {0} {2}'.format(k, ng, tng));
+                        else if (ng)
+                            toast.err(t, '{0} {1}'.format(ks, tng));
+
+                        timer.rm(etafun);
+                        op_minh = 0;
+                    }
+                    else {
+                        timer.add(etafun, false);
+                        ebi('u2etas').style.textAlign = 'left';
+                    }
+                    etafun();
+                    if (pvis.act == 'bz')
+                        pvis.changecard('bz');
                 }
 
                 if (flag) {
                     if (is_busy) {
                         flag.take(now);
+                        ebi('u2flagblock').style.display = flag.ours ? '' : 'block';
                         if (!flag.ours)
                             return defer();
                     }
@@ -1101,7 +1341,6 @@ function up2k_init(subtle) {
     function exec_hash() {
         var t = st.todo.hash.shift();
         st.busy.hash.push(t);
-        st.bytes.hashed += t.size;
         t.bytes_uploaded = 0;
 
         var bpend = 0,
@@ -1127,6 +1366,7 @@ function up2k_init(subtle) {
                 cdr = t.size;
 
             bpend += cdr - car;
+            st.bytes.hashed += cdr - car;
 
             function orz(e) {
                 if (!min_filebuf && nch == 1) {
@@ -1157,11 +1397,11 @@ function up2k_init(subtle) {
                 if (handled) {
                     pvis.move(t.n, 'ng');
                     apop(st.busy.hash, t);
-                    st.bytes.uploaded += t.size;
+                    st.bytes.finished += t.size;
-                    return tasker();
+                    return;
                 }
 
-                toast.err(0, 'y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
+                toast.err(0, 'y o u   b r o k e    i t\nfile: ' + esc(t.name + '') + '\nerror: ' + err);
             };
             reader.readAsArrayBuffer(
                 bobslice.call(t.fobj, car, cdr));
@@ -1234,7 +1474,6 @@ function up2k_init(subtle) {
             console.log('head onerror, retrying', t);
             apop(st.busy.head, t);
             st.todo.head.unshift(t);
-            tasker();
         };
         function orz(e) {
             var ok = false;
@@ -1243,27 +1482,33 @@ function up2k_init(subtle) {
                     srv_ts = xhr.getResponseHeader('Last-Modified');
 
                 ok = t.size == srv_sz;
-                if (ok && datechk) {
+                if (ok && uc.datechk) {
                     srv_ts = new Date(srv_ts) / 1000;
                     ok = Math.abs(srv_ts - t.lmod) < 2;
                 }
             }
             apop(st.busy.head, t);
-            if (!ok)
+            if (!ok && !t.srch) {
-                return push_t(st.todo.hash, t);
+                push_t(st.todo.hash, t);
+                tasker();
+                return;
+            }
 
             t.done = true;
+            t.fobj = null;
             st.bytes.hashed += t.size;
-            st.bytes.uploaded += t.size;
+            st.bytes.finished += t.size;
-            pvis.seth(t.n, 1, 'YOLO');
+            pvis.move(t.n, 'bz');
+            pvis.seth(t.n, 1, ok ? 'YOLO' : '404');
             pvis.seth(t.n, 2, "turbo'd");
-            pvis.move(t.n, 'ok');
+            pvis.move(t.n, ok ? 'ok' : 'ng');
+            tasker();
         };
         xhr.onload = function (e) {
             try { orz(e); } catch (ex) { vis_exh(ex + '', '', '', '', ex); }
         };
 
-        xhr.open('HEAD', t.purl + t.name, true);
+        xhr.open('HEAD', t.purl + uricom_enc(t.name) + '?raw', true);
         xhr.send();
     }
 
@@ -1294,7 +1539,6 @@ function up2k_init(subtle) {
             apop(st.busy.handshake, t);
             st.todo.handshake.unshift(t);
             t.keepalive = keepalive;
-            tasker();
         };
         function orz(e) {
             if (t.t_busied != me) {
@@ -1320,32 +1564,37 @@ function up2k_init(subtle) {
                     }
                     else {
                         smsg = 'found';
-                        var hit = response.hits[0],
+                        var msg = [];
-                            msg = linksplit(hit.rp).join(''),
+                        for (var a = 0, aa = Math.min(20, response.hits.length); a < aa; a++) {
+                            var hit = response.hits[a],
                                 tr = unix2iso(hit.ts),
                                 tu = unix2iso(t.lmod),
                                 diff = parseInt(t.lmod) - parseInt(hit.ts),
                                 cdiff = (Math.abs(diff) <= 2) ? '3c0' : 'f0b',
                                 sdiff = '<span style="color:#' + cdiff + '">diff ' + diff;
 
-                        msg += '<br /><small>' + tr + ' (srv), ' + tu + ' (You), ' + sdiff + '</span></span>';
+                            msg.push(linksplit(hit.rp).join('') + '<br /><small>' + tr + ' (srv), ' + tu + ' (You), ' + sdiff + '</small></span>');
+                        }
+                        msg = msg.join('<br />\n');
                     }
                     pvis.seth(t.n, 2, msg);
                     pvis.seth(t.n, 1, smsg);
                     pvis.move(t.n, smsg == '404' ? 'ng' : 'ok');
                     apop(st.busy.handshake, t);
-                    st.bytes.uploaded += t.size;
+                    st.bytes.finished += t.size;
                     t.done = true;
+                    t.fobj = null;
                     tasker();
                     return;
                 }
 
-                if (response.purl !== t.purl || response.name !== t.name) {
+                var rsp_purl = url_enc(response.purl);
+                if (rsp_purl !== t.purl || response.name !== t.name) {
                     // server renamed us (file exists / path restrictions)
-                    console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + response.purl + "] [" + response.name + "]");
+                    console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + rsp_purl + "] [" + response.name + "]");
-                    t.purl = response.purl;
+                    t.purl = rsp_purl;
                     t.name = response.name;
-                    pvis.seth(t.n, 0, linksplit(t.purl + t.name).join(' '));
+                    pvis.seth(t.n, 0, linksplit(t.purl + uricom_enc(t.name)).join(' '));
                 }
 
                 var chunksize = get_chunksize(t.size),
@@ -1374,7 +1623,7 @@ function up2k_init(subtle) {
                 pvis.prog(t, 0, cbd[0]);
 
                 var done = true,
-                    msg = '&#x1f3b7;&#x1f41b;';
+                    msg = 'done';
 
                 if (t.postlist.length) {
                     var arr = st.todo.upload,
@@ -1401,12 +1650,13 @@ function up2k_init(subtle) {
 
                 if (done) {
                     t.done = true;
-                    st.bytes.uploaded += t.size - t.bytes_uploaded;
+                    t.fobj = null;
+                    st.bytes.finished += t.size - t.bytes_uploaded;
                     var spd1 = (t.size / ((t.t_hashed - t.t_hashing) / 1000.)) / (1024 * 1024.),
                         spd2 = (t.size / ((t.t_uploaded - t.t_uploading) / 1000.)) / (1024 * 1024.);
 
                     pvis.seth(t.n, 2, 'hash {0}, up {1} MB/s'.format(
-                        spd1.toFixed(2), spd2.toFixed(2)));
+                        f2f(spd1, 2), isNaN(spd2) ? '--' : f2f(spd2, 2)));
 
                     pvis.move(t.n, 'ok');
                 }
@@ -1434,13 +1684,19 @@ function up2k_init(subtle) {
                     return;
                 }
 
-                st.bytes.uploaded += t.size;
+                st.bytes.finished += t.size;
-                if (rsp.indexOf('partial upload exists') !== -1 ||
+                var err_pend = rsp.indexOf('partial upload exists') + 1,
-                    rsp.indexOf('file already exists') !== -1) {
+                    err_dupe = rsp.indexOf('file already exists') + 1;
+
+                if (err_pend || err_dupe) {
                     err = rsp;
                     ofs = err.indexOf('\n/');
                     if (ofs !== -1) {
-                        err = err.slice(0, ofs + 1) + linksplit(err.slice(ofs + 2)).join(' ');
+                        err = err.slice(0, ofs + 1) + linksplit(err.slice(ofs + 2).trimEnd()).join(' ');
+                    }
+                    if (!t.rechecks && err_pend) {
+                        t.rechecks = 0;
+                        t.want_recheck = true;
                     }
                 }
                 if (err != "") {
@@ -1469,7 +1725,7 @@ function up2k_init(subtle) {
             "lmod": t.lmod,
             "hash": t.hash
         };
-        if (fsearch)
+        if (t.srch)
             req.srch = 1;
 
         xhr.open('POST', t.purl, true);
@@ -1487,7 +1743,8 @@ function up2k_init(subtle) {
         st.busy.upload.push(upt);
 
         var npart = upt.npart,
-            t = st.files[upt.nfile];
+            t = st.files[upt.nfile],
+            tries = 0;
 
         if (!t.t_uploading)
             t.t_uploading = Date.now();
@@ -1505,6 +1762,7 @@ function up2k_init(subtle) {
             var txt = ((xhr.response && xhr.response.err) || xhr.responseText) + '';
             if (xhr.status == 200) {
                 pvis.prog(t, npart, cdr - car);
+                st.bytes.finished += cdr - car;
                 st.bytes.uploaded += cdr - car;
                 t.bytes_uploaded += cdr - car;
             }
@@ -1537,8 +1795,9 @@ function up2k_init(subtle) {
                 if (crashed)
                     return;
 
-                console.log('chunkpit onerror, retrying', t);
+                toast.err(9.98, "failed to upload a chunk,\n" + tries + " retries so far -- retrying in 10sec\n\n" + t.name);
-                do_send();
+                console.log('chunkpit onerror,', ++tries, t);
+                setTimeout(do_send, 10 * 1000);
             };
             xhr.open('POST', t.purl, true);
             xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
@@ -1572,6 +1831,7 @@ function up2k_init(subtle) {
             parent.appendChild(btn);
             ebi('u2conf').setAttribute('class', wide ? 'has_btn' : '');
             ebi('u2cards').setAttribute('class', wide ? 'w' : '');
+            ebi('u2etaw').setAttribute('class', wide ? 'w' : '');
         }
     }
     window.addEventListener('resize', onresize);
@@ -1632,42 +1892,21 @@ function up2k_init(subtle) {
         bumpthread({ "target": 1 })
     }
 
-    function tgl_multitask() {
-        multitask = !multitask;
-        bcfg_set('multitask', multitask);
-    }
-
-    function tgl_ask_up() {
-        ask_up = !ask_up;
-        bcfg_set('ask_up', ask_up);
-    }
-
     function tgl_fsearch() {
-        set_fsearch(!fsearch);
+        set_fsearch(!uc.fsearch);
-    }
-
-    function tgl_turbo() {
-        turbo = !turbo;
-        bcfg_set('u2turbo', turbo);
-        draw_turbo();
-    }
-
-    function tgl_datechk() {
-        datechk = !datechk;
-        bcfg_set('u2tdate', datechk);
     }
 
     function draw_turbo() {
         var msgu = '<p class="warn">WARNING: turbo enabled, <span>&nbsp;client may not detect and resume incomplete uploads; see turbo-button tooltip</span></p>',
-            msgs = '<p class="warn">WARNING: turbo enabled, <span>&nbsp;search may give false-positives; see turbo-button tooltip</span></p>',
+            msgs = '<p class="warn">WARNING: turbo enabled, <span>&nbsp;search results can be incorrect; see turbo-button tooltip</span></p>',
-            msg = fsearch ? msgs : msgu,
+            msg = uc.fsearch ? msgs : msgu,
-            omsg = fsearch ? msgu : msgs,
+            omsg = uc.fsearch ? msgu : msgs,
             html = ebi('u2foot').innerHTML,
             ohtml = html;
 
-        if (turbo && html.indexOf(msg) === -1)
+        if (uc.turbo && html.indexOf(msg) === -1)
             html = html.replace(omsg, '') + msg;
-        else if (!turbo)
+        else if (!uc.turbo)
             html = html.replace(msgu, '').replace(msgs, '');
 
         if (html !== ohtml)
@@ -1693,8 +1932,8 @@ function up2k_init(subtle) {
         }
 
         if (new_state !== undefined) {
-            fsearch = new_state;
+            uc.fsearch = new_state;
-            bcfg_set('fsearch', fsearch);
+            bcfg_set('fsearch', uc.fsearch);
         }
 
         try {
@@ -1703,11 +1942,10 @@ function up2k_init(subtle) {
         catch (ex) { }
 
         try {
-            var fun = fsearch ? 'add' : 'remove',
+            var ico = uc.fsearch ? '🔎' : '🚀',
-                ico = fsearch ? '🔎' : '🚀',
+                desc = uc.fsearch ? 'Search' : 'Upload';
-                desc = fsearch ? 'Search' : 'Upload';
 
-            ebi('op_up2k').classList[fun]('srch');
+            clmod(ebi('op_up2k'), 'srch', uc.fsearch);
             ebi('u2bm').innerHTML = ico + ' <sup>' + desc + '</sup>';
         }
         catch (ex) { }
@@ -1716,23 +1954,20 @@ function up2k_init(subtle) {
         onresize();
     }
 
-    function tgl_flag_en() {
-        flag_en = !flag_en;
-        bcfg_set('flag_en', flag_en);
-        apply_flag_cfg();
-    }
-
     function apply_flag_cfg() {
-        if (flag_en && !flag) {
+        if (uc.flag_en && !flag) {
             try {
                 flag = up2k_flagbus();
             }
             catch (ex) {
-                console.log("flag error: " + ex.toString());
+                toast.err(5, "not supported on your browser:\n" + esc(basenames(ex)));
-                tgl_flag_en();
+                bcfg_set('flag_en', false);
             }
         }
-        else if (!flag_en && flag) {
+        else if (!uc.flag_en && flag) {
+            if (flag.ours)
+                flag.give();
+
             flag.ch.close();
             flag = false;
         }
@@ -1753,26 +1988,19 @@ function up2k_init(subtle) {
     };
 
     ebi('nthread').onkeydown = bumpthread2;
-    ebi('nthread').addEventListener('input', bumpthread, false);
+    ebi('nthread').oninput = bumpthread;
-    ebi('multitask').addEventListener('click', tgl_multitask, false);
-    ebi('ask_up').addEventListener('click', tgl_ask_up, false);
-    ebi('flag_en').addEventListener('click', tgl_flag_en, false);
-    ebi('u2turbo').addEventListener('click', tgl_turbo, false);
-    ebi('u2tdate').addEventListener('click', tgl_datechk, false);
-    var o = ebi('fsearch');
-    if (o)
-        o.addEventListener('click', tgl_fsearch, false);
 
-    var nodes = ebi('u2conf').getElementsByTagName('a');
+    ebi('u2etas').onclick = function (e) {
-    for (var a = nodes.length - 1; a >= 0; a--)
+        ev(e);
-        nodes[a].addEventListener('touchend', nop, false);
+        clmod(ebi('u2etas'), 'o', 't');
+    };
 
     set_fsearch();
     bumpthread({ "target": 1 });
     if (parallel_uploads < 1)
         bumpthread(1);
 
-    return { "init_deps": init_deps, "set_fsearch": set_fsearch }
+    return { "init_deps": init_deps, "set_fsearch": set_fsearch, "ui": pvis }
 }
 
 

copyparty/web/util.js

@@ -8,7 +8,8 @@ if (!window['console'])
 
 var is_touch = 'ontouchstart' in window,
     IPHONE = /iPhone|iPad|iPod/i.test(navigator.userAgent),
-    ANDROID = /android/i.test(navigator.userAgent);
+    ANDROID = /android/i.test(navigator.userAgent),
+    WINDOWS = navigator.platform ? navigator.platform == 'Win32' : /Windows/.test(navigator.userAgent);
 
 
 var ebi = document.getElementById.bind(document),
@@ -28,6 +29,50 @@ function esc(txt) {
         }[c];
     });
 }
+function basenames(txt) {
+    return (txt + '').replace(/https?:\/\/[^ \/]+\//g, '/').replace(/js\?_=[a-zA-Z]{4}/g, 'js');
+}
+if ((document.location + '').indexOf(',rej,') + 1)
+    window.onunhandledrejection = function (e) {
+        var err = e.reason;
+        try {
+            err += '\n' + e.reason.stack;
+        }
+        catch (e) { }
+        err = basenames(err);
+        console.log("REJ: " + err);
+        try {
+            toast.warn(30, err);
+        }
+        catch (e) { }
+    };
+
+try {
+    console.hist = [];
+    var hook = function (t) {
+        var orig = console[t].bind(console),
+            cfun = function () {
+                console.hist.push(Date.now() + ' ' + t + ': ' + Array.from(arguments).join(', '));
+                if (console.hist.length > 100)
+                    console.hist = console.hist.slice(50);
+
+                orig.apply(console, arguments);
+            };
+
+        console['std' + t] = orig;
+        console[t] = cfun;
+    };
+    hook('log');
+    console.log('log-capture ok');
+    hook('debug');
+    hook('warn');
+    hook('error');
+}
+catch (ex) {
+    if (console.stdlog)
+        console.log = console.stdlog;
+    console.log(ex);
+}
 var crashed = false, ignexd = {};
 function vis_exh(msg, url, lineNo, columnNo, error) {
     if ((msg + '').indexOf('ResizeObserver') !== -1)
@@ -39,22 +84,59 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
 
     crashed = true;
     window.onerror = undefined;
-    var html = ['<h1>you hit a bug!</h1><p style="font-size:1.3em;margin:0">try to <a href="#" onclick="localStorage.clear();location.reload();">reset copyparty settings</a> if you are stuck here, or <a href="#" onclick="ignex();">ignore this</a> / <a href="#" onclick="ignex(true);">ignore all</a></p><p>please send me a screenshot arigathanks gozaimuch: <code>ed/irc.rizon.net</code> or <code>ed#2644</code><br />&nbsp; (and if you can, press F12 and include the "Console" tab in the screenshot too)</p><p>',
+    var html = [
-        esc(url + ' @' + lineNo + ':' + columnNo), '<br />' + esc(String(msg)) + '</p>'];
+        '<h1>you hit a bug!</h1>',
+        '<p style="font-size:1.3em;margin:0">try to <a href="#" onclick="localStorage.clear();location.reload();">reset copyparty settings</a> if you are stuck here, or <a href="#" onclick="ignex();">ignore this</a> / <a href="#" onclick="ignex(true);">ignore all</a></p>',
+        '<p style="color:#fff">please send me a screenshot arigathanks gozaimuch: <a href="<ghi>" target="_blank">github issue</a> or <code>ed#2644</code></p>',
+        '<p class="b">' + esc(url + ' @' + lineNo + ':' + columnNo), '<br />' + esc(String(msg)) + '</p>',
+        '<p><b>UA:</b> ' + esc(navigator.userAgent + '')
+    ];
+
+    try {
+        var ua = '',
+            ad = navigator.userAgentData,
+            adb = ad.brands;
+
+        for (var a = 0; a < adb.length; a++)
+            if (!/Not.*A.*Brand/.exec(adb[a].brand))
+                ua += adb[a].brand + '/' + adb[a].version + ', ';
+        ua += ad.platform;
+
+        html.push('<br /><b>UAD:</b> ' + esc(ua.slice(0, 100)));
+    }
+    catch (e) { }
+    html.push('</p>');
 
     try {
         if (error) {
             var find = ['desc', 'stack', 'trace'];
             for (var a = 0; a < find.length; a++)
                 if (String(error[find[a]]) !== 'undefined')
-                    html.push('<h3>' + find[a] + '</h3>' +
+                    html.push('<p class="b"><b>' + find[a] + ':</b><br />' +
-                        esc(String(error[find[a]])).replace(/\n/g, '<br />\n'));
+                        esc(String(error[find[a]])).replace(/\n/g, '<br />\n') + '</p>');
         }
         ignexd[ekey] = true;
-        html.push('<h3>localStore</h3>' + esc(JSON.stringify(localStorage)));
+
+        var ls = jcp(localStorage);
+        if (ls.fman_clip)
+            ls.fman_clip = ls.fman_clip.length + ' items';
+
+        var lsk = Object.keys(ls);
+        lsk.sort();
+        html.push('<p class="b">');
+        for (var a = 0; a < lsk.length; a++)
+            html.push(' <b>' + esc(lsk[a]) + '</b> <code>' + esc(ls[lsk[a]]) + '</code> ');
+        html.push('</p>');
     }
     catch (e) { }
 
+    if (console.hist.length) {
+        html.push('<p class="b"><b>console:</b><ul><li>' + Date.now() + ' @</li>');
+        for (var a = console.hist.length - 1, aa = Math.max(0, console.hist.length - 20); a >= aa; a--)
+            html.push('<li>' + esc(console.hist[a]) + '</li>');
+        html.push('</ul>')
+    }
+
     try {
         var exbox = ebi('exbox');
         if (!exbox) {
@@ -63,10 +145,19 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
             document.body.appendChild(exbox);
 
             var s = mknod('style');
-            s.innerHTML = '#exbox{background:#333;color:#ddd;font-family:sans-serif;font-size:0.8em;padding:0 1em 1em 1em;z-index:80386;position:fixed;top:0;left:0;right:0;bottom:0;width:100%;height:100%} #exbox h1{margin:.5em 1em 0 0;padding:0} #exbox h3{border-top:1px solid #999;margin:1em 0 0 0} #exbox a{text-decoration:underline;color:#fc0} #exbox code{color:#bf7;background:#222;padding:.1em;margin:.2em;font-size:1.1em;font-family:monospace,monospace} #exbox *{line-height:1.5em}';
+            s.innerHTML = (
+                '#exbox{background:#333;color:#ddd;font-family:sans-serif;font-size:0.8em;padding:0 1em 1em 1em;z-index:80386;position:fixed;top:0;left:0;right:0;bottom:0;width:100%;height:100%;overflow:auto;width:calc(100% - 2em)} ' +
+                '#exbox,#exbox *{line-height:1.5em;overflow-wrap:break-word} ' +
+                '#exbox code{color:#bf7;background:#222;padding:.1em;margin:.2em;font-size:1.1em;font-family:monospace,monospace} ' +
+                '#exbox a{text-decoration:underline;color:#fc0} ' +
+                '#exbox h1{margin:.5em 1em 0 0;padding:0} ' +
+                '#exbox p.b{border-top:1px solid #999;margin:1em 0 0 0;font-size:1em} ' +
+                '#exbox ul, #exbox li {margin:0 0 0 .5em;padding:0} ' +
+                '#exbox b{color:#fff}'
+            );
             document.head.appendChild(s);
         }
-        exbox.innerHTML = html.join('\n');
+        exbox.innerHTML = basenames(html.join('\n')).replace(/<ghi>/, 'https://github.com/9001/copyparty/issues/new?labels=bug&template=bug_report.md');
         exbox.style.display = 'block';
     }
     catch (e) {
@@ -84,6 +175,9 @@ function ignex(all) {
 }
 
 
+function noop() { }
+
+
 function ctrl(e) {
     return e && (e.ctrlKey || e.metaKey);
 }
@@ -109,29 +203,40 @@ function ev(e) {
 
 
 // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/endsWith
-if (!String.prototype.endsWith) {
+if (!String.prototype.endsWith)
     String.prototype.endsWith = function (search, this_len) {
         if (this_len === undefined || this_len > this.length) {
             this_len = this.length;
         }
         return this.substring(this_len - search.length, this_len) === search;
     };
-}
+
-if (!String.startsWith) {
+if (!String.startsWith)
     String.prototype.startsWith = function (s, i) {
         i = i > 0 ? i | 0 : 0;
         return this.substring(i, i + s.length) === s;
     };
-}
+
-if (!Element.prototype.closest) {
+if (!String.trimEnd)
+    String.prototype.trimEnd = String.prototype.trimRight = function () {
+        return this.replace(/[ \t\r\n]+$/m, '');
+    };
+
+if (!Element.prototype.matches)
+    Element.prototype.matches =
+        Element.prototype.oMatchesSelector ||
+        Element.prototype.msMatchesSelector ||
+        Element.prototype.mozMatchesSelector ||
+        Element.prototype.webkitMatchesSelector;
+
+if (!Element.prototype.closest)
     Element.prototype.closest = function (s) {
         var el = this;
         do {
-            if (el.msMatchesSelector(s)) return el;
+            if (el.matches(s)) return el;
             el = el.parentElement || el.parentNode;
         } while (el !== null && el.nodeType === 1);
-    }
+    };
-}
 
 
 // https://stackoverflow.com/a/950146
@@ -142,7 +247,9 @@ function import_js(url, cb) {
     script.src = url;
     script.onload = cb;
     script.onerror = function () {
-        toast.err(0, 'Failed to load module:\n' + url);
+        var m = 'Failed to load module:\n' + url;
+        console.log(m);
+        toast.err(0, m);
     };
     head.appendChild(script);
 }
@@ -170,12 +277,37 @@ function crc32(str) {
 }
 
 
-function clmod(obj, cls, add) {
+function clmod(el, cls, add) {
-    var re = new RegExp('\\s*\\b' + cls + '\\s*\\b', 'g');
+    if (el.classList) {
+        var have = el.classList.contains(cls);
         if (add == 't')
-        add = !re.test(obj.className);
+            add = !have;
 
-    obj.className = obj.className.replace(re, ' ') + (add ? ' ' + cls : '');
+        if (add != have)
+            el.classList[add ? 'add' : 'remove'](cls);
+
+        return;
+    }
+
+    var re = new RegExp('\\s*\\b' + cls + '\\s*\\b', 'g'),
+        n1 = el.className;
+
+    if (add == 't')
+        add = !re.test(n1);
+
+    var n2 = n1.replace(re, ' ') + (add ? ' ' + cls : '');
+
+    if (n1 != n2)
+        el.className = n2;
+}
+
+
+function clgot(el, cls) {
+    if (el.classList)
+        return el.classList.contains(cls);
+
+    var lst = (el.getAttribute('class') + '').split(/ /g);
+    return has(lst, cls);
 }
 
 
@@ -253,8 +385,16 @@ function makeSortable(table, cb) {
 
 
 function linksplit(rp) {
-    var ret = [];
+    var ret = [],
-    var apath = '/';
+        apath = '/',
+        q = null;
+
+    if (rp && rp.indexOf('?') + 1) {
+        q = rp.split('?', 2);
+        rp = q[0];
+        q = '?' + q[1];
+    }
+
     if (rp && rp.charAt(0) == '/')
         rp = rp.slice(1);
 
@@ -268,16 +408,17 @@ function linksplit(rp) {
             link = rp.slice(0, ofs + 1);
             rp = rp.slice(ofs + 1);
         }
-        var vlink = esc(link),
+        var vlink = esc(uricom_dec(link)[0]);
-            elink = uricom_enc(link);
 
         if (link.indexOf('/') !== -1) {
             vlink = vlink.slice(0, -1) + '<span>/</span>';
-            elink = elink.slice(0, -3) + '/';
         }
 
-        ret.push('<a href="' + apath + elink + '">' + vlink + '</a>');
+        if (!rp && q)
-        apath += elink;
+            link += q;
+
+        ret.push('<a href="' + apath + link + '">' + vlink + '</a>');
+        apath += link;
     }
     return ret;
 }
@@ -308,6 +449,16 @@ function uricom_enc(txt, do_fb_enc) {
     }
 }
 
+function url_enc(txt) {
+    var parts = txt.split('/'),
+        ret = [];
+
+    for (var a = 0; a < parts.length; a++)
+        ret.push(uricom_enc(parts[a]));
+
+    return ret.join('/');
+}
+
 
 function uricom_dec(txt) {
     try {
@@ -320,10 +471,12 @@ function uricom_dec(txt) {
 }
 
 
-function uricom_adec(arr) {
+function uricom_adec(arr, li) {
     var ret = [];
-    for (var a = 0; a < arr.length; a++)
+    for (var a = 0; a < arr.length; a++) {
-        ret.push(uricom_dec(arr[a])[0]);
+        var txt = uricom_dec(arr[a])[0];
+        ret.push(li ? '<li>' + esc(txt) + '</li>' : txt);
+    }
 
     return ret;
 }
@@ -347,6 +500,11 @@ function get_vpath() {
 }
 
 
+function noq_href(el) {
+    return el.getAttribute('href').split('?')[0];
+}
+
+
 function get_pwd() {
     var pwd = ('; ' + document.cookie).split('; cppwd=');
     if (pwd.length < 2)
@@ -368,6 +526,41 @@ function s2ms(s) {
 }
 
 
+function f2f(val, nd) {
+    // 10.toFixed(1) returns 10.00 for certain values of 10
+    val = (val * Math.pow(10, nd)).toFixed(0).split('.')[0];
+    return nd ? (val.slice(0, -nd) || '0') + '.' + val.slice(-nd) : val;
+}
+
+
+function humansize(b, terse) {
+    var i = 0, u = terse ? ['B', 'K', 'M', 'G'] : ['B', 'KB', 'MB', 'GB'];
+    while (b >= 1000 && i < u.length) {
+        b /= 1024;
+        i += 1;
+    }
+    return f2f(b, b >= 100 ? 0 : b >= 10 ? 1 : 2) + ' ' + u[i];
+}
+
+
+function humantime(v) {
+    if (v >= 60 * 60 * 24)
+        return v;
+
+    try {
+        return /.*(..:..:..).*/.exec(new Date(v * 1000).toUTCString())[1];
+    }
+    catch (ex) {
+        return v;
+    }
+}
+
+
+function clamp(v, a, b) {
+    return Math.min(Math.max(v, a), b);
+}
+
+
 function has(haystack, needle) {
     for (var a = 0; a < haystack.length; a++)
         if (haystack[a] == needle)
@@ -465,6 +658,24 @@ function bcfg_upd_ui(name, val) {
     }
 }
 
+function bcfg_bind(obj, oname, cname, defval, cb, un_ev) {
+    var v = bcfg_get(cname, defval),
+        el = ebi(cname);
+
+    obj[oname] = v;
+    if (el)
+        el.onclick = function (e) {
+            if (un_ev !== false)
+                ev(e);
+
+            obj[oname] = bcfg_set(cname, !obj[oname]);
+            if (cb)
+                cb(obj[oname]);
+        };
+
+    return v;
+}
+
 
 function hist_push(url) {
     console.log("h-push " + url);
@@ -494,7 +705,10 @@ var timer = (function () {
         apop(r.q, fun);
     };
 
-    function impl() {
+    function doevents() {
+        if (crashed)
+            return;
+
         if (Date.now() - r.last < 69)
             return;
 
@@ -504,7 +718,7 @@ var timer = (function () {
 
         r.last = Date.now();
     }
-    setInterval(impl, 100);
+    setInterval(doevents, 100);
 
     return r;
 })();
@@ -521,11 +735,21 @@ var tt = (function () {
     r.tt.setAttribute('id', 'tt');
     document.body.appendChild(r.tt);
 
+    var prev = null;
+    r.cshow = function () {
+        if (this !== prev)
+            r.show.bind(this)();
+
+        prev = this;
+    };
+
     r.show = function () {
         if (r.skip) {
             r.skip = false;
             return;
         }
+        if (QS('body.bbox-open'))
+            return;
 
         var cfg = sread('tooltips');
         if (cfg !== null && cfg != '1')
@@ -538,54 +762,77 @@ var tt = (function () {
         r.el = this;
         var pos = this.getBoundingClientRect(),
             dir = this.getAttribute('ttd') || '',
-            left = pos.left < window.innerWidth / 2,
+            margin = parseFloat(this.getAttribute('ttm') || 0),
             top = pos.top < window.innerHeight / 2,
             big = this.className.indexOf(' ttb') !== -1;
 
         if (dir.indexOf('u') + 1) top = false;
         if (dir.indexOf('d') + 1) top = true;
-        if (dir.indexOf('l') + 1) left = false;
-        if (dir.indexOf('r') + 1) left = true;
 
         clmod(r.tt, 'b', big);
-        r.tt.style.top = top ? pos.bottom + 'px' : 'auto';
+        r.tt.style.left = '0';
-        r.tt.style.bottom = top ? 'auto' : (window.innerHeight - pos.top) + 'px';
+        r.tt.style.top = '0';
-        r.tt.style.left = left ? pos.left + 'px' : 'auto';
-        r.tt.style.right = left ? 'auto' : (window.innerWidth - pos.right) + 'px';
 
         r.tt.innerHTML = msg.replace(/\$N/g, "<br />");
         r.el.addEventListener('mouseleave', r.hide);
+        window.addEventListener('scroll', r.hide);
         clmod(r.tt, 'show', 1);
+
+        var tw = r.tt.offsetWidth,
+            x = pos.left + (pos.right - pos.left) / 2 - tw / 2;
+
+        if (x + tw >= window.innerWidth - 24)
+            x = window.innerWidth - tw - 24;
+
+        if (x < 0)
+            x = 12;
+
+        r.tt.style.left = x + 'px';
+        r.tt.style.top = top ? (margin + pos.bottom) + 'px' : 'auto';
+        r.tt.style.bottom = top ? 'auto' : (margin + window.innerHeight - pos.top) + 'px';
     };
 
     r.hide = function (e) {
         ev(e);
+        window.removeEventListener('scroll', r.hide);
         clmod(r.tt, 'show');
+        clmod(r.tt, 'b');
         if (r.el)
             r.el.removeEventListener('mouseleave', r.hide);
     };
 
     if (is_touch && IPHONE) {
         var f1 = r.show,
-            f2 = r.hide;
+            f2 = r.hide,
+            q = [];
+
+        // if an onclick-handler creates a new timer,
+        // iOS 13.1.2 delays the entire handler by up to 401ms,
+        // win by using a shared timer instead
+
+        timer.add(function () {
+            while (q.length && Date.now() >= q[0][0])
+                q.shift()[1]();
+        });
 
         r.show = function () {
-            setTimeout(f1.bind(this), 301);
+            q.push([Date.now() + 100, f1.bind(this)]);
         };
         r.hide = function () {
-            setTimeout(f2.bind(this), 301);
+            q.push([Date.now() + 100, f2.bind(this)]);
         };
     }
 
     r.tt.onclick = r.hide;
 
     r.att = function (ctr) {
-        var _show = r.en ? r.show : null,
+        var _cshow = r.en ? r.cshow : null,
+            _show = r.en ? r.show : null,
             _hide = r.en ? r.hide : null,
             o = ctr.querySelectorAll('*[tt]');
 
         for (var a = o.length - 1; a >= 0; a--) {
-            o[a].onfocus = _show;
+            o[a].onfocus = _cshow;
             o[a].onblur = _hide;
             o[a].onmouseenter = _show;
             o[a].onmouseleave = _hide;
@@ -624,43 +871,72 @@ function lf2br(txt) {
 var toast = (function () {
     var r = {},
         te = null,
+        scrolling = false,
         obj = mknod('div');
 
     obj.setAttribute('id', 'toast');
     document.body.appendChild(obj);
     r.visible = false;
+    r.txt = null;
+
+    function scrollchk() {
+        if (scrolling)
+            return;
+
+        var tb = ebi('toastb'),
+            vis = tb.offsetHeight,
+            all = tb.scrollHeight;
+
+        if (8 + vis >= all)
+            return;
+
+        clmod(obj, 'scroll', 1);
+        scrolling = true;
+    }
+
+    function unscroll() {
+        timer.rm(scrollchk);
+        clmod(obj, 'scroll');
+        scrolling = false;
+    }
 
     r.hide = function (e) {
         ev(e);
+        unscroll();
         clearTimeout(te);
         clmod(obj, 'vis');
         r.visible = false;
     };
 
-    r.show = function (cl, ms, txt) {
+    r.show = function (cl, sec, txt) {
         clearTimeout(te);
-        if (ms)
+        if (sec)
-            te = setTimeout(r.hide, ms * 1000);
+            te = setTimeout(r.hide, sec * 1000);
 
-        obj.innerHTML = '<a href="#" id="toastc">x</a>' + lf2br(txt);
+        if (txt.indexOf('<body>') + 1)
+            txt = txt.slice(0, txt.indexOf('<')) + ' [...]';
+
+        obj.innerHTML = '<a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
         obj.className = cl;
-        ms += obj.offsetWidth;
+        sec += obj.offsetWidth;
         obj.className += ' vis';
         ebi('toastc').onclick = r.hide;
+        timer.add(scrollchk);
         r.visible = true;
+        r.txt = txt;
     };
 
-    r.ok = function (ms, txt) {
+    r.ok = function (sec, txt) {
-        r.show('ok', ms, txt);
+        r.show('ok', sec, txt);
     };
-    r.inf = function (ms, txt) {
+    r.inf = function (sec, txt) {
-        r.show('inf', ms, txt);
+        r.show('inf', sec, txt);
     };
-    r.warn = function (ms, txt) {
+    r.warn = function (sec, txt) {
-        r.show('warn', ms, txt);
+        r.show('warn', sec, txt);
     };
-    r.err = function (ms, txt) {
+    r.err = function (sec, txt) {
-        r.show('err', ms, txt);
+        r.show('err', sec, txt);
     };
 
     return r;
@@ -671,8 +947,12 @@ var modal = (function () {
     var r = {},
         q = [],
         o = null,
+        cb_up = null,
         cb_ok = null,
-        cb_ng = null;
+        cb_ng = null,
+        prim = '<a href="#" id="modal-ok">OK</a>',
+        sec = '<a href="#" id="modal-ng">Cancel</a>',
+        ok_cancel = WINDOWS ? prim + sec : sec + prim;
 
     r.busy = false;
 
@@ -691,12 +971,24 @@ var modal = (function () {
         a = ebi('modal-ok');
         a.onclick = ok;
 
-        (ebi('modali') || a).focus();
+        var inp = ebi('modali');
+        (inp || a).focus();
+        if (inp)
+            setTimeout(function () {
+                inp.setSelectionRange(0, inp.value.length, "forward");
+            }, 0);
+
+        document.addEventListener('focus', onfocus);
+        timer.add(onfocus);
+        if (cb_up)
+            setTimeout(cb_up, 1);
     };
 
     r.hide = function () {
-        o.parentNode.removeChild(o);
+        timer.rm(onfocus);
+        document.removeEventListener('focus', onfocus);
         document.removeEventListener('keydown', onkey);
+        o.parentNode.removeChild(o);
         r.busy = false;
         setTimeout(next, 50);
     };
@@ -715,6 +1007,17 @@ var modal = (function () {
             cb_ng(null);
     }
 
+    function onfocus(e) {
+        var ctr = ebi('modalc');
+        if (!ctr || !ctr.contains || !document.activeElement || ctr.contains(document.activeElement))
+            return;
+
+        setTimeout(function () {
+            ebi('modal-ok').focus();
+        }, 20);
+        ev(e);
+    }
+
     function onkey(e) {
         if (e.code == 'Enter') {
             var a = ebi('modal-ng');
@@ -733,41 +1036,44 @@ var modal = (function () {
             q.shift()();
     }
 
-    r.alert = function (html, cb) {
+    r.alert = function (html, cb, fun) {
         q.push(function () {
-            _alert(lf2br(html), cb);
+            _alert(lf2br(html), cb, fun);
         });
         next();
     };
-    function _alert(html, cb) {
+    function _alert(html, cb, fun) {
         cb_ok = cb_ng = cb;
+        cb_up = fun;
         html += '<div id="modalb"><a href="#" id="modal-ok">OK</a></div>';
         r.show(html);
     }
 
-    r.confirm = function (html, cok, cng) {
+    r.confirm = function (html, cok, cng, fun) {
         q.push(function () {
-            _confirm(lf2br(html), cok, cng);
+            _confirm(lf2br(html), cok, cng, fun);
         });
         next();
     }
-    function _confirm(html, cok, cng) {
+    function _confirm(html, cok, cng, fun) {
         cb_ok = cok;
-        cb_ng = cng === undefined ? cok : null;
+        cb_ng = cng === undefined ? cok : cng;
-        html += '<div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        cb_up = fun;
+        html += '<div id="modalb">' + ok_cancel + '</div>';
         r.show(html);
     }
 
-    r.prompt = function (html, v, cok, cng) {
+    r.prompt = function (html, v, cok, cng, fun) {
         q.push(function () {
-            _prompt(lf2br(html), v, cok, cng);
+            _prompt(lf2br(html), v, cok, cng, fun);
         });
         next();
     }
-    function _prompt(html, v, cok, cng) {
+    function _prompt(html, v, cok, cng, fun) {
         cb_ok = cok;
         cb_ng = cng === undefined ? cok : null;
-        html += '<input id="modali" type="text" /><div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        cb_up = fun;
+        html += '<input id="modali" type="text" /><div id="modalb">' + ok_cancel + '</div>';
         r.show(html);
 
         ebi('modali').value = v || '';
@@ -775,3 +1081,95 @@ var modal = (function () {
 
     return r;
 })();
+
+
+function winpopup(txt) {
+    fetch(get_evpath(), {
+        method: 'POST',
+        headers: {
+            'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'
+        },
+        body: 'msg=' + uricom_enc(Date.now() + ', ' + txt)
+    });
+}
+
+
+var last_repl = null;
+function repl_load() {
+    var ipre = ebi('repl_pre'),
+        tb = ebi('modali');
+
+    function getpres() {
+        var o, ret = jread("repl_pre", []);
+        if (!ret.length)
+            ret = [
+                'var v=Object.keys(localStorage); v.sort(); JSON.stringify(v)',
+                'console.hist.slice(-10).join("\\n")'
+            ];
+
+        ipre.innerHTML = '<option value=""></option>';
+        for (var a = 0; a < ret.length; a++) {
+            o = mknod('option');
+            o.setAttribute('value', ret[a]);
+            o.textContent = ret[a];
+            ipre.appendChild(o);
+        }
+        last_repl = ipre.value = (last_repl || (ret.length ? ret.slice(-1)[0] : ''));
+        return ret;
+    }
+    ebi('repl_pdel').onclick = function (e) {
+        var val = ipre.value,
+            pres = getpres();
+
+        apop(pres, val);
+        jwrite('repl_pre', pres);
+        getpres();
+    };
+    ebi('repl_pnew').onclick = function (e) {
+        var val = tb.value,
+            pres = getpres();
+
+        apop(pres, ipre.value);
+        pres.push(val);
+        jwrite('repl_pre', pres);
+        getpres();
+        ipre.value = val;
+    };
+    ipre.oninput = ipre.onchange = function () {
+        tb.value = last_repl = ipre.value;
+    };
+    tb.oninput = function () {
+        last_repl = this.value;
+    };
+    getpres();
+    tb.value = last_repl;
+    setTimeout(function () {
+        tb.setSelectionRange(0, tb.value.length, "forward");
+    }, 10);
+}
+function repl(e) {
+    ev(e);
+    var html = [
+        '<p>js repl (prefix with <code>,</code> to allow raise)</p>',
+        '<p><select id="repl_pre"></select>',
+        ' &nbsp; <button id="repl_pdel">❌ del</button>',
+        ' &nbsp; <button id="repl_pnew">💾 SAVE</button></p>'
+    ];
+
+    modal.prompt(html.join(''), '', function (cmd) {
+        if (!cmd)
+            return toast.inf(3, 'eval aborted');
+
+        if (cmd.startsWith(','))
+            return modal.alert(esc(eval(cmd.slice(1)) + ''))
+
+        try {
+            modal.alert(esc(eval(cmd) + ''));
+        }
+        catch (ex) {
+            modal.alert('<h6>exception</h6>' + esc(ex + ''));
+        }
+    }, undefined, repl_load);
+}
+if (ebi('repl'))
+    ebi('repl').onclick = repl;

docs/README.md

@@ -1,11 +1,21 @@
-# example `.epilogue.html`
+**NOTE:** there's more stuff (sharex config, service scripts, nginx configs, ...) in [`/contrib/`](/contrib/)
+
+
+
+# example resource files
+
+can be provided to copyparty to tweak things
+
+
+
+## example `.epilogue.html`
 save one of these as `.epilogue.html` inside a folder to customize it:
 
 * [`minimal-up2k.html`](minimal-up2k.html) will [simplify the upload ui](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
 
 
 
-# example browser-css
+## example browser-css
 point `--css-browser` to one of these by URL:
 
 * [`browser.css`](browser.css) changes the background
@@ -19,4 +29,23 @@ point `--css-browser` to one of these by URL:
 * notes on using rclone as a fuse client/server
 
 ## [`example.conf`](example.conf)
-* example config file for `-c` which never really happened
+* example config file for `-c` (supports accounts, volumes, and volume-flags)
+
+
+
+# junk
+
+alphabetical list of the remaining files
+
+| what | why |
+| -- | -- |
+| [biquad.html](biquad.html) | bruteforce calibrator for the audio equalizer since im not that good at maths |
+| [design.txt](design.txt) | initial brainstorming of the copyparty design, unmaintained, incorrect, sentimental value only |
+| [hls.html](hls.html) | experimenting with hls playback using `hls.js`, works p well, almost became a thing |
+| [music-analysis.sh](music-analysis.sh) | testing various bpm/key detection libraries before settling on the ones used in [`/bin/mtag/`](/bin/mtag/) |
+| [notes.sh](notes.sh) | notepad, just scraps really |
+| [nuitka.txt](nuitka.txt) | how to build a copyparty exe using nuitka (not maintained) |
+| [pretend-youre-qnap.patch](pretend-youre-qnap.patch) | simulate a NAS which keeps returning old cached data even though you just modified the file yourself |
+| [tcp-debug.sh](tcp-debug.sh) | looks like this was to debug stuck tcp connections? |
+| [unirange.py](unirange.py) | uhh |
+| [up2k.txt](up2k.txt) | initial ideas for how up2k should work, another unmaintained sentimental-value-only thing |

docs/biquad.html

@@ -3,6 +3,24 @@
 setTimeout(location.reload.bind(location), 700);
 document.documentElement.scrollLeft = 0;
 
+var cali = (function() {
+    var ac = new AudioContext(),
+        fi = ac.createBiquadFilter(),
+        freqs = new Float32Array(1),
+        mag = new Float32Array(1),
+        phase = new Float32Array(1);
+
+    freqs[0] = 14000;
+    fi.type = 'peaking';
+    fi.frequency.value = 18000;
+    fi.Q.value = 0.8;
+    fi.gain.value = 1;
+    fi.getFrequencyResponse(freqs, mag, phase);
+
+    return mag[0];  // 1.0407 good, 1.0563 bad
+})(),
+    mp = cali < 1.05;
+
 var can = document.createElement('canvas'),
     cc = can.getContext('2d'),
     w = 2048,
@@ -28,12 +46,12 @@ var cfg = [ // hz, q, g
     [1000, 0.9, 1.1],
     [2000, 0.9, 1.105],
     [4000, 0.88, 1.05],
-    [8000 * 1.006, 0.73, 1.24],
+    [8000 * 1.006, 0.73, mp ? 1.24 : 1.2],
     //[16000 * 1.00, 0.5, 1.75],  // peak.v1
     //[16000 * 1.19, 0, 1.8]  // shelf.v1
-    [16000 * 0.89, 0.7, 1.26],  // peak
+    [16000 * 0.89, 0.7, mp ? 1.26 : 1.2],  // peak
-    [16000 * 1.13, 0.82, 1.09],  // peak
+    [16000 * 1.13, 0.82, mp ? 1.09 : 0.75],  // peak
-    [16000 * 1.205, 0, 1.9]  // shelf
+    [16000 * 1.205, 0, mp ? 1.9 : 1.85]  // shelf
 ];
 
 var freqs = new Float32Array(22000),

docs/browser-icons.css

@@ -1,37 +1,7 @@
-/* put filetype icons inline with text
+/* video, alternative 1:
-#ggrid>a>span:before,
+   top-left icon, just like the other formats
-#ggrid>a>span.dir:before {
+=======================================================================
-	display: inline;
-	line-height: 0;
-	font-size: 1.7em;
-	margin: -.7em .1em -.5em -.6em;
-}
-*/
 
-
-/* move folder icons top-left */
-#ggrid>a>span.dir:before {
-	content: initial;
-}
-#ggrid>a[href$="/"]:before {
-	content: '📂';
-}
-
-
-/* put filetype icons top-left */
-#ggrid>a:before {
-    display: block;
-    position: absolute;
-	padding: .3em 0;
-    margin: -.4em;
-    text-shadow: 0 0 .1em #000;
-	background: linear-gradient(135deg,rgba(255,255,255,0) 50%,rgba(255,255,255,0.2));
-	border-radius: .3em;
-    font-size: 2em;
-}
-
-
-/* video */
 #ggrid>a:is(
 [href$=".mkv"i],
 [href$=".mp4"i],
@@ -39,6 +9,40 @@
 ):before {
     content: '📺';
 }
+*/
+
+
+
+/* video, alternative 2:
+   play-icon in the middle of the thumbnail
+=======================================================================
+*/
+#ggrid>a:is(
+[href$=".mkv"i],
+[href$=".mp4"i],
+[href$=".webm"i],
+) {
+	position: relative;
+	overflow: hidden;
+}
+#ggrid>a:is(
+[href$=".mkv"i],
+[href$=".mp4"i],
+[href$=".webm"i],
+):before {
+    content: '▶';
+	opacity: .8;
+	margin: 0;
+	padding: 1em .5em 1em .7em;
+	border-radius: 9em;
+	line-height: 0;
+	color: #fff;
+	text-shadow: none;
+	background: rgba(0, 0, 0, 0.7);
+	left: calc(50% - 1em);
+	top: calc(50% - 1.4em);
+}
+
 
 
 /* audio */
@@ -54,6 +58,7 @@
 }
 
 
+
 /* image */
 #ggrid>a:is(
 [href$=".jpg"i],

docs/browser.css

@@ -17,6 +17,7 @@ html.light {
 html.light #files th {
     background: rgba(255, 255, 255, 0.9) !important;
 }
+html.light .logue,
 html.light #ops,
 html.light #treeul,
 html.light #files td {

docs/example.conf

@@ -47,5 +47,5 @@ c e2d
 c nodupe
 
 # this entire config file can be replaced with these arguments:
-# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d:c,nodupe
+# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d,nodupe
 # but note that the config file always wins in case of conflicts

docs/minimal-up2k.html

@@ -11,7 +11,9 @@
 
     #u2cleanup, #u2conf tr:first-child>td[rowspan]:not(#u2btn_cw),  /* most of the config options */
 
-    #u2cards  /* and the upload progress tabs */
+    #srch_dz, #srch_zd,  /* the filesearch dropzone */
+
+    #u2cards, #u2etaw  /* and the upload progress tabs */
 
     {display: none !important}  /* do it! */
 
@@ -19,7 +21,7 @@
 
     /* add some margins because now it's weird */
     .opview {margin-top: 2.5em}
-    #op_up2k {margin-top: 3em}
+    #op_up2k {margin-top: 6em}
 
     /* and embiggen the upload button */
     #u2conf #u2btn, #u2btn {padding:1.5em 0}
@@ -27,6 +29,9 @@
     /* adjust the button area a bit */
     #u2conf.has_btn {width: 35em !important; margin: 5em auto}
 
+    /* a */
+    #op_up2k {min-height: 0}
+
 </style>
 
 <a href="#" onclick="this.parentNode.innerHTML='';">show advanced options</a>

docs/mistakes.py

@@ -1,26 +0,0 @@
-
-                method = self.s.recv(4)
-                self.s.unrecv(method)
-                print("xxx unrecv'd [{}]".format(method))
-
-                # jython used to do this, they stopped since it's broken
-                # but reimplementing sendall is out of scope for now
-                if not getattr(self.s.s, "sendall", None):
-                    self.s.s.sendall = self.s.s.send
-
-                # TODO this is also pretty bad
-                have = dir(self.s)
-                for k in self.s.s.__dict__:
-                    if k not in have and not k.startswith("__"):
-                        if k == "recv":
-                            raise Exception("wait what")
-
-                        self.s.__dict__[k] = self.s.s.__dict__[k]
-
-                have = dir(self.s)
-                for k in dir(self.s.s):
-                    if k not in have and not k.startswith("__"):
-                        if k == "recv":
-                            raise Exception("wait what")
-
-                        setattr(self.s, k, getattr(self.s.s, k))

docs/notes.sh

@@ -41,9 +41,9 @@ avg() { awk 'function pr(ncsz) {if (nsmp>0) {printf "%3s %s\n", csz, sum/nsmp} c
 ##
 ## bad filenames
 
-dirs=("$HOME/vfs/ほげ" "$HOME/vfs/ほげ/ぴよ" "$HOME/vfs/$(printf \\xed\\x91)" "$HOME/vfs/$(printf \\xed\\x91/\\xed\\x92)")
+dirs=("./ほげ" "./ほげ/ぴよ" "./$(printf \\xed\\x91)" "./$(printf \\xed\\x91/\\xed\\x92)" './qw,er;ty%20as df?gh+jkl%zxc&vbn <qwe>"rty'"'"'uio&asd&nbsp;fgh')
 mkdir -p "${dirs[@]}"
-for dir in "${dirs[@]}"; do for fn in ふが "$(printf \\xed\\x93)" 'qwe,rty;asd fgh+jkl%zxc&vbn <qwe>"rty'"'"'uio&asd&nbsp;fgh'; do echo "$dir" > "$dir/$fn.html"; done; done
+for dir in "${dirs[@]}"; do for fn in ふが "$(printf \\xed\\x93)" 'qw,er;ty%20as df?gh+jkl%zxc&vbn <qwe>"rty'"'"'uio&asd&nbsp;fgh'; do echo "$dir" > "$dir/$fn.html"; done; done
 # qw er+ty%20ui%%20op<as>df&gh&amp;jk#zx'cv"bn`m=qw*er^ty?ui@op,as.df-gh_jk
 
 ##
@@ -79,10 +79,8 @@ command -v gdate && date() { gdate "$@"; }; while true; do t=$(date +%s.%N); (ti
 # get all up2k search result URLs
 var t=[]; var b=document.location.href.split('#')[0].slice(0, -1); document.querySelectorAll('#u2tab .prog a').forEach((x) => {t.push(b+encodeURI(x.getAttribute("href")))}); console.log(t.join("\n"));
 
-# rename all selected songs to <leading-track-number> + <Title> + <extension>
+# debug md-editor line tracking
-var sel=msel.getsel(), ci=find_file_col('Title')[0], re=[]; for (var a=0; a<sel.length; a++) { var url=sel[a].vp, tag=ebi(sel[a].id).closest('tr').querySelectorAll('td')[ci].textContent, name=uricom_dec(vsplit(url)[1])[0], m=/^([0-9]+[\. -]+)?.*(\.[^\.]+$)/.exec(name), name2=(m[1]||'')+tag+m[2], url2=vsplit(url)[0]+uricom_enc(name2,false); if (url!=url2) re.push([url, url2]); }
+var s=mknod('style');s.innerHTML='*[data-ln]:before {content:attr(data-ln)!important;color:#f0c;background:#000;position:absolute;left:-1.5em;font-size:1rem}';document.head.appendChild(s);
-console.log(JSON.stringify(re, null, '  '));
-function f() { if (!re.length) return treectl.goto(get_evpath()); var [u1,u2] = re.shift(); fetch(u1+'?move='+u2).then((rsp) => {if (rsp.ok) f(); }); }; f();
 
 ##
 ## bash oneliners
@@ -126,6 +124,13 @@ e=6; s=10; d=~/dev/copyparty/srv/aus; n=1; p=0; e=$((e*60)); rm -rf $d; mkdir $d
 -v srv/aus:aus:r:ce2dsa:ce2ts:cmtp=fgsfds=bin/mtag/sleep.py
 sqlite3 .hist/up2k.db 'select * from mt where k="fgsfds" or k="t:mtp"' | tee /dev/stderr | wc -l
 
+# generate the sine meme
+for ((f=420;f<1200;f++)); do sz=$(ffmpeg -y -f lavfi -i sine=frequency=$f:duration=2 -vf volume=0.1 -ac 1 -ar 44100 -f s16le /dev/shm/a.wav 2>/dev/null; base64 -w0 </dev/shm/a.wav | gzip -c | wc -c); printf '%d %d\n' $f $sz; done | tee /dev/stderr | sort -nrk2,2
+ffmpeg -y -f lavfi -i sine=frequency=1050:duration=2 -vf volume=0.1 -ac 1 -ar 44100 /dev/shm/a.wav
+
+# play icon calibration pics
+for w in 150 170 190 210 230 250; do for h in 130 150 170 190 210; do /c/Program\ Files/ImageMagick-7.0.11-Q16-HDRI/magick.exe convert -size ${w}x${h} xc:brown -fill orange -draw "circle $((w/2)),$((h/2)) $((w/2)),$((h/3))" $w-$h.png; done; done
+
 
 ##
 ## vscode
@@ -157,7 +162,7 @@ brew install python@2
 pip install virtualenv
 
 # readme toc
-cat README.md | awk '!/^#/{next} {lv=length($1);sub(/[^ ]+ /,"");bab=$0;gsub(/ /,"-",bab)} {printf "%" ((lv-1)*4+1) "s [%s](#%s)\n", "*",$0,bab}'
+cat README.md | awk 'function pr() { if (!h) {return}; if (/^ *[*!#]/||!s) {printf "%s\n",h;h=0;return}; if (/.../) {printf "%s - %s\n",h,$0;h=0}; };    /^#/{s=1;pr()} /^#* *(file indexing|install on android|dev env setup|just the sfx|complete release|optional gpl stuff)|`$/{s=0} /^#/{lv=length($1);sub(/[^ ]+ /,"");bab=$0;gsub(/ /,"-",bab); h=sprintf("%" ((lv-1)*4+1) "s [%s](#%s)", "*",$0,bab);next} !h{next} {sub(/  .*/,"");sub(/[:,]$/,"")} {pr()}' > toc; grep -E '^## readme toc' -B1000 -A2 <README.md >p1; grep -E '^## quickstart' -B2 -A999999 <README.md >p2; (cat p1; grep quickstart -A1000 <toc; cat p2) >README.md; rm p1 p2 toc
 
 # fix firefox phantom breakpoints,
 # suggestions from bugtracker, doesnt work (debugger is not attachable)
@@ -173,7 +178,7 @@ about:config >> devtools.debugger.prefs-schema-version = -1
 git pull; git reset --hard origin/HEAD && git log --format=format:"%H %ai %d" --decorate=full > ../revs && cat ../{util,browser,up2k}.js >../vr && cat ../revs | while read -r rev extra; do (git reset --hard $rev >/dev/null 2>/dev/null && dsz=$(cat copyparty/web/{util,browser,up2k}.js >../vg 2>/dev/null && diff -wNarU0 ../{vg,vr} | wc -c) && printf '%s %6s %s\n' "$rev" $dsz "$extra") </dev/null; done                
 
 # download all sfx versions
-curl https://api.github.com/repos/9001/copyparty/releases?per_page=100 | jq -r '.[] | .tag_name + " " + .name' | while read v t; do fn="copyparty $v $t.py"; [ -e $fn ] || curl https://github.com/9001/copyparty/releases/download/$v/copyparty-sfx.py -Lo "$fn"; done
+curl https://api.github.com/repos/9001/copyparty/releases?per_page=100 | jq -r '.[] | .tag_name + " " + .name' | tr -d '\r' | while read v t; do fn="copyparty $v $t.py"; [ -e "$fn" ] || curl https://github.com/9001/copyparty/releases/download/$v/copyparty-sfx.py -Lo "$fn"; done
 
 
 ##

scripts/deps-docker/Dockerfile

@@ -1,11 +1,11 @@
-FROM    alpine:3.13
+FROM    alpine:3.14
 WORKDIR /z
 ENV     ver_asmcrypto=5b994303a9d3e27e0915f72a10b6c2c51535a4dc \
-        ver_hashwasm=4.7.0 \
+        ver_hashwasm=4.9.0 \
-        ver_marked=1.1.0 \
+        ver_marked=3.0.4 \
-        ver_ogvjs=1.8.0 \
+        ver_ogvjs=1.8.4 \
-        ver_mde=2.14.0 \
+        ver_mde=2.15.0 \
-        ver_codemirror=5.59.3 \
+        ver_codemirror=5.62.3 \
         ver_fontawesome=5.13.0 \
         ver_zopfli=1.0.3
 
@@ -74,23 +74,16 @@ RUN     cd hash-wasm \
 # build ogvjs
 RUN     cd ogvjs-$ver_ogvjs \
         && cp -pv \
-            ogv.js \
             ogv-worker-audio.js \
             ogv-demuxer-ogg-wasm.js \
             ogv-demuxer-ogg-wasm.wasm \
-            ogv-demuxer-webm-wasm.js \
-            ogv-demuxer-webm-wasm.wasm \
             ogv-decoder-audio-opus-wasm.js \
             ogv-decoder-audio-opus-wasm.wasm \
             ogv-decoder-audio-vorbis-wasm.js \
             ogv-decoder-audio-vorbis-wasm.wasm \
-            /z/dist
+            /z/dist \
-
+        && cp -pv \
-#            ogv-demuxer-ogg.js \
+            ogv-es2017.js /z/dist/ogv.js
-#            ogv-demuxer-webm.js \
-#            ogv-decoder-audio-opus.js \
-#            ogv-decoder-audio-vorbis.js \
-#            dynamicaudio.swf \
 
 
 # build marked
@@ -120,9 +113,10 @@ RUN     cd CodeMirror-$ver_codemirror \
 COPY    easymde.patch /z/
 RUN     cd easy-markdown-editor-$ver_mde \
         && patch -p1 < /z/easymde.patch \
-        && sed -ri 's`https://registry.npmjs.org/marked/-/marked-0.8.2.tgz`file:/z/nodepkgs/marked`' package-lock.json \
+        && sed -ri 's`https://registry.npmjs.org/marked/-/marked-[0-9\.]+.tgz`file:/z/nodepkgs/marked`' package-lock.json \
         && sed -ri 's`("marked": ")[^"]+`\1file:/z/nodepkgs/marked`' ./package.json \
         && sed -ri 's`("codemirror": ")[^"]+`\1file:/z/nodepkgs/codemirror`' ./package.json \
+        && sed -ri 's`^var marked = require\(.marked/lib/marked.\);$`var marked = window.marked;`' src/js/easymde.js \
         && npm install
 
 COPY    easymde-ln.patch /z/

scripts/deps-docker/marked-ln.patch

@@ -1,15 +1,15 @@
 diff --git a/src/Lexer.js b/src/Lexer.js
-adds linetracking to marked.js v1.0.0 +git;
+adds linetracking to marked.js v3.0.4;
 add data-ln="%d" to most tags, %d is the source markdown line
 --- a/src/Lexer.js
 +++ b/src/Lexer.js
-@@ -49,4 +49,5 @@ function mangle(text) {
+@@ -50,4 +50,5 @@ function mangle(text) {
  module.exports = class Lexer {
    constructor(options) {
 +    this.ln = 1;  // like most editors, start couting from 1
      this.tokens = [];
      this.tokens.links = Object.create(null);
-@@ -108,4 +109,15 @@ module.exports = class Lexer {
+@@ -127,4 +128,15 @@ module.exports = class Lexer {
    }
  
 +  set_ln(token, ln = this.ln) {
@@ -25,122 +25,123 @@ add data-ln="%d" to most tags, %d is the source markdown line
 +
    /**
     * Lexing
-@@ -113,10 +125,15 @@ module.exports = class Lexer {
+@@ -134,7 +146,11 @@ module.exports = class Lexer {
-   blockTokens(src, tokens = [], top = true) {
        src = src.replace(/^ +$/gm, '');
--    let token, i, l, lastToken;
+     }
-+    let token, i, l, lastToken, ln;
+-    let token, lastToken, cutSrc, lastParagraphClipped;
++    let token, lastToken, cutSrc, lastParagraphClipped, ln;
  
      while (src) {
 +      // this.ln will be bumped by recursive calls into this func;
 +      // reset the count and rely on the outermost token's raw only
 +      ln = this.ln;
 +
-       // newline
+       if (this.options.extensions
+         && this.options.extensions.block
+@@ -142,4 +158,5 @@ module.exports = class Lexer {
+           if (token = extTokenizer.call({ lexer: this }, src, tokens)) {
+             src = src.substring(token.raw.length);
++            this.set_ln(token, ln);
+             tokens.push(token);
+             return true;
+@@ -153,4 +170,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.space(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token); // is \n if not type
++        this.set_ln(token, ln); // is \n if not type
          if (token.type) {
            tokens.push(token);
-@@ -128,4 +145,5 @@ module.exports = class Lexer {
+@@ -162,4 +180,5 @@ module.exports = class Lexer {
-       if (token = this.tokenizer.code(src, tokens)) {
+       if (token = this.tokenizer.code(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
-         if (token.type) {
+         lastToken = tokens[tokens.length - 1];
-           tokens.push(token);
+         // An indented code block cannot interrupt a paragraph.
-@@ -141,4 +159,5 @@ module.exports = class Lexer {
+@@ -177,4 +196,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.fences(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -148,4 +167,5 @@ module.exports = class Lexer {
+@@ -184,4 +204,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.heading(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -155,4 +175,5 @@ module.exports = class Lexer {
+@@ -191,4 +212,5 @@ module.exports = class Lexer {
-       if (token = this.tokenizer.nptable(src)) {
-         src = src.substring(token.raw.length);
-+        this.set_ln(token);
-         tokens.push(token);
-         continue;
-@@ -162,4 +183,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.hr(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -170,4 +192,7 @@ module.exports = class Lexer {
+@@ -198,4 +220,5 @@ module.exports = class Lexer {
+       if (token = this.tokenizer.blockquote(src)) {
          src = src.substring(token.raw.length);
-         token.tokens = this.blockTokens(token.text, [], top);
-+        // recursive call to blockTokens probably bumped this.ln,
-+        // token.raw is more reliable so reset this.ln and use that
 +        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -180,5 +205,9 @@ module.exports = class Lexer {
+@@ -205,4 +228,5 @@ module.exports = class Lexer {
-         for (i = 0; i < l; i++) {
+       if (token = this.tokenizer.list(src)) {
-           token.items[i].tokens = this.blockTokens(token.items[i].text, [], false);
+         src = src.substring(token.raw.length);
-+          // list entries don't bump the linecounter, so let's
-+          this.ln++;
-         }
-+        // then reset like blockquote
 +        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -188,4 +217,5 @@ module.exports = class Lexer {
+@@ -212,4 +236,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.html(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -195,4 +225,5 @@ module.exports = class Lexer {
+@@ -219,4 +244,5 @@ module.exports = class Lexer {
-       if (top && (token = this.tokenizer.def(src))) {
+       if (token = this.tokenizer.def(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
-         if (!this.tokens.links[token.tag]) {
+         lastToken = tokens[tokens.length - 1];
-           this.tokens.links[token.tag] = {
+         if (lastToken && (lastToken.type === 'paragraph' || lastToken.type === 'text')) {
-@@ -207,4 +238,5 @@ module.exports = class Lexer {
+@@ -236,4 +262,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.table(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -214,4 +246,5 @@ module.exports = class Lexer {
+@@ -243,4 +270,5 @@ module.exports = class Lexer {
        if (token = this.tokenizer.lheading(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
          tokens.push(token);
          continue;
-@@ -221,4 +254,5 @@ module.exports = class Lexer {
+@@ -263,4 +291,5 @@ module.exports = class Lexer {
-       if (top && (token = this.tokenizer.paragraph(src))) {
+       }
+       if (this.state.top && (token = this.tokenizer.paragraph(cutSrc))) {
++        this.set_ln(token, ln);
+         lastToken = tokens[tokens.length - 1];
+         if (lastParagraphClipped && lastToken.type === 'paragraph') {
+@@ -280,4 +309,6 @@ module.exports = class Lexer {
+       if (token = this.tokenizer.text(src)) {
          src = src.substring(token.raw.length);
-+        this.set_ln(token);
++        this.set_ln(token, ln);
-         tokens.push(token);
++        this.ln++;
-         continue;
+         lastToken = tokens[tokens.length - 1];
-@@ -228,4 +262,5 @@ module.exports = class Lexer {
+         if (lastToken && lastToken.type === 'text') {
-       if (token = this.tokenizer.text(src, tokens)) {
+@@ -355,4 +386,5 @@ module.exports = class Lexer {
+           if (token = extTokenizer.call({ lexer: this }, src, tokens)) {
              src = src.substring(token.raw.length);
-+        this.set_ln(token);
-         if (token.type) {
-           tokens.push(token);
-@@ -263,4 +298,7 @@ module.exports = class Lexer {
-     for (i = 0; i < l; i++) {
-       token = tokens[i];
-+      // this.ln is at EOF when inline() is invoked;
-+      // all this affects <br> tags only so no biggie if it breaks
 +            this.ln = token.ln || this.ln;
-       switch (token.type) {
+             tokens.push(token);
-         case 'paragraph':
+             return true;
-@@ -386,4 +424,6 @@ module.exports = class Lexer {
+@@ -420,4 +452,6 @@ module.exports = class Lexer {
        if (token = this.tokenizer.br(src)) {
          src = src.substring(token.raw.length);
 +        // no need to reset (no more blockTokens anyways)
 +        token.ln = this.ln++;
          tokens.push(token);
          continue;
+@@ -462,4 +496,5 @@ module.exports = class Lexer {
+       if (token = this.tokenizer.inlineText(cutSrc, smartypants)) {
+         src = src.substring(token.raw.length);
++        this.ln = token.ln || this.ln;
+         if (token.raw.slice(-1) !== '_') { // Track prevChar before string of ____ started
+           prevChar = token.raw.slice(-1);
 diff --git a/src/Parser.js b/src/Parser.js
 --- a/src/Parser.js
 +++ b/src/Parser.js
@@ -150,17 +151,16 @@ diff --git a/src/Parser.js b/src/Parser.js
 +    this.ln = 0; // error indicator; should always be set >=1 from tokens
    }
  
-@@ -55,4 +56,9 @@ module.exports = class Parser {
+@@ -64,4 +65,8 @@ module.exports = class Parser {
      for (i = 0; i < l; i++) {
        token = tokens[i];
 +      // take line-numbers from tokens whenever possible
 +      // and update the renderer's html attribute with the new value
 +      this.ln = token.ln || this.ln;
 +      this.renderer.tag_ln(this.ln);
-+
+ 
-       switch (token.type) {
+       // Run any renderer extensions
-         case 'space': {
+@@ -124,7 +129,10 @@ module.exports = class Parser {
-@@ -105,7 +111,10 @@ module.exports = class Parser {
              }
  
 -            body += this.renderer.tablerow(cell);
@@ -173,7 +173,7 @@ diff --git a/src/Parser.js b/src/Parser.js
 +          out += this.renderer.tag_ln(token.ln).table(header, body);
            continue;
          }
-@@ -148,8 +157,12 @@ module.exports = class Parser {
+@@ -167,8 +175,12 @@ module.exports = class Parser {
  
              itemBody += this.parse(item.tokens, loose);
 -            body += this.renderer.listitem(itemBody, task, checked);
@@ -188,7 +188,7 @@ diff --git a/src/Parser.js b/src/Parser.js
 +          out += this.renderer.tag_ln(token.ln).list(body, ordered, start);
            continue;
          }
-@@ -160,5 +173,6 @@ module.exports = class Parser {
+@@ -179,5 +191,6 @@ module.exports = class Parser {
          }
          case 'paragraph': {
 -          out += this.renderer.paragraph(this.parseInline(token.tokens));
@@ -196,22 +196,14 @@ diff --git a/src/Parser.js b/src/Parser.js
 +          out += this.renderer.tag_ln(token.ln).paragraph(t);
            continue;
          }
-@@ -199,4 +213,6 @@ module.exports = class Parser {
+@@ -221,4 +234,7 @@ module.exports = class Parser {
-     for (i = 0; i < l; i++) {
        token = tokens[i];
+ 
 +      // another thing that only affects <br/> and other inlines
 +      this.ln = token.ln || this.ln;
-       switch (token.type) {
++
-         case 'escape': {
+       // Run any renderer extensions
-@@ -229,5 +245,7 @@ module.exports = class Parser {
+       if (this.options.extensions && this.options.extensions.renderers && this.options.extensions.renderers[token.type]) {
-         }
-         case 'br': {
--          out += renderer.br();
-+          // update the html attribute before writing each <br/>,
-+          // don't care about the others
-+          out += renderer.tag_ln(this.ln).br();
-           break;
-         }
 diff --git a/src/Renderer.js b/src/Renderer.js
 --- a/src/Renderer.js
 +++ b/src/Renderer.js
@@ -228,7 +220,7 @@ diff --git a/src/Renderer.js b/src/Renderer.js
 +  
    code(code, infostring, escaped) {
      const lang = (infostring || '').match(/\S*/)[0];
-@@ -24,10 +30,10 @@ module.exports = class Renderer {
+@@ -26,10 +32,10 @@ module.exports = class Renderer {
  
      if (!lang) {
 -      return '<pre><code>'
@@ -241,58 +233,69 @@ diff --git a/src/Renderer.js b/src/Renderer.js
 +    return '<pre' + this.ln + '><code class="'
        + this.options.langPrefix
        + escape(lang, true)
-@@ -38,5 +44,5 @@ module.exports = class Renderer {
+@@ -40,5 +46,5 @@ module.exports = class Renderer {
  
    blockquote(quote) {
 -    return '<blockquote>\n' + quote + '</blockquote>\n';
 +    return '<blockquote' + this.ln + '>\n' + quote + '</blockquote>\n';
    }
  
-@@ -49,4 +55,5 @@ module.exports = class Renderer {
+@@ -51,4 +57,5 @@ module.exports = class Renderer {
        return '<h'
          + level
 +        + this.ln
          + ' id="'
          + this.options.headerPrefix
-@@ -59,5 +66,5 @@ module.exports = class Renderer {
+@@ -61,5 +68,5 @@ module.exports = class Renderer {
      }
      // ignore IDs
 -    return '<h' + level + '>' + text + '</h' + level + '>\n';
 +    return '<h' + level + this.ln + '>' + text + '</h' + level + '>\n';
    }
  
-@@ -73,5 +80,5 @@ module.exports = class Renderer {
+@@ -75,5 +82,5 @@ module.exports = class Renderer {
  
    listitem(text) {
 -    return '<li>' + text + '</li>\n';
 +    return '<li' + this.ln + '>' + text + '</li>\n';
    }
  
-@@ -85,5 +92,5 @@ module.exports = class Renderer {
+@@ -87,5 +94,5 @@ module.exports = class Renderer {
  
    paragraph(text) {
 -    return '<p>' + text + '</p>\n';
 +    return '<p' + this.ln + '>' + text + '</p>\n';
    }
  
-@@ -100,5 +107,5 @@ module.exports = class Renderer {
+@@ -102,5 +109,5 @@ module.exports = class Renderer {
  
    tablerow(content) {
 -    return '<tr>\n' + content + '</tr>\n';
 +    return '<tr' + this.ln + '>\n' + content + '</tr>\n';
    }
  
-@@ -125,5 +132,5 @@ module.exports = class Renderer {
+@@ -127,5 +134,5 @@ module.exports = class Renderer {
  
    br() {
 -    return this.options.xhtml ? '<br/>' : '<br>';
 +    return this.options.xhtml ? '<br' + this.ln + '/>' : '<br' + this.ln + '>';
    }
  
-@@ -151,5 +158,5 @@ module.exports = class Renderer {
+@@ -153,5 +160,5 @@ module.exports = class Renderer {
      }
  
 -    let out = '<img src="' + href + '" alt="' + text + '"';
 +    let out = '<img' + this.ln + ' src="' + href + '" alt="' + text + '"';
      if (title) {
        out += ' title="' + title + '"';
+diff --git a/src/Tokenizer.js b/src/Tokenizer.js
+--- a/src/Tokenizer.js
++++ b/src/Tokenizer.js
+@@ -301,4 +301,7 @@ module.exports = class Tokenizer {
+       const l = list.items.length;
+ 
++      // each nested list gets +1 ahead; this hack makes every listgroup -1 but atleast it doesn't get infinitely bad
++      this.lexer.ln--;
++
+       // Item child tokens handled here at end because we needed to have the final item to trim it first
+       for (i = 0; i < l; i++) {

scripts/deps-docker/marked.patch

@@ -1,52 +1,52 @@
 diff --git a/src/Lexer.js b/src/Lexer.js
 --- a/src/Lexer.js
 +++ b/src/Lexer.js
-@@ -5,5 +5,5 @@ const { block, inline } = require('./rules.js');
+@@ -6,5 +6,5 @@ const { repeatString } = require('./helpers.js');
  /**
   * smartypants text replacement
 - */
 + *
  function smartypants(text) {
    return text
-@@ -26,5 +26,5 @@ function smartypants(text) {
+@@ -27,5 +27,5 @@ function smartypants(text) {
  /**
   * mangle email addresses
 - */
 + *
  function mangle(text) {
    let out = '',
-@@ -439,5 +439,5 @@ module.exports = class Lexer {
+@@ -465,5 +465,5 @@ module.exports = class Lexer {
  
        // autolink
 -      if (token = this.tokenizer.autolink(src, mangle)) {
 +      if (token = this.tokenizer.autolink(src)) {
          src = src.substring(token.raw.length);
          tokens.push(token);
-@@ -446,5 +446,5 @@ module.exports = class Lexer {
+@@ -472,5 +472,5 @@ module.exports = class Lexer {
  
        // url (gfm)
--      if (!inLink && (token = this.tokenizer.url(src, mangle))) {
+-      if (!this.state.inLink && (token = this.tokenizer.url(src, mangle))) {
-+      if (!inLink && (token = this.tokenizer.url(src))) {
++      if (!this.state.inLink && (token = this.tokenizer.url(src))) {
          src = src.substring(token.raw.length);
          tokens.push(token);
-@@ -453,5 +453,5 @@ module.exports = class Lexer {
+@@ -493,5 +493,5 @@ module.exports = class Lexer {
- 
+         }
-       // text
+       }
--      if (token = this.tokenizer.inlineText(src, inRawBlock, smartypants)) {
+-      if (token = this.tokenizer.inlineText(cutSrc, smartypants)) {
-+      if (token = this.tokenizer.inlineText(src, inRawBlock)) {
++      if (token = this.tokenizer.inlineText(cutSrc)) {
          src = src.substring(token.raw.length);
-         tokens.push(token);
+         this.ln = token.ln || this.ln;
 diff --git a/src/Renderer.js b/src/Renderer.js
 --- a/src/Renderer.js
 +++ b/src/Renderer.js
-@@ -140,5 +140,5 @@ module.exports = class Renderer {
+@@ -142,5 +142,5 @@ module.exports = class Renderer {
  
    link(href, title, text) {
 -    href = cleanUrl(this.options.sanitize, this.options.baseUrl, href);
 +    href = cleanUrl(this.options.baseUrl, href);
      if (href === null) {
        return text;
-@@ -153,5 +153,5 @@ module.exports = class Renderer {
+@@ -155,5 +155,5 @@ module.exports = class Renderer {
  
    image(href, title, text) {
 -    href = cleanUrl(this.options.sanitize, this.options.baseUrl, href);
@@ -56,22 +56,23 @@ diff --git a/src/Renderer.js b/src/Renderer.js
 diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 --- a/src/Tokenizer.js
 +++ b/src/Tokenizer.js
-@@ -287,11 +287,8 @@ module.exports = class Tokenizer {
+@@ -321,14 +321,7 @@ module.exports = class Tokenizer {
-     if (cap) {
+         type: 'html',
-       return {
--        type: this.options.sanitize
--          ? 'paragraph'
--          : 'html',
-+        type: 'html',
          raw: cap[0],
 -        pre: !this.options.sanitizer
 -          && (cap[1] === 'pre' || cap[1] === 'script' || cap[1] === 'style'),
--        text: this.options.sanitize ? (this.options.sanitizer ? this.options.sanitizer(cap[0]) : escape(cap[0])) : cap[0]
++        pre: (cap[1] === 'pre' || cap[1] === 'script' || cap[1] === 'style'),
-+        pre: cap[1] === 'pre' || cap[1] === 'script' || cap[1] === 'style',
+         text: cap[0]
-+        text: cap[0]
        };
+-      if (this.options.sanitize) {
+-        token.type = 'paragraph';
+-        token.text = this.options.sanitizer ? this.options.sanitizer(cap[0]) : escape(cap[0]);
+-        token.tokens = [];
+-        this.lexer.inline(token.text, token.tokens);
+-      }
+       return token;
      }
-@@ -421,15 +418,9 @@ module.exports = class Tokenizer {
+@@ -477,15 +470,9 @@ module.exports = class Tokenizer {
  
        return {
 -        type: this.options.sanitize
@@ -79,8 +80,8 @@ diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 -          : 'html',
 +        type: 'html',
          raw: cap[0],
-         inLink,
+         inLink: this.lexer.state.inLink,
-         inRawBlock,
+         inRawBlock: this.lexer.state.inRawBlock,
 -        text: this.options.sanitize
 -          ? (this.options.sanitizer
 -            ? this.options.sanitizer(cap[0])
@@ -89,7 +90,7 @@ diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 +        text: cap[0]
        };
      }
-@@ -550,10 +541,10 @@ module.exports = class Tokenizer {
+@@ -672,10 +659,10 @@ module.exports = class Tokenizer {
    }
  
 -  autolink(src, mangle) {
@@ -102,7 +103,7 @@ diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 +        text = escape(cap[1]);
          href = 'mailto:' + text;
        } else {
-@@ -578,10 +569,10 @@ module.exports = class Tokenizer {
+@@ -700,10 +687,10 @@ module.exports = class Tokenizer {
    }
  
 -  url(src, mangle) {
@@ -115,15 +116,15 @@ diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 +        text = escape(cap[0]);
          href = 'mailto:' + text;
        } else {
-@@ -615,12 +606,12 @@ module.exports = class Tokenizer {
+@@ -737,12 +724,12 @@ module.exports = class Tokenizer {
    }
  
--  inlineText(src, inRawBlock, smartypants) {
+-  inlineText(src, smartypants) {
-+  inlineText(src, inRawBlock) {
++  inlineText(src) {
      const cap = this.rules.inline.text.exec(src);
      if (cap) {
        let text;
-       if (inRawBlock) {
+       if (this.lexer.state.inRawBlock) {
 -        text = this.options.sanitize ? (this.options.sanitizer ? this.options.sanitizer(cap[0]) : escape(cap[0])) : cap[0];
 +        text = cap[0];
        } else {
@@ -134,7 +135,7 @@ diff --git a/src/Tokenizer.js b/src/Tokenizer.js
 diff --git a/src/defaults.js b/src/defaults.js
 --- a/src/defaults.js
 +++ b/src/defaults.js
-@@ -8,12 +8,8 @@ function getDefaults() {
+@@ -9,12 +9,8 @@ function getDefaults() {
      highlight: null,
      langPrefix: 'language-',
 -    mangle: true,
@@ -170,7 +171,7 @@ diff --git a/src/helpers.js b/src/helpers.js
 +function cleanUrl(base, href) {
    if (base && !originIndependentUrl.test(href)) {
      href = resolveUrl(base, href);
-@@ -223,10 +210,4 @@ function findClosingBracket(str, b) {
+@@ -227,10 +214,4 @@ function findClosingBracket(str, b) {
  }
  
 -function checkSanitizeDeprecation(opt) {
@@ -179,14 +180,13 @@ diff --git a/src/helpers.js b/src/helpers.js
 -  }
 -}
 -
- module.exports = {
+ // copied from https://stackoverflow.com/a/5450113/806777
-   escape,
+ function repeatString(pattern, count) {
-@@ -239,5 +220,4 @@ module.exports = {
+@@ -260,5 +241,4 @@ module.exports = {
-   splitCells,
    rtrim,
--  findClosingBracket,
+   findClosingBracket,
--  checkSanitizeDeprecation
+-  checkSanitizeDeprecation,
-+  findClosingBracket
+   repeatString
  };
 diff --git a/src/marked.js b/src/marked.js
 --- a/src/marked.js
@@ -203,8 +203,14 @@ diff --git a/src/marked.js b/src/marked.js
 -  checkSanitizeDeprecation(opt);
  
    if (callback) {
-@@ -108,5 +106,5 @@ function marked(src, opt, callback) {
+@@ -302,5 +300,4 @@ marked.parseInline = function(src, opt) {
-     return Parser.parse(tokens, opt);
+ 
+   opt = merge({}, marked.defaults, opt || {});
+-  checkSanitizeDeprecation(opt);
+ 
+   try {
+@@ -311,5 +308,5 @@ marked.parseInline = function(src, opt) {
+     return Parser.parseInline(tokens, opt);
    } catch (e) {
 -    e.message += '\nPlease report this to https://github.com/markedjs/marked.';
 +    e.message += '\nmake issue @ https://github.com/9001/copyparty';
@@ -252,86 +258,87 @@ diff --git a/test/bench.js b/test/bench.js
 diff --git a/test/specs/run-spec.js b/test/specs/run-spec.js
 --- a/test/specs/run-spec.js
 +++ b/test/specs/run-spec.js
-@@ -22,8 +22,4 @@ function runSpecs(title, dir, showCompletionTable, options) {
+@@ -22,9 +22,4 @@ function runSpecs(title, dir, showCompletionTable, options) {
            }
  
 -          if (spec.options.sanitizer) {
 -            // eslint-disable-next-line no-eval
 -            spec.options.sanitizer = eval(spec.options.sanitizer);
 -          }
- 
+-
            (spec.only ? fit : (spec.skip ? xit : it))('should ' + passFail + example, async() => {
-@@ -53,3 +49,2 @@ runSpecs('Original', './original', false, { gfm: false, pedantic: true });
+             const before = process.hrtime();
+@@ -53,3 +48,2 @@ runSpecs('Original', './original', false, { gfm: false, pedantic: true });
  runSpecs('New', './new');
  runSpecs('ReDOS', './redos');
 -runSpecs('Security', './security', false, { silent: true }); // silent - do not show deprecation warning
 diff --git a/test/unit/Lexer-spec.js b/test/unit/Lexer-spec.js
 --- a/test/unit/Lexer-spec.js
 +++ b/test/unit/Lexer-spec.js
-@@ -465,5 +465,5 @@ a | b
+@@ -589,5 +589,5 @@ paragraph
      });
  
 -    it('sanitize', () => {
 +    /*it('sanitize', () => {
        expectTokens({
          md: '<div>html</div>',
-@@ -483,5 +483,5 @@ a | b
+@@ -607,5 +607,5 @@ paragraph
          ]
        });
 -    });
 +    });*/
    });
  
-@@ -587,5 +587,5 @@ a | b
+@@ -652,5 +652,5 @@ paragraph
        });
  
 -      it('html sanitize', () => {
 +      /*it('html sanitize', () => {
          expectInlineTokens({
            md: '<div>html</div>',
-@@ -597,5 +597,5 @@ a | b
+@@ -660,5 +660,5 @@ paragraph
            ]
          });
 -      });
 +      });*/
  
        it('link', () => {
-@@ -909,5 +909,5 @@ a | b
+@@ -971,5 +971,5 @@ paragraph
          });
  
 -        it('autolink mangle email', () => {
 +        /*it('autolink mangle email', () => {
            expectInlineTokens({
              md: '<test@example.com>',
-@@ -929,5 +929,5 @@ a | b
+@@ -991,5 +991,5 @@ paragraph
              ]
            });
 -        });
 +        });*/
  
          it('url', () => {
-@@ -966,5 +966,5 @@ a | b
+@@ -1028,5 +1028,5 @@ paragraph
          });
  
 -        it('url mangle email', () => {
 +        /*it('url mangle email', () => {
            expectInlineTokens({
              md: 'test@example.com',
-@@ -986,5 +986,5 @@ a | b
+@@ -1048,5 +1048,5 @@ paragraph
              ]
            });
 -        });
 +        });*/
        });
  
-@@ -1002,5 +1002,5 @@ a | b
+@@ -1064,5 +1064,5 @@ paragraph
        });
  
 -      describe('smartypants', () => {
 +      /*describe('smartypants', () => {
          it('single quotes', () => {
            expectInlineTokens({
-@@ -1072,5 +1072,5 @@ a | b
+@@ -1134,5 +1134,5 @@ paragraph
            });
          });
 -      });

scripts/deps-docker/zopfli.makefile

@@ -2,7 +2,7 @@ all: $(addsuffix .gz, $(wildcard *.*))
 
 %.gz: %
 	#brotli -q 11 $<
-	pigz -11 -J 34 -I 573 $<
+	pigz -11 -I 573 $<
 
 # pigz -11 -J 34 -I 100 -F < $< > $@.first
 

scripts/make-sfx.sh

@@ -16,14 +16,14 @@ help() { exec cat <<'EOF'
 #
 # `no-sh` makes just the python sfx, skips the sh/unix sfx
 #
-# `no-ogv` saves ~500k by removing the opus/vorbis audio codecs
+# `no-ogv` saves ~192k by removing the opus/vorbis audio codecs
 #   (only affects apple devices; everything else has native support)
 #
-# `no-cm` saves ~90k by removing easymde/codemirror
+# `no-cm` saves ~92k by removing easymde/codemirror
 #   (the fancy markdown editor)
 #
 # `no-fnt` saves ~9k by removing the source-code-pro font
-#   (mainly used my the markdown viewer/editor)
+#   (browsers will try to use 'Consolas' instead)
 #
 # `no-dd` saves ~2k by removing the mouse cursor
 
@@ -37,6 +37,8 @@ gtar=$(command -v gtar || command -v gnutar) || true
 	sed()  { gsed  "$@"; }
 	find() { gfind "$@"; }
 	sort() { gsort "$@"; }
+	shuf() { gshuf "$@"; }
+	nproc() { gnproc; }
 	sha1sum() { shasum "$@"; }
 	unexpand() { gunexpand "$@"; }
 	command -v grealpath >/dev/null &&
@@ -65,6 +67,7 @@ pybin=$(command -v python3 || command -v python) || {
 use_gz=
 do_sh=1
 do_py=1
+zopf=2560
 while [ ! -z "$1" ]; do
 	case $1 in
 		clean)  clean=1  ; ;;
@@ -76,6 +79,7 @@ while [ ! -z "$1" ]; do
 		no-cm)  no_cm=1  ; ;;
 		no-sh)  do_sh=   ; ;;
 		no-py)  do_py=   ; ;;
+		fast)   zopf=100 ; ;;
 		*)      help     ; ;;
 	esac
 	shift
@@ -134,7 +138,7 @@ tmpdir="$(
 	# msys2 tar is bad, make the best of it
 	echo collecting source
 	[ $clean ] && {
-		(cd .. && git archive master >tar) && tar -xf ../tar copyparty
+		(cd .. && git archive hovudstraum >tar) && tar -xf ../tar copyparty
 		(cd .. && tar -cf tar copyparty/web/deps) && tar -xf ../tar
 	}
 	[ $clean ] || {
@@ -144,6 +148,7 @@ tmpdir="$(
 }
 
 ver=
+[ -z "$repack" ] &&
 git describe --tags >/dev/null 2>/dev/null && {
 	git_ver="$(git describe --tags)";  # v0.5.5-2-gb164aa0
 	ver="$(printf '%s\n' "$git_ver" | sed -r 's/^v//')";
@@ -175,7 +180,7 @@ git describe --tags >/dev/null 2>/dev/null && {
 
 [ -z "$ver" ] && 
 	ver="$(awk '/^VERSION *= \(/ {
-		gsub(/[^0-9,]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
+		gsub(/[^0-9,a-g-]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
 
 ts=$(date -u +%s)
 hts=$(date -u +%Y-%m%d-%H%M%S) # --date=@$ts (thx osx)
@@ -201,6 +206,15 @@ while IFS= read -r x; do
 	tmv "$x"
 done
 
+find copyparty | LC_ALL=C sort | sed 's/\.gz$//;s/$/,/' > have
+cat have | while IFS= read -r x; do
+	grep -qF -- "$x" ../scripts/sfx.ls || {
+		echo "unexpected file: $x"
+		exit 1
+	}
+done
+rm have
+
 [ $no_ogv ] &&
 	rm -rf copyparty/web/deps/{dynamicaudio,ogv}*
 
@@ -214,17 +228,17 @@ done
 
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
-	f=copyparty/web/md.css
+	f=copyparty/web/ui.css
-	gzip -d "$f"
+	gzip -d "$f.gz" || true
-	sed -r '/scp\.woff2/d' <$f >t
+	sed -r "s/src:.*scp.*\)/src:local('Consolas')/" <$f >t
 	tmv "$f"
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
-	gzip -d "$f"
+	gzip -d "$f.gz" || true
-	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t
+	sed -r 's/(cursor: ?)url\([^)]+\), ?(pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: ?cursor/d' <$f >t
 	tmv "$f"
 }
 
@@ -238,6 +252,12 @@ f=dep-j2/jinja2/constants.py
 awk '/^LOREM_IPSUM_WORDS/{o=1;print "LOREM_IPSUM_WORDS = u\"a\"";next} !o; /"""/{o=0}' <$f >t
 tmv "$f"
 
+grep -rLE '^#[^a-z]*coding: utf-8' dep-j2 |
+while IFS= read -r f; do
+	(echo "# coding: utf-8"; cat "$f") >t
+	tmv "$f"
+done
+
 # up2k goes from 28k to 22k laff
 awk 'BEGIN{gensub(//,"",1)}' </dev/null &&
 echo entabbening &&
@@ -262,14 +282,27 @@ done
 
 gzres() {
 	command -v pigz &&
-		pk='pigz -11 -I 256' ||
+		pk="pigz -11 -I $zopf" ||
 		pk='gzip'
 
-	echo "$pk"
+	np=$(nproc)
-	find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
+	echo "$pk #$np"
-		echo -n .
+
-		$pk "$f"
+	while IFS=' ' read -r _ f; do
+		while true; do
+			na=$(ps auxwww | grep -F "$pk" | wc -l)
+			[ $na -le $np ] && break
+			sleep 0.2
 		done
+		echo -n .
+		$pk "$f" &
+	done < <(
+		find -printf '%s %p\n' |
+		grep -E '\.(js|css)$' |
+		grep -vF /deps/ |
+		sort -nr
+	)
+	wait
 	echo
 }
 
@@ -303,7 +336,7 @@ for d in copyparty dep-j2; do find $d -type f; done |
 sed -r 's/(.*)\.(.*)/\2 \1/' | LC_ALL=C sort |
 sed -r 's/([^ ]*) (.*)/\2.\1/' | grep -vE '/list1?$' > list1
 
-(grep -vE '\.(gz|br)$' list1; grep -E '\.(gz|br)$' list1) >list || true
+(grep -vE '\.(gz|br)$' list1; grep -E '\.(gz|br)$' list1 | shuf) >list || true
 
 echo creating tar
 args=(--owner=1000 --group=1000)

scripts/make-tgz-release.sh

@@ -61,7 +61,7 @@ rls_dir="$tmp/copyparty-$ver"
 mkdir "$rls_dir"
 
 echo ">>> export from git"
-git archive master | tar -xC "$rls_dir"
+git archive hovudstraum | tar -xC "$rls_dir"
 
 echo ">>> export untracked deps"
 tar -c copyparty/web/deps | tar -xC "$rls_dir"
@@ -122,5 +122,5 @@ echo "  $zip_path"
 echo "  $tgz_path"
 echo
 
-# function alr() { ls -alR copyparty-$1 | sed -r "s/copyparty-$1/copyparty/" | sed -r 's/[A-Z][a-z]{2} [0-9 ]{2} [0-9]{2}:[0-9]{2}//' > $1; }; for x in master rls src ; do alr $x; done
+# function alr() { ls -alR copyparty-$1 | sed -r "s/copyparty-$1/copyparty/" | sed -r 's/[A-Z][a-z]{2} [0-9 ]{2} [0-9]{2}:[0-9]{2}//' > $1; }; for x in hovudstraum rls src ; do alr $x; done
 

scripts/rls.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+set -e
+
+cd ~/dev/copyparty/scripts
+
+v=$1
+printf '%s\n' "$v" | grep -qE '^[0-9\.]+$' || exit 1
+grep -E "(${v//./, })" ../copyparty/__version__.py || exit 1
+
+git tag v$v
+git push origin --tags
+
+rm -rf ../dist
+
+./make-pypi-release.sh u
+(cd .. && python3 ./setup.py clean2)
+
+./make-tgz-release.sh $v
+
+rm -f ../dist/copyparty-sfx.*
+./make-sfx.sh no-sh
+../dist/copyparty-sfx.py -h
+
+ar=
+while true; do
+    for ((a=0; a<100; a++)); do
+        for f in ../dist/copyparty-sfx.{py,sh}; do
+            [ -e $f ] || continue;
+            mv $f $f.$(wc -c <$f | awk '{print$1}')
+        done
+        ./make-sfx.sh re $ar
+    done
+    ar=no-sh
+done
+
+# git tag -d v$v; git push --delete origin v$v

scripts/sfx.ls

@@ -0,0 +1,77 @@
+copyparty,
+copyparty/__init__.py,
+copyparty/__main__.py,
+copyparty/__version__.py,
+copyparty/authsrv.py,
+copyparty/bos,
+copyparty/bos/__init__.py,
+copyparty/bos/bos.py,
+copyparty/bos/path.py,
+copyparty/broker_mp.py,
+copyparty/broker_mpw.py,
+copyparty/broker_thr.py,
+copyparty/broker_util.py,
+copyparty/httpcli.py,
+copyparty/httpconn.py,
+copyparty/httpsrv.py,
+copyparty/ico.py,
+copyparty/mtag.py,
+copyparty/res,
+copyparty/res/insecure.pem,
+copyparty/star.py,
+copyparty/stolen,
+copyparty/stolen/__init__.py,
+copyparty/stolen/surrogateescape.py,
+copyparty/sutil.py,
+copyparty/svchub.py,
+copyparty/szip.py,
+copyparty/tcpsrv.py,
+copyparty/th_cli.py,
+copyparty/th_srv.py,
+copyparty/u2idx.py,
+copyparty/up2k.py,
+copyparty/util.py,
+copyparty/web,
+copyparty/web/baguettebox.js,
+copyparty/web/browser.css,
+copyparty/web/browser.html,
+copyparty/web/browser.js,
+copyparty/web/browser2.html,
+copyparty/web/copyparty.gif,
+copyparty/web/dd,
+copyparty/web/dd/2.png,
+copyparty/web/dd/3.png,
+copyparty/web/dd/4.png,
+copyparty/web/dd/5.png,
+copyparty/web/deps,
+copyparty/web/deps/easymde.css,
+copyparty/web/deps/easymde.js,
+copyparty/web/deps/marked.js,
+copyparty/web/deps/mini-fa.css,
+copyparty/web/deps/mini-fa.woff,
+copyparty/web/deps/ogv-decoder-audio-opus-wasm.js,
+copyparty/web/deps/ogv-decoder-audio-opus-wasm.wasm,
+copyparty/web/deps/ogv-decoder-audio-vorbis-wasm.js,
+copyparty/web/deps/ogv-decoder-audio-vorbis-wasm.wasm,
+copyparty/web/deps/ogv-demuxer-ogg-wasm.js,
+copyparty/web/deps/ogv-demuxer-ogg-wasm.wasm,
+copyparty/web/deps/ogv-worker-audio.js,
+copyparty/web/deps/ogv.js,
+copyparty/web/deps/scp.woff2,
+copyparty/web/deps/sha512.ac.js,
+copyparty/web/deps/sha512.hw.js,
+copyparty/web/md.css,
+copyparty/web/md.html,
+copyparty/web/md.js,
+copyparty/web/md2.css,
+copyparty/web/md2.js,
+copyparty/web/mde.css,
+copyparty/web/mde.html,
+copyparty/web/mde.js,
+copyparty/web/msg.css,
+copyparty/web/msg.html,
+copyparty/web/splash.css,
+copyparty/web/splash.html,
+copyparty/web/ui.css,
+copyparty/web/up2k.js,
+copyparty/web/util.js,

scripts/sfx.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: latin-1
 from __future__ import print_function, unicode_literals
 

scripts/speedtest-fs.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 import os
 import sys

scripts/test/smoketest.py

@@ -60,7 +60,7 @@ class Cpp(object):
                 pass
 
 
-def tc1():
+def tc1(vflags):
     ub = "http://127.0.0.1:4321/"
     td = os.path.join("srv", "smoketest")
     try:
@@ -100,16 +100,17 @@ def tc1():
     for d1 in ["r", "w", "a"]:
         pdirs.append("{}/{}".format(td, d1))
         pdirs.append("{}/{}/j".format(td, d1))
-        for d2 in ["r", "w", "a"]:
+        for d2 in ["r", "w", "a", "c"]:
             d = os.path.join(td, d1, "j", d2)
             pdirs.append(d)
             os.makedirs(d)
 
     pdirs = [x.replace("\\", "/") for x in pdirs]
     udirs = [x.split("/", 2)[2] for x in pdirs]
-    perms = [x.rstrip("j/")[-1] for x in pdirs]
+    perms = [x.rstrip("cj/")[-1] for x in pdirs]
+    perms = ["rw" if x == "a" else x for x in perms]
     for pd, ud, p in zip(pdirs, udirs, perms):
-        if ud[-1] == "j":
+        if ud[-1] == "j" or ud[-1] == "c":
             continue
 
         hp = None
@@ -122,47 +123,55 @@ def tc1():
             hp = "-"
             hpaths[ud] = os.path.join(pd, ".hist")
 
-        arg = "{}:{}:{}".format(pd, ud, p, hp)
+        arg = "{}:{}:{}".format(pd, ud, p)
         if hp:
             arg += ":c,hist=" + hp
 
-        args += ["-v", arg]
+        args += ["-v", arg + vflags]
 
     # return
     cpp = Cpp(args)
     CPP.append(cpp)
     cpp.await_idle(ub, 3)
 
-    for d in udirs:
+    for d, p in zip(udirs, perms):
         vid = ovid + "\n{}".format(d).encode("utf-8")
-        try:
+        r = requests.post(
-            requests.post(ub + d, data={"act": "bput"}, files={"f": ("a.h264", vid)})
+            ub + d,
-        except:
+            data={"act": "bput"},
-            pass
+            files={"f": (d.replace("/", "") + ".h264", vid)},
+        )
+        c = r.status_code
+        if c == 200 and p not in ["w", "rw"]:
+            raise Exception("post {} with perm {} at {}".format(c, p, d))
+        elif c == 403 and p not in ["r"]:
+            raise Exception("post {} with perm {} at {}".format(c, p, d))
+        elif c not in [200, 403]:
+            raise Exception("post {} with perm {} at {}".format(c, p, d))
 
     cpp.clean()
 
     # GET permission
     for d, p in zip(udirs, perms):
-        u = "{}{}/a.h264".format(ub, d)
+        u = "{}{}/{}.h264".format(ub, d, d.replace("/", ""))
         r = requests.get(u)
         ok = bool(r)
-        if ok != (p in ["a"]):
+        if ok != (p in ["rw"]):
             raise Exception("get {} with perm {} at {}".format(ok, p, u))
 
     # stat filesystem
     for d, p in zip(pdirs, perms):
-        u = "{}/a.h264".format(d)
+        u = "{}/{}.h264".format(d, d.split("test/")[-1].replace("/", ""))
         ok = os.path.exists(u)
-        if ok != (p in ["a", "w"]):
+        if ok != (p in ["rw", "w"]):
             raise Exception("stat {} with perm {} at {}".format(ok, p, u))
 
     # GET thumbnail, vreify contents
     for d, p in zip(udirs, perms):
-        u = "{}{}/a.h264?th=j".format(ub, d)
+        u = "{}{}/{}.h264?th=j".format(ub, d, d.replace("/", ""))
         r = requests.get(u)
         ok = bool(r and r.content[:3] == b"\xff\xd8\xff")
-        if ok != (p in ["a"]):
+        if ok != (p in ["rw"]):
             raise Exception("thumb {} with perm {} at {}".format(ok, p, u))
 
     # check tags
@@ -179,10 +188,10 @@ def tc1():
         r_ok = bool(j)
         w_ok = bool(r_ok and j.get("files"))
 
-        if not r_ok or w_ok != (p in ["a"]):
+        if not r_ok or w_ok != (p in ["rw"]):
             raise Exception("ls {} with perm {} at {}".format(ok, p, u))
 
-        if (tag and p != "a") or (not tag and p == "a"):
+        if (tag and p != "rw") or (not tag and p == "rw"):
             raise Exception("tag {} with perm {} at {}".format(tag, p, u))
 
         if tag is not None and tag != "48x32":
@@ -191,9 +200,9 @@ def tc1():
     cpp.stop(True)
 
 
-def run(tc):
+def run(tc, *a):
     try:
-        tc()
+        tc(*a)
     finally:
         try:
             CPP[0].stop(False)
@@ -202,7 +211,8 @@ def run(tc):
 
 
 def main():
-    run(tc1)
+    run(tc1, "")
+    run(tc1, ":c,fk")
 
 
 if __name__ == "__main__":

scripts/uncomment.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
@@ -8,7 +8,7 @@ import tokenize
 
 
 def uncomment(fpath):
-    """ modified https://stackoverflow.com/a/62074206 """
+    """modified https://stackoverflow.com/a/62074206"""
 
     with open(fpath, "rb") as f:
         orig = f.read().decode("utf-8")

setup.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import print_function
 
@@ -61,7 +61,7 @@ class clean2(Command):
             pass
 
         nuke = []
-        for (dirpath, dirnames, filenames) in os.walk("."):
+        for (dirpath, _, filenames) in os.walk("."):
             for fn in filenames:
                 if (
                     fn.startswith("MANIFEST")
@@ -86,7 +86,7 @@ args = {
     "url": "https://github.com/9001/copyparty",
     "license": "MIT",
     "classifiers": [
-        "Development Status :: 4 - Beta",
+        "Development Status :: 5 - Production/Stable",
         "License :: OSI Approved :: MIT License",
         "Programming Language :: Python",
         "Programming Language :: Python :: 2",
@@ -101,6 +101,7 @@ args = {
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: Implementation :: CPython",
+        "Programming Language :: Python :: Implementation :: Jython",
         "Programming Language :: Python :: Implementation :: PyPy",
         "Environment :: Console",
         "Environment :: No Input/Output (Daemon)",
@@ -113,7 +114,7 @@ args = {
     "install_requires": ["jinja2"],
     "extras_require": {"thumbnails": ["Pillow"], "audiotags": ["mutagen"]},
     "entry_points": {"console_scripts": ["copyparty = copyparty.__main__:main"]},
-    "scripts": ["bin/copyparty-fuse.py"],
+    "scripts": ["bin/copyparty-fuse.py", "bin/up2k.py"],
     "cmdclass": {"clean2": clean2},
 }
 

srv/test.md

@@ -1,11 +1,17 @@
 ### hello world
 
 * qwe
-* asd
+* rty
+* uio
+  * asd
+  * fgh
+  * jkl
     * zxc
+    * vbn
     * 573
       * one
       * two
+      * three
     
   * |||
     |--|--|
@@ -134,12 +140,12 @@ a newline toplevel
 | a table | on the right |
 | second row | foo bar |
 
-||
+a||a
 --|:-:|-:
 a table | big text in this | aaakbfddd
 second row | centred | bbb
 
-||
+||||
 --|--|--
 foo
 

tests/test_httpcli.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
@@ -39,6 +39,8 @@ class Cfg(Namespace):
             no_scandir=False,
             no_sendfile=True,
             no_rescan=True,
+            no_logues=False,
+            no_readme=False,
             re_maxage=0,
             ihead=False,
             nih=True,
@@ -96,7 +98,7 @@ class TestHttpCli(unittest.TestCase):
                 if not vol.startswith(top):
                     continue
 
-                mode = vol[-2].replace("a", "rwmd")
+                mode = vol[-2].replace("a", "rw")
                 usr = vol[-1]
                 if usr == "a":
                     usr = ""
@@ -151,6 +153,7 @@ class TestHttpCli(unittest.TestCase):
                     tar = tarfile.open(fileobj=io.BytesIO(b)).getnames()
                 except:
                     tar = []
+                tar = [x[4:] if x.startswith("top/") else x for x in tar]
                 tar = ["/".join([y for y in [top, durl, x] if y]) for x in tar]
                 tar = [[x] + self.can_rw(x) for x in tar]
                 tar_ok = [x[0] for x in tar if x[1]]

tests/test_vfs.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
@@ -26,6 +26,8 @@ class Cfg(Namespace):
             "no_hash": False,
             "css_browser": None,
             "no_voldump": True,
+            "no_logues": False,
+            "no_readme": False,
             "re_maxage": 0,
             "rproxy": 0,
         }
@@ -195,10 +197,10 @@ class TestVFS(unittest.TestCase):
         self.assertEqual(n.realpath, os.path.join(td, "a"))
         self.assertAxs(n.axs.uread, ["*"])
         self.assertAxs(n.axs.uwrite, [])
-        self.assertEqual(vfs.can_access("/", "*"), [False, False, False, False])
+        self.assertEqual(vfs.can_access("/", "*"), [False, False, False, False, False])
-        self.assertEqual(vfs.can_access("/", "k"), [True, True, False, False])
+        self.assertEqual(vfs.can_access("/", "k"), [True, True, False, False, False])
-        self.assertEqual(vfs.can_access("/a", "*"), [True, False, False, False])
+        self.assertEqual(vfs.can_access("/a", "*"), [True, False, False, False, False])
-        self.assertEqual(vfs.can_access("/a", "k"), [True, False, False, False])
+        self.assertEqual(vfs.can_access("/a", "k"), [True, False, False, False, False])
 
         # breadth-first construction
         vfs = AuthSrv(

tests/util.py

@@ -3,6 +3,7 @@ import sys
 import time
 import shutil
 import jinja2
+import threading
 import tempfile
 import platform
 import subprocess as sp
@@ -28,7 +29,7 @@ if MACOS:
     # 25% faster; until any tests do symlink stuff
 
 
-from copyparty.util import Unrecv
+from copyparty.util import Unrecv, FHC
 
 
 def runcmd(argv):
@@ -132,6 +133,8 @@ class VHttpConn(object):
         self.log_src = "a"
         self.lf_url = None
         self.hsrv = VHttpSrv()
+        self.u2fh = FHC()
+        self.mutex = threading.Lock()
         self.nreq = 0
         self.nbyte = 0
         self.ico = None