v0.10.22

README.md

@@ -10,7 +10,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 
 * server runs on anything with `py2.7` or `py3.3+`
 * browse/upload with IE4 / netscape4.0 on win3.11 (heh)
-* *resumable* uploads need `firefox 34+` / `chrome 37+` / `safari 7+`
+* *resumable* uploads need `firefox 34+` / `chrome 41+` / `safari 7+` for full speed
 * code standard: `black`
 
 📷 screenshots: [browser](#the-browser) // [upload](#uploading) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
@@ -23,6 +23,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
     * [notes](#notes)
     * [status](#status)
 * [bugs](#bugs)
+    * [general bugs](#general-bugs)
     * [not my bugs](#not-my-bugs)
 * [the browser](#the-browser)
     * [tabs](#tabs)
@@ -55,11 +56,11 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 
 download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
 
-running the sfx without arguments (for example doubleclicking it on Windows) will let anyone access the current folder; see `-h` for help if you want accounts and volumes etc
+running the sfx without arguments (for example doubleclicking it on Windows) will give everyone full access to the current folder; see `-h` for help if you want accounts and volumes etc
 
 you may also want these, especially on servers:
 * [contrib/systemd/copyparty.service](contrib/systemd/copyparty.service) to run copyparty as a systemd service
-* [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to reverse-proxy behind nginx (for legit https)
+* [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to reverse-proxy behind nginx (for better https)
 
 
 ## notes
@@ -92,8 +93,8 @@ you may also want these, especially on servers:
   * ☑ tree-view
   * ☑ media player
   * ✖ thumbnails
-  * ✖ SPA (browse while uploading)
-    * currently safe using the file-tree on the left only, not folders in the file list
+  * ☑ SPA (browse while uploading)
+    * if you use the file-tree on the left only, not folders in the file list
 * server indexing
   * ☑ locate files by contents
   * ☑ search by name/path/date/size
@@ -110,6 +111,11 @@ summary: it works! you can use it! (but technically not even close to beta)
 * Windows: python 3.7 and older cannot read tags with ffprobe, so use mutagen or upgrade
 * Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
 * Windows: python 2.7 cannot handle filenames with mojibake
+
+## general bugs
+
+* all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
+* cannot mount something at `/d1/d2/d3` unless `d2` exists inside `d1`
 * hiding the contents at url `/d1/d2/d3` using `-v :d1/d2/d3:cd2d` has the side-effect of creating databases (for files/tags) inside folders d1 and d2, and those databases take precedence over the main db at the top of the vfs - this means all files in d2 and below will be reindexed unless you already had a vfs entry at or below d2
 * probably more, pls let me know
 
@@ -176,8 +182,8 @@ you can also zip a selection of files or folders by clicking them in the browser
 ## uploading
 
 two upload methods are available in the html client:
-* 🎈 bup, the basic uploader, supports almost every browser since netscape 4.0
-* 🚀 up2k, the fancy one
+* `🎈 bup`, the basic uploader, supports almost every browser since netscape 4.0
+* `🚀 up2k`, the fancy one
 
 up2k has several advantages:
 * you can drop folders into the browser (files are added recursively)
@@ -212,14 +218,14 @@ and then theres the tabs below it,
 
 ![copyparty-fsearch-fs8](https://user-images.githubusercontent.com/241032/116008320-36919780-a614-11eb-803f-04162326a700.png)
 
-in the 🚀 up2k tab, after toggling the `[🔎]` switch green, any files/folders you drop onto the dropzone will be hashed on the client-side. Each hash is sent to the server which checks if that file exists somewhere already
+in the `[🚀 up2k]` tab, after toggling the `[🔎]` switch green, any files/folders you drop onto the dropzone will be hashed on the client-side. Each hash is sent to the server which checks if that file exists somewhere already
 
 files go into `[ok]` if they exist (and you get a link to where it is), otherwise they land in `[ng]`
-* the main reason filesearch is combined with the uploader is cause the code was too spaghetti to separate it out somewhere else
+* the main reason filesearch is combined with the uploader is cause the code was too spaghetti to separate it out somewhere else, this is no longer the case but now i've warmed up to the idea too much
 
 adding the same file multiple times is blocked, so if you first search for a file and then decide to upload it, you have to click the `[cleanup]` button to discard `[done]` files
 
-note that since up2k has to read the file twice, 🎈 bup can be up to 2x faster if your internet connection is faster than the read-speed of your HDD
+note that since up2k has to read the file twice, `[🎈 bup]` can be up to 2x faster in extreme cases (if your internet connection is faster than the read-speed of your HDD)
 
 up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well thanks to tls also functioning as an integrity check
 
@@ -299,7 +305,7 @@ copyparty can invoke external programs to collect additional metadata for files
 * `-mtp key=f,t5,~/bin/audio-key.py` uses `~/bin/audio-key.py` to get the `key` tag, replacing any existing metadata tag (`f,`), aborting if it takes longer than 5sec (`t5,`)
 * `-v ~/music::r:cmtp=.bpm=~/bin/audio-bpm.py:cmtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
 
-*but wait, there's more!* `-mtp` can be used for non-audio files as well using the `a` flag: `ay` only do audio files, `an` audio files are skipped, or `ad` always do it (d as in dontcare) 
+*but wait, there's more!* `-mtp` can be used for non-audio files as well using the `a` flag: `ay` only do audio files, `an` only do non-audio files, or `ad` do all files (d as in dontcare) 
 
 * `-mtp ext=an,~/bin/file-ext.py` runs `~/bin/file-ext.py` to get the `ext` tag only if file is not audio (`an`)
 
@@ -338,14 +344,18 @@ copyparty can invoke external programs to collect additional metadata for files
 * `*2` using a wasm decoder which can sometimes get stuck and consumes a bit more power
 
 quick summary of more eccentric web-browsers trying to view a directory index:
-* safari (14.0.3/macos) is chrome with janky wasm, so playing opus can deadlock the javascript engine
-* safari (14.0.1/iOS) same as macos, except it recovers from the deadlocks if you poke it a bit
-* links (2.21/macports) can browse, login, upload/mkdir/msg
-* lynx (2.8.9/macports) can browse, login, upload/mkdir/msg
-* w3m (0.5.3/macports) can browse, login, upload at 100kB/s, mkdir/msg
-* netsurf (3.10/arch) is basically ie6 with much better css (javascript has almost no effect)
-* ie4 and netscape 4.0 can browse (text is yellow on white), upload with `?b=u`
-* SerenityOS (22d13d8) hits a page fault, works with `?b=u`, file input not-impl, url params are multiplying
+
+| browser | will it blend |
+| ------- | ------------- |
+| **safari** (14.0.3/macos) | is chrome with janky wasm, so playing opus can deadlock the javascript engine |
+| **safari** (14.0.1/iOS)   | same as macos, except it recovers from the deadlocks if you poke it a bit |
+| **links** (2.21/macports) | can browse, login, upload/mkdir/msg |
+| **lynx** (2.8.9/macports) | can browse, login, upload/mkdir/msg |
+| **w3m** (0.5.3/macports)  | can browse, login, upload at 100kB/s, mkdir/msg |
+| **netsurf** (3.10/arch)   | is basically ie6 with much better css (javascript has almost no effect) | 
+| **ie4** and **netscape** 4.0  | can browse (text is yellow on white), upload with `?b=u` |
+| **SerenityOS** (22d13d8)  | hits a page fault, works with `?b=u`, file input not-impl, url params are multiplying |
+
 
 # client examples
 
@@ -376,7 +386,7 @@ copyparty returns a truncated sha512sum of your PUT/POST as base64; you can gene
 quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
 * the up2k client splits a file into an "optimal" number of chunks
   * 1 MiB each, unless that becomes more than 256 chunks
-  * tries 1.5M, 2M, 3, 4, 6, ... until <= 256# or chunksize >= 32M
+  * tries 1.5M, 2M, 3, 4, 6, ... until <= 256 chunks or size >= 32M
 * client posts the list of hashes, filename, size, last-modified
 * server creates the `wark`, an identifier for this upload
   * `sha512( salt + filesize + chunk_hashes )`
@@ -409,8 +419,8 @@ these are standalone programs and will never be imported / evaluated by copypart
 # sfx
 
 currently there are two self-contained "binaries":
-* [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) -- pure python, works everywhere
-* [copyparty-sfx.sh](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.sh) -- smaller, but only for linux and macos
+* [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) -- pure python, works everywhere, **recommended**
+* [copyparty-sfx.sh](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.sh) -- smaller, but only for linux and macos, kinda deprecated
 
 launch either of them (**use sfx.py on systemd**) and it'll unpack and run copyparty, assuming you have python installed of course
 
@@ -478,6 +488,7 @@ roughly sorted by priority
   * terminate client on bad data
 * `os.copy_file_range` for up2k cloning
 * support pillow-simd
+* single sha512 across all up2k chunks? maybe
 * figure out the deal with pixel3a not being connectable as hotspot
   * pixel3a having unpredictable 3sec latency in general :||||
 

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 10, 20)
+VERSION = (0, 10, 22)
 CODENAME = "zip it"
-BUILD_DT = (2021, 5, 16)
+BUILD_DT = (2021, 5, 18)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/up2k.py

@@ -675,11 +675,16 @@ class Up2k(object):
             cur.close()
 
     def _get_parsers(self, ptop, have):
+        try:
+            all_parsers = self.mtp_parsers[ptop]
+        except:
+            return {}
+
         audio = self.mtp_audio[ptop]
         force = self.mtp_force[ptop]
         entags = self.entags[ptop]
         parsers = {}
-        for k, v in self.mtp_parsers[ptop].items():
+        for k, v in all_parsers.items():
             if ".dur" in entags:
                 if ".dur" in have:
                     # is audio, require non-audio?

copyparty/web/browser.css

@@ -281,29 +281,48 @@ a, #files tbody div a:last-child {
 	padding: .2em 0 0 .07em;
 	color: #fff;
 }
-#wzip {
+#wzip, #wnp {
 	display: none;
 	margin-right: .3em;
 	padding-right: .3em;
 	border-right: .1em solid #555;
 }
+#wnp a {
+	position: relative;
+	font-size: .47em;
+	margin: 0 .1em;
+	top: -.4em;
+}
+#wnp a+a {
+	margin-left: .33em;
+}
 #wtoggle,
 #wtoggle * {
 	line-height: 1em;
 }
+#wtoggle.np {
+	width: 5.5em;
+}
 #wtoggle.sel {
 	width: 6.4em;
 }
-#wtoggle.sel #wzip {
+#wtoggle.sel #wzip,
+#wtoggle.np #wnp {
 	display: inline-block;
 }
-#wtoggle.sel #wzip a {
+#wtoggle.sel.np #wnp {
+	display: none;
+}
+#wzip a {
 	font-size: .4em;
 	padding: 0 .3em;
 	margin: -.3em .2em;
 	position: relative;
 	display: inline-block;
 }
+#wzip a+a {
+	margin-left: .8em;
+}
 #wtoggle.sel #wzip #selzip {
 	top: -.6em;
 	padding: .4em .3em;
@@ -508,7 +527,6 @@ input[type="checkbox"]:checked+label {
 	left: 0;
 	bottom: 0;
 	top: 7em;
-	padding-top: .2em;
 	overflow-y: auto;
 	-ms-scroll-chaining: none;
 	overscroll-behavior-y: none;
@@ -556,6 +574,7 @@ input[type="checkbox"]:checked+label {
 #detree {
 	padding: .3em .5em;
 	font-size: 1.5em;
+	line-height: 1.5em;
 }
 #tree ul,
 #tree li {

copyparty/web/browser.html

@@ -114,22 +114,7 @@
     <div id="srv_info"><span>{{ srv_info }}</span></div>
     {%- endif %}
 
-    <div id="widget">
-        <div id="wtoggle">
-            <span id="wzip">
-                <a href="#" id="selall">sel.<br />all</a>
-                <a href="#" id="selinv">sel.<br />inv.</a>
-                <a href="#" id="selzip">zip</a>
-            </span><a
-                href="#" id="wtico">♫</a>
-        </div>
-        <div id="widgeti">
-            <div id="pctl"><a href="#" id="bprev">⏮</a><a href="#" id="bplay">▶</a><a href="#" id="bnext">⏭</a></div>
-            <canvas id="pvol" width="288" height="38"></canvas>
-            <canvas id="barpos"></canvas>
-            <canvas id="barbuf"></canvas>
-        </div>
-    </div>
+    <div id="widget"></div>
 
     <script>
         var tag_order_cfg = {{ tag_order }};

copyparty/web/browser.js

@@ -7,6 +7,28 @@ function dbg(msg) {
 }
 
 
+// add widget buttons
+ebi('widget').innerHTML = (
+	'<div id="wtoggle">' +
+	'<span id="wzip"><a' +
+	' href="#" id="selall">sel.<br />all</a><a' +
+	' href="#" id="selinv">sel.<br />inv.</a><a' +
+	' href="#" id="selzip">zip</a>' +
+	'</span><span id="wnp"><a' +
+	' href="#" id="npirc">📋irc</a><a' +
+	' href="#" id="nptxt">📋txt</a>' +
+	'</span><a' +
+	'	href="#" id="wtico">♫</a>' +
+	'</div>' +
+	'<div id="widgeti">' +
+	'	<div id="pctl"><a href="#" id="bprev">⏮</a><a href="#" id="bplay">▶</a><a href="#" id="bnext">⏭</a></div>' +
+	'	<canvas id="pvol" width="288" height="38"></canvas>' +
+	'	<canvas id="barpos"></canvas>' +
+	'	<canvas id="barbuf"></canvas>' +
+	'</div>'
+);
+
+
 // extract songs + add play column
 function MPlayer() {
 	this.id = Date.now();
@@ -79,6 +101,8 @@ var widget = (function () {
 	var ret = {},
 		widget = ebi('widget'),
 		wtico = ebi('wtico'),
+		nptxt = ebi('nptxt'),
+		npirc = ebi('npirc'),
 		touchmode = false,
 		side_open = false,
 		was_paused = true;
@@ -116,6 +140,35 @@ var widget = (function () {
 
 		return false;
 	};
+	npirc.onclick = nptxt.onclick = function (e) {
+		ev(e);
+		var th = ebi('files').tHead.rows[0].cells,
+			tr = QS('#files tr.play').cells,
+			irc = this.getAttribute('id') == 'npirc',
+			ck = irc ? '06' : '',
+			cv = irc ? '07' : '',
+			m = ck + 'np: ';
+
+		for (var a = 1, aa = th.length; a < aa; a++) {
+			var tk = a == 1 ? '' : th[a].getAttribute('name').split('/').slice(-1)[0];
+			var tv = tr[a].getAttribute('html') || tr[a].textContent;
+			m += tk + '(' + cv + tv + ck + ') // ';
+		}
+
+		m += '[' + cv + s2ms(mp.au.currentTime) + ck + '/' + cv + s2ms(mp.au.duration) + ck + ']';
+
+		var o = document.createElement('input');
+		o.style.cssText = 'position:fixed;top:45%;left:48%;padding:1em;z-index:9';
+		o.value = m;
+		document.body.appendChild(o);
+		o.focus();
+		o.select();
+		document.execCommand("copy");
+		o.value = 'copied to clipboard ';
+		setTimeout(function () {
+			document.body.removeChild(o);
+		}, 500);
+	};
 	return ret;
 })();
 
@@ -189,8 +242,8 @@ var pbar = (function () {
 	};
 
 	r.drawpos = function () {
-		if (!mp.au)
-			return;
+		if (!mp.au || isNaN(mp.au.duration) || isNaN(mp.au.currentTime))
+			return;  // not-init || unsupp-codec
 
 		var bc = r.buf,
 			pc = r.pos,
@@ -513,6 +566,7 @@ function play(tid, seek, call_depth) {
 		clmod(trs[a], 'play');
 	}
 	ebi(oid).parentElement.parentElement.className += ' play';
+	clmod(ebi('wtoggle'), 'np', 1);
 
 	try {
 		if (attempt_play)
@@ -928,7 +982,6 @@ var treectl = (function () {
 		treesz = icfg_get('treesz', 16);
 
 	treesz = Math.min(Math.max(treesz, 4), 50);
-	console.log('treesz [' + treesz + ']');
 
 	function entree(e) {
 		ev(e);
@@ -987,7 +1040,7 @@ var treectl = (function () {
 		}
 		else {
 			var top = Math.max(0, parseInt(wrap.offsetTop)),
-				treeh = (winh - atop) - 4;
+				treeh = winh - atop;
 
 			tree.style.top = top + 'px';
 			tree.style.height = treeh < 10 ? '' : treeh + 'px';

copyparty/web/up2k.js

@@ -20,11 +20,12 @@ var up2k = null;
 try {
     var cf = crypto.subtle || crypto.webkitSubtle;
     cf.digest('SHA-512', new Uint8Array(1)).then(
-        function (x) { up2k = up2k_init(cf) },
-        function (x) { up2k = up2k_init(false) }
+        function (x) { console.log('sha-ok'); up2k = up2k_init(cf); },
+        function (x) { console.log('sha-ng:', x); up2k = up2k_init(false); }
     );
 }
 catch (ex) {
+    console.log('sha-na:', ex);
     try {
         up2k = up2k_init(false);
     }

docs/minimal-up2k.html

@@ -7,7 +7,7 @@
 
     /* make the up2k ui REALLY minimal by hiding a bunch of stuff: */
 
-    #ops, #tree, #path, #wrap>h2:last-child  /* main tabs and navigators (tree/breadcrumbs) */
+    #ops, #tree, #path, #wrap>h2:last-child,  /* main tabs and navigators (tree/breadcrumbs) */
 
     #u2cleanup, #u2conf tr:first-child>td[rowspan]:not(#u2btn_cw),  /* most of the config options */